Xingjun.Wang 210ab40c54 Upgrade datasets (#921) ...

* del _datasets_server import in hf_dataset_util

* fix streaming for youku-mplug and adopt latest datasets

* fix download config copy

* update ut

* add youku in test_general_datasets

* update UT for general dataset

* adapt to datasets version: 2.19.0 or later

* add assert for youku data UT

* fix disable_tqdm in some functions for 2.19.0 or later

* update get_module_with_script

* set trust_remote_code is True in load_dataset_with_ctx

* update print info

* update requirements for datasets version restriction

* fix _dataset_info

* add pillow

* update comments

* update comment

* reuse _download function in DataDownloadManager

* remove unused code

* update test_run_modelhub in Human3DAnimationTest

* set datasets>=2.18.0

2024-07-23 22:26:12 +08:00

..

cli

add donwload command line and local_dir parameter (#866)

2024-05-25 14:21:55 +08:00

export

fix ci compatible issues，fix llmpipeline lazy import issue (#725)

2024-01-17 22:19:05 +08:00

fileio

fix jsonplus, support ndarray

2023-03-21 18:10:10 +08:00

hub

add dataset download (#906)

2024-07-12 19:06:44 +08:00

models

new Feat/0817 (#504)

2023-08-29 16:43:36 +08:00

msdatasets

Upgrade datasets (#921)

2024-07-23 22:26:12 +08:00

outputs

[to #42322933]fix bugs relate to token cls

2022-10-31 20:52:27 +08:00

pipelines

Upgrade datasets (#921)

2024-07-23 22:26:12 +08:00

preprocessors

skip huggingface download cases

2023-09-15 15:26:18 +08:00

taskdataset

Refactor the task_datasets module

2023-03-10 09:03:32 +08:00

test_metrics

add self supervised depth completion. (#711)

2024-02-22 22:30:48 +08:00

trainers

fix issue #845 (#861)

2024-05-23 20:34:52 +08:00

utils

fix huggingface position_ids compatible issue

2023-10-24 15:18:55 +08:00

__init__.py

[to #41402703] add basic modules

2022-05-17 10:15:00 +08:00

json_call_test.py

fix issue #845 (#861)

2024-05-23 20:34:52 +08:00

run_analysis.py

fix issue #845 (#861)

2024-05-23 20:34:52 +08:00

run_config.yaml

add PASD image super-resolution for facechain and more Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13924655

2023-09-07 16:53:54 +08:00

run.py

fix ci compatible issues，fix llmpipeline lazy import issue (#725)

2024-01-17 22:19:05 +08:00