6 Commits

Author	SHA1	Message	Date
Xingjun.Wang	71a54c1a51	support latest datasets version (#1163)	2025-01-02 17:06:46 +08:00
Xingjun.Wang	134fe72f06	hotfix for datasets 3.0.2 (#1046)	2024-10-23 16:25:53 +08:00
Xingjun.Wang	2c4505e13a	Adapt new datasets (#1002) ... * update ds==3.0 * update * add http_get_ms func * del unused code * fix pr issue and update requirements	2024-09-30 16:46:00 +08:00
tastelikefeet	4c518db424	patch hf hub (#987)	2024-09-14 12:24:19 +08:00
Xingjun.Wang	210ab40c54	Upgrade datasets (#921) ... * del _datasets_server import in hf_dataset_util * fix streaming for youku-mplug and adopt latest datasets * fix download config copy * update ut * add youku in test_general_datasets * update UT for general dataset * adapt to datasets version: 2.19.0 or later * add assert for youku data UT * fix disable_tqdm in some functions for 2.19.0 or later * update get_module_with_script * set trust_remote_code is True in load_dataset_with_ctx * update print info * update requirements for datasets version restriction * fix _dataset_info * add pillow * update comments * update comment * reuse _download function in DataDownloadManager * remove unused code * update test_run_modelhub in Human3DAnimationTest * set datasets>=2.18.0	2024-07-23 22:26:12 +08:00
liuyhwangyh	f7a32e48e3	add dataset download (#906) ... * add dataset download * fix cr issue * fix cv matplotlib issue * refactor code * fix ut issue * remove debug code * remove unused import * fix import issue * sleep 65s before start docker avoid kill and run failed --------- Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>	2024-07-12 19:06:44 +08:00