Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2026-02-24 20:19:51 +01:00
Code Issues Packages Projects Releases Wiki Activity
2,423 Commits 267 Branches 107 Tags
dev/dataset_upgrade
Commit Graph

2423 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
xingjun.wang
929d7dc8e6 ok Merge branch 'master' of github.com:modelscope/modelscope into dev/dataset_upgrade 2024-07-23 14:27:51 +08:00
xingjun.wang
04ebb03acc set datasets>=2.18.0 2024-07-23 14:27:41 +08:00
xingjun.wang
56b0219135 update test_run_modelhub in Human3DAnimationTest 2024-07-22 22:22:40 +08:00
liuyhwangyh
4e2555c5a3 Release/1.16 merge to master (#922) 
* update version

* merge master

* fix fairseq issue

* fix pip compatible issue

* fix bug

* version to 1.16.1

* xformers<0.0.27

* fix matplotlib.cm compatible issue

* remove transformers from frameowrk

* vllm to 0.5.1

* not upgrade pip

* add Pillow to framework

* Update version.py

---------

Co-authored-by: xingjun.wang <xingjun.wxj@alibaba-inc.com>
Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>
 2024-07-22 20:58:32 +08:00
xingjun.wang
4493786e04 remove unused code 2024-07-22 20:51:01 +08:00
xingjun.wang
c2e954245d reuse _download function in DataDownloadManager 2024-07-22 20:49:33 +08:00
xingjun.wang
b2157f1476 update comment 2024-07-22 20:45:00 +08:00
xingjun.wang
c9e1591798 update comments 2024-07-22 03:08:27 +08:00
xingjun.wang
71e6b7d72e add pillow 2024-07-22 03:00:21 +08:00
xingjun.wang
72d05d1f75 fix _dataset_info 2024-07-22 02:40:00 +08:00
xingjun.wang
271f2a1590 update requirements for datasets version restriction 2024-07-22 02:08:53 +08:00
xingjun.wang
2b0d79253c update print info 2024-07-21 20:20:47 +08:00
xingjun.wang
829a27e9ea set trust_remote_code is True in load_dataset_with_ctx 2024-07-21 20:02:21 +08:00
xingjun.wang
671504b410 update get_module_with_script 2024-07-21 19:37:27 +08:00
xingjun.wang
6445b8c6d1 fix disable_tqdm in some functions for 2.19.0 or later 2024-07-21 19:29:28 +08:00
xingjun.wang
32ceec8f74 add assert for youku data UT 2024-07-21 19:13:37 +08:00
xingjun.wang
de98268558 adapt to datasets version: 2.19.0 or later 2024-07-21 18:51:53 +08:00
xingjun.wang
f3f9480654 update UT for general dataset 2024-07-21 18:37:12 +08:00
xingjun.wang
dd5ef32b5c add youku in test_general_datasets 2024-07-21 03:05:17 +08:00
xingjun.wang
f8285e1fd7 update ut 2024-07-21 02:52:37 +08:00
xingjun.wang
db6add8e1c fix download config copy 2024-07-20 19:39:35 +08:00
xingjun.wang
b4b7e29b28 fix streaming for youku-mplug and adopt latest datasets 2024-07-20 18:59:19 +08:00
xingjun.wang
ef5f7d6543 del _datasets_server import in hf_dataset_util 2024-07-19 23:17:23 +08:00
Yingda Chen
710fdaf081 Merge pull request #910 from SNESNya/patch-1 
将 "library" 改为 "Library"
 2024-07-15 08:42:23 +08:00
LAGSNES
c069e1cbf4 将 "library" 改为 "Library" 2024-07-15 00:23:04 +08:00
wenmeng zhou
09d7a231de update discord in readme (#908) 2024-07-13 19:50:54 +08:00
wenmeng zhou
5c9e5a6283 Update discord in README.md 2024-07-13 19:29:23 +08:00
liuyhwangyh
f7a32e48e3 add dataset download (#906) 
* add dataset download

* fix cr issue

* fix cv matplotlib issue

* refactor code

* fix ut issue

* remove debug code

* remove unused import

* fix import issue

* sleep 65s before start docker avoid kill and run failed

---------

Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>
 2024-07-12 19:06:44 +08:00
liuyhwangyh
53acf79d04 fix logger issue (#899) 
Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>
 2024-07-03 20:03:46 +08:00
liuyhwangyh
083aa80d09 refactor ast index file generator (#897) 
* refactor ast index file generator

* fix lint issue

---------

Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>
 2024-07-03 18:23:50 +08:00
Zhongbing-Chen
3bafc5c344 exchange the order of replacement (#893) 2024-07-01 20:49:08 +08:00
liuyhwangyh
6995662f38 shorten check model timeout and snapshot_download file patter folder/ to folder/* (#896) 
Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>
 2024-07-01 15:52:16 +08:00
zhyncs
5b8e1e971c fix ignore_file_pattern (#886) 
* fix ignore_file_pattern

---------

Co-authored-by: root <root@kunkun>
 2024-07-01 14:31:39 +08:00
liuyhwangyh
ff77845772 add command line usage, optimize command line log, optimize packaging… (#888) 
* add command line usage, optimize command line log, optimize packaging version compatible

* Update command.md

* fix logging dup

* Refactor install dependencies (#889)

* refactor install dependencies, default only depencies of hub and datasets

* move pandas import to function

* update hub deps

* test

* remove generate ast file

* remove gast dependency

* replace gast with ast

* add dependency version

* remove lap for compile error

* fix comments issue

* add install ollama

---------

Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>

---------

Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>
 2024-06-30 20:15:03 +08:00
Firmament-cyou
8cdbeec3f6 Support swift model inference for llm pipeline (#880) 
* Support swift model inference for llm pipeline

* fix bug

* Add 'swfit' llm_framework and fix stream bug

* For pass the unittest run action
 2024-06-30 19:43:13 +08:00
yfchenmodelscope
f65f45959d Add Eres2netv2ep4 (#892) 
* add eres2netv2ep4
 2024-06-27 21:09:51 +08:00
wenmeng zhou
5d00b7c304 Merge pull request #876 from modelscope/release/1.15 
Release/1.15
 2024-06-06 18:01:22 +08:00
mulin.lyh
f34817e95d modify version merge master 2024-06-06 16:23:19 +08:00
wenmeng zhou
894bc90d96 remove assets and download from oss when running app (#875) 2024-06-05 21:18:21 +08:00
mulin.lyh
909e54fa33 modify build script 2024-06-05 19:08:46 +08:00
mulin.lyh
339ebab9dd fix download retry bug, add get_cache_dir to hut/util v1.15.0 2024-06-04 16:18:52 +08:00
mulin.lyh
7b18f99d03 release to 06.05 2024-06-03 10:09:07 +08:00
mulin.lyh
4a22b05891 restore http_get_file interface 2024-05-28 15:09:05 +08:00
mulin.lyh
e2d8a6d45f restore TDNN.py 2024-05-28 14:57:05 +08:00
mulin.lyh
8947d69fe2 Merge branch 'master' into release/1.15 2024-05-28 14:55:46 +08:00
liuyhwangyh
17da5e2264 fix error report (#868) 
Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>
 2024-05-28 14:38:19 +08:00
mulin.lyh
b46c5bc3f7 add TDNN.py and version to 1.15.0 2024-05-25 22:35:29 +08:00
mulin.lyh
f69ddba0bb fix TDNN.py and tdnn.py 2024-05-25 22:34:11 +08:00
liuyhwangyh
f93a184d88 add donwload command line and local_dir parameter (#866) 
* add donwload command line and local_dir parameter

Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>
 2024-05-25 14:21:55 +08:00
liuyhwangyh
5c470f8941 Download optimize (#862) 
* fix #845

Supports resumption of downloads from breakpoints, optimized download progress bar, finer display granularity, better experience under low bandwidth, and added function of downloading specified directories.

* restore push to hub

* fix merge issue

* fix ut issue

---------

Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>
 2024-05-24 15:37:43 +08:00