Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2025-12-15 15:57:42 +01:00
Code Issues Packages Projects Releases Wiki Activity
2,836 Commits 262 Branches 106 Tags
master
Commit Graph

263 Commits

Author SHA1 Message Date
Yunlin Mao
9cb7ce7029 Update build file and workflow (#1551) 
* update build file

* update setup file

* update fairseq dep

* fix error log
 2025-12-02 10:50:39 +08:00
Yunlin Mao
2719e90528 [Feature] Add pyproject.toml (#1542) 
* add pyproject.toml

* fix
 2025-11-06 12:40:04 +08:00
tastelikefeet
3bad15b779 add ms-agent into dockerfile (#1436) 2025-08-04 17:08:48 +08:00
Tian-Jionglu
7ed173be8b fix(deps): add missing filelock dependency (#1427) 
Co-authored-by: Tian-Jionglu <tianjionglu@outlook.com>
 2025-08-04 16:57:45 +08:00
tastelikefeet
556aa76dc3 fix pyyaml according to: https://github.com/Anchor0221/CVE-2025-50460 (#1428) 2025-08-01 22:04:44 +08:00
Xingjun.Wang
f4217e6183 Upgrade datasets (#1393) 
* upgrade datasets to 3.6.0

* fix mcp lint
 2025-07-04 10:59:07 +08:00
Xingjun.Wang
b4e682ec8e Add setuptools for hub requirements (#1343) 
* add setuptools in hub.txt and update setuptools version
 2025-05-22 10:44:47 +08:00
Jintao
03dbc7bd11 Merge release1.25 to master (#1319) 2025-04-23 13:40:54 +08:00
Jintao
32aefd7428 Merge release1.24 to master (#1298) 2025-04-07 13:32:09 +08:00
Xingjun.Wang
6b9bfdf25a Set kwsbp==0.0.6 (#1293) 
* set kwsbp==0.0.6 to avoid install 0.1.0 version which is invalid wheel package on PyPI

* update ddpm_guided_diffusion, shotdetect_scenedetect_lgss,MinDAEC
 2025-04-07 10:24:45 +08:00
tastelikefeet
7246d0b359 Merge 1.23 hotfix to master (#1227) 2025-02-17 10:30:24 +08:00
Xingjun.Wang
71a54c1a51 support latest datasets version (#1163) 2025-01-02 17:06:46 +08:00
Yunlin Mao
31bf308e44 downgrade moviepy to 1.0.3 (#1133) 2024-12-10 10:10:01 +08:00
tastelikefeet
328bbc0494 Fix some bugs (#1056) 
* fix some bugs

* remove install tf-kersa

* fix

* use bin bash as default
 2024-10-30 15:49:38 +08:00
Xingjun.Wang
134fe72f06 hotfix for datasets 3.0.2 (#1046) 2024-10-23 16:25:53 +08:00
Xingjun.Wang
2c4505e13a Adapt new datasets (#1002) 
* update ds==3.0

* update

* add http_get_ms func

* del unused code

* fix pr issue and update requirements
 2024-09-30 16:46:00 +08:00
tastelikefeet
4c518db424 patch hf hub (#987) 2024-09-14 12:24:19 +08:00
liuyhwangyh
92ce1aa41d fix dataset page bug, framework add transformers dependency (#927) 
* fix dataset page bug, framework add transformers dependency
Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>
 2024-07-26 19:03:39 +08:00
Xingjun.Wang
210ab40c54 Upgrade datasets (#921) 
* del _datasets_server import in hf_dataset_util

* fix streaming for youku-mplug and adopt latest datasets

* fix download config copy

* update ut

* add youku in test_general_datasets

* update UT for general dataset

* adapt to datasets version: 2.19.0 or later

* add assert for youku data UT

* fix disable_tqdm in some functions for 2.19.0 or later

* update get_module_with_script

* set trust_remote_code is True in load_dataset_with_ctx

* update print info

* update requirements for datasets version restriction

* fix _dataset_info

* add pillow

* update comments

* update comment

* reuse _download function in DataDownloadManager

* remove unused code

* update test_run_modelhub in Human3DAnimationTest

* set datasets>=2.18.0
 2024-07-23 22:26:12 +08:00
liuyhwangyh
4e2555c5a3 Release/1.16 merge to master (#922) 
* update version

* merge master

* fix fairseq issue

* fix pip compatible issue

* fix bug

* version to 1.16.1

* xformers<0.0.27

* fix matplotlib.cm compatible issue

* remove transformers from frameowrk

* vllm to 0.5.1

* not upgrade pip

* add Pillow to framework

* Update version.py

---------

Co-authored-by: xingjun.wang <xingjun.wxj@alibaba-inc.com>
Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>
 2024-07-22 20:58:32 +08:00
liuyhwangyh
f7a32e48e3 add dataset download (#906) 
* add dataset download

* fix cr issue

* fix cv matplotlib issue

* refactor code

* fix ut issue

* remove debug code

* remove unused import

* fix import issue

* sleep 65s before start docker avoid kill and run failed

---------

Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>
 2024-07-12 19:06:44 +08:00
liuyhwangyh
ff77845772 add command line usage, optimize command line log, optimize packaging… (#888) 
* add command line usage, optimize command line log, optimize packaging version compatible

* Update command.md

* fix logging dup

* Refactor install dependencies (#889)

* refactor install dependencies, default only depencies of hub and datasets

* move pandas import to function

* update hub deps

* test

* remove generate ast file

* remove gast dependency

* replace gast with ast

* add dependency version

* remove lap for compile error

* fix comments issue

* add install ollama

---------

Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>

---------

Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>
 2024-06-30 20:15:03 +08:00
xingjun.wang
09779f1735 set datasets <2.19.0 2024-04-19 19:40:00 +08:00
xingjun.wang
7b8e10123b fix dataset hf utils 2024-04-18 23:13:26 +08:00
Xingjun.Wang
1a66f069c4 Dataset refactor (#807) 
* add main entry in ms_dataset

* update func get_data_patterns import

* modify return_config_only

* modify return_config_only to dataset_info_only

* udpate version for test

* del get_logger(__name__)

* fix py script loading

* fix loading py and without py

* add subset support

* add hf_datasets_util; refine list_repo_tree_ms; fix private datasets loading issue

* update version to rc5

* fix and support preview for dataset_info_only mode

* fix urlencode

* update to rc7

* loading of dataset_infos.json is deprecated; 2. add some ut

* update version

* add escapechar for read_csv and to_csv

* add params: Source=SDK

* add create_dataset func

* overwrite _get_paths_info

* update & version

* update list_repo_tree name

* add get_module_with_script, fix download imports

* fix py script loading issue in dataset_module_factory

* fix create dataset

* update log info in api
 2024-03-22 17:30:34 +08:00
liuyhwangyh
59760b123b Merge branch 'master' into master-merge-internal20240110 2024-01-14 10:43:11 +08:00
zhifu gao
49c04ea47e update funasr1.0 (#715) 
* funasr1.0  modelscope

* fix lint issue

---------

Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>
 2024-01-12 12:02:01 +08:00
ly119399
0e772d35f3 upgrade diffusers version 2024-01-03 17:22:35 +08:00
neo.dzh
e7f86a751e add audio codec and codec-based TTS model 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/15128959
 2023-12-26 20:54:59 +08:00
xingjun.wxj
2a991a5c6b update datasets version 
Update datasets version.
compatibility check： 2.14.5, 2.14.6, 2.15.0
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/14916111
 2023-12-06 16:25:20 +08:00
xingjun.wxj
a8e9e0a48f set datasets==2.14.6 2023-12-01 17:33:07 +08:00
mulin.lyh
51a1b76e91 fix python3.10 compatible issue 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/14678226
* modify librosa version

* fix python3.10 compatible issue

* remove healpy in requirements for windowns compatible
 2023-11-29 17:41:44 +08:00
mulin.lyh
6c7fca8307 支持modelscope直接拉起推理服务 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/14702876

* add inference server code

* add server requirement

* fix import issue

* debug

* add command line

* add llmpipeline support

* modify port to int

* add serer usage

* remove unused code

* fix lint issue

* add inference server code

* upgrade env to VLLM_USE_MODELSCOPE
 2023-11-29 17:37:56 +08:00
mulin.lyh
b8e86060f5 numpy version unrestrict 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13398805
* numpy version unrestrict
 2023-11-27 13:56:33 +08:00
xingjun.wxj
6833bdabfc set datasets==2.14.6 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/14593950
 2023-11-17 10:46:58 +08:00
mulin.lyh
23f1f474bf Merge branch 'master-github' into master-merge-github925 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/14164566
 2023-09-26 21:15:41 +08:00
mulin.lyh
59ec83cd2d remove cv diffusers version 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13843028
* remove cv diffusers version
 2023-08-30 15:04:43 +08:00
Xiaobin
8ce3b4de72 Feature/seq gpt (#507) 2023-08-29 17:47:59 +08:00
tastelikefeet
f2e9ea4199 Replace code with swift wheel (#467) 2023-08-29 17:27:18 +08:00
Wang Qiang
de33f4dc87 Support sdxl finetune by lora method (#468) 
* support sdxl finetune by lora

* remove useless imports

* support sdxl finetune

* upgrade diffusers to 0.19.0

* sdxl finetune

* fix bugs

* pre commit

* diffusers>=0.19.0
 2023-08-23 11:23:34 +08:00
wenmeng zhou
e3abfe0564 Update framework.txt (#453) 
remove version restrict for numpy and pandas
 2023-08-09 18:00:44 +08:00
suluyan.sly
f080603802 remove requirement realesrgan for basicsr 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13488773
* remove requirement realesrgan for basicsr
 2023-08-01 16:55:47 +08:00
suluyan.sly
dc17c7d6ee remove cv requirements: basicsr 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13471366
* rm req basicsr
 2023-07-31 18:20:23 +08:00
suluyan.sly
05e1357c32 Merge branch 'master-github' into master-merge-github-230728 2023-07-28 16:40:34 +08:00
frozoul
2566d028cd cv/cv nerf 3d reconstruction 4k nerf damo (#389) 
* add 4k-nerf core files

* update configure file

* update dataloader and model path

* update unittest

* Delete test_4k.py

* update unittest

* update unittest

* update pre-commit

* update dataloader

* update cuda code path

* check with pre-commit

---------

Co-authored-by: zhongshu.wzs <zhongshu.wzs@alibaba-inc.com>
Co-authored-by: wenmeng zhou <wenmeng.zwm@alibaba-inc.com>
 2023-07-28 10:37:13 +08:00
suluyan.sly
1c6f5fe775 Merge branch 'master-github' into master-merge-github-230727 
Conflicts:
       examples/pytorch/baichuan/finetune_baichuan.py
       examples/pytorch/chatglm6b/finetune.py
 2023-07-27 17:29:27 +08:00
mengyang.fmy
18f998a85c add text-to-360pano-image pipeline, mod cv requirements 
7月份计划上线的360全景图生成模型，自研

模型权重文件地址https://www.modelscope.cn/models/damo/cv_diffusion_text-to-360panorama-image_generation/summary


#### 依赖项说明

##### 由于要使用xformers，torch版本最好使用1.13.1
```
pip install torch==1.13.1+cu116 torchvision==0.14.1+cu116 torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/cu116
```
##### 对应的diffusers和xformers版本如下
```
pip install -U diffusers==0.18.0
pip install xformers==0.0.16
pip install triton, accelerate, transformers
```

##### ModelScope Library 需要使用cv
```
pip install modelscope
pip install "modelscope[cv]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html
```

##### 此外，还需要安装第三方的一个库，Real-ESRGAN， 安装方法如下
```
# Install basicsr - https://github.com/xinntao/BasicSR
# We use BasicSR for both training and inference
pip install basicsr
# facexlib and gfpgan are for face enhancement
pip install facexlib
pip install gfpgan
pip install Pillow
pip install tqdm
pip install realesrgan==0.3.0
```
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13346430
* add text-to-360pano-image pipeline

* add text-to-360pano-image pipeline, mod cv requirements

* rm redundant files and cv requirements; add standard input and output definations

* fix diffusers==0.18.0 and run test

* fix diffusers==0.18.0 in multi-modal and run test again

* add model_revision='v1.0.0'

* fix yapf

* add trycatch for enabling xformers

* fix key error

* add install xformers in test/setup

* skip highres.fix in ci

* feat: Fix conflict, auto commit by WebIDE
 2023-07-27 11:33:39 +08:00
Xingjun.Wang
41c2c95083 dataset fix version (#413) 
* modify datasets version
 2023-07-25 22:18:25 +08:00
jiaqi.sjq
1f05acef88 fix pip install error when pysptk>=0.1.19 2023-07-25 21:13:31 +08:00
tongmu.wh
ba1a333ba6 add language recognition pipelines and models 
新增语种识别pipeline和model
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13385083
* add language recognition pipelines and models

* add a clustering method for speaker diarization

* define input and output type for language recognition
 2023-07-25 21:07:56 +08:00