Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2025-12-25 20:49:37 +01:00
Code Issues Packages Projects Releases Wiki Activity
2,234 Commits 263 Branches 106 Tags
f7238b3515d3ef161d84b89e7fca8fee12faeb4b
Commit Graph

51 Commits

Author SHA1 Message Date
mulin.lyh
c039b7348d force upgrade funasr build image 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/14411115
* force upgrade funasr build image
 2023-10-24 15:19:01 +08:00
mulin.lyh
6e424cdb69 force upgrade transformers when build image 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/14293744
* force upgrade transformers when build image
 2023-10-13 19:54:48 +08:00
mulin.lyh
23f1f474bf Merge branch 'master-github' into master-merge-github925 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/14164566
 2023-09-26 21:15:41 +08:00
mulin.lyh
de2d7cbc69 fix select case error 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13889257
* fix select case error
 2023-09-04 11:16:28 +08:00
mulin.lyh
7c0f6aa5c5 Merge branch 'master-github' into master-merge-github0829 2023-08-29 21:20:54 +08:00
mulin.lyh
cba4e40bc1 fix numpy pandas compatible issue 
明确受影响的模型(damo)：  
ONE-PEACE-4B	ModuleNotFoundError: MyCustomPipeline: MyCustomModel: No module named 'one_peace'，缺少依赖。
cv_resnet50_face-reconstruction	 不兼容tf2  
nlp_automatic_post_editing_for_translation_en2de	tf2.0兼容性问题，tf1.x需要  
cv_resnet18_ocr-detection-word-level_damo	tf2.x兼容性问题  
cv_resnet18_ocr-detection-line-level_damo	tf兼容性问题  
cv_resnet101_detection_fewshot-defrcn	模型限制必须detection0.3+torch1.11.0"  
speech_dfsmn_ans_psm_48k_causal	"librosa， numpy兼容性问题  
cv_mdm_motion-generation	"依赖numpy版本兼容性问题：   File ""/opt/conda/lib/python3.8/site-packages/smplx/body_models.py"",  
cv_resnet50_ocr-detection-vlpt	numpy兼容性问题  
cv_clip-it_video-summarization_language-guided_en	tf兼容性问题

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13744636
* numpy and pandas no version

* modify compatible issue

* fix numpy compatible issue

* modify ci

* fix lint issue

* replace Image.ANTIALIAS to Image.Resampling.LANCZOS pillow compatible

* skip uncompatible cases

* fix numpy compatible issue, skip cases that can not compatbile numpy or tensorflow2.x

* skip compatible cases

* fix clip model issue

* fix body 3d keypoints compatible issue
 2023-08-22 23:04:31 +08:00
liuyhwangyh
75a14a36ba get github diff files (#446) 
* get github diff files

* add github environment

---------

Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>
 2023-08-10 16:22:37 +08:00
wenmeng zhou
7b8bb091a5 Merge pull request #437 from modelscope/merge_gitlab_0803 
Merge gitlab 0803
 2023-08-03 12:25:11 +08:00
liuyhwangyh
0fff03d640 modify gpu/cpu binding (#436) 2023-08-03 11:27:37 +08:00
mulin.lyh
da2badaacd [to #51274424]fix: easycv extension and tinycudann conflict 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13471150
* fix easycv extension and tinycudann conflict
 2023-07-31 18:08:53 +08:00
mulin.lyh
a780935317 torch1.11 and torch2.0.1 build script refactor 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13408751

* [to #51122016]feat: build image support torch1.x 2.x cuda11.3.0 11.7.1

* remove base  version

* env cudatookit-version

* build script merge

* torch 1.11.0 fix
 2023-07-25 21:15:00 +08:00
mulin.lyh
247ae5f637 default aliyuncs pip 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13320919
* default aliyuncs pip

* pip to aliyun source
 2023-07-18 19:59:23 +08:00
mulin.lyh
bb9163f71a remove hard code tsinghua 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13314592
* remove hard code tsinghua
 2023-07-18 14:51:18 +08:00
mulin.lyh
fa7562fd96 [to #50384694]fix: ci hang when build torch extension 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12976382
 2023-06-19 17:56:04 +08:00
mulin.lyh
d6bf9e80fb [to #50129142]feat: support pre build torch extension build image, first extension megatron_util 2023-06-12 11:03:28 +08:00
mulin.lyh
25a7398aea docker file py38 and py37 compatible merge 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12722824
* debug

* add missing deps

* add stanza

* fix numpy issue

* numpy version

* tf case

* numpy <= 1.22.0

* fix tf 1.15.5 case

* add python38 and python37 compatible docker file

* cv add fvcore

* add easycv

* remove debug code

* fix style issue

* fix python3.7 python3.8 compatible issue

* comment

* modify run config

* scipy<=1.7.3

* revert scipy<=1.7.3

* fix compatible bug

* fix compatible bug

* add easycv

* update version: 1.6.1rc0
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12754942
fix python3.8 and python3.8 compatible issue

* numpy version change

* numpy version

* numpy version

* fix numpy version

* fix numpy version

* update version: 1.6.1rc0
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12754942
fix python3.8 and python3.8 compatible issue

* restore setup.py

* restore setup.py

* add build base image

* fix style

* numpy version fix

* optimize build image, split base and modelscope image

* fix style issue

* modify example path

* modify examples folder

* add SETUPTOOLS_USE_DISTUTILS=stdlib comments
 2023-06-08 10:01:58 +08:00
chenxujun
20b3a679e7 Fix some words (#141) 2023-03-02 11:06:56 +08:00
mulin.lyh
7378f94455 [to #47749867] modify audio install command 2023-02-10 12:04:08 +00:00
mulin.lyh
86128cd3a8 [to #47612577]feat: image supoort 3d models, python3.8 test version 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11542009


* mmcv-full<=1.7.0 for mmdet3d

* fix code style issue

* add speechbrain==0.5.7

* add compatible comments
 2023-02-07 02:28:20 +00:00
mulin.lyh
e54694690f [to #46993990]feat: run ci cases base on code diff to reduct ci test time 2023-02-06 08:00:19 +00:00
mulin.lyh
f663f420c4 [to #46480415]feat: ci command custom support regression case run all case in subprocess 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10936241
 2022-12-01 19:33:25 +08:00
pengzhendong
b0cf09d7b0 [ci] chang pypi url to tsinghua 2022-11-24 22:12:58 +08:00
wenmeng.zwm
584fa8f586 merge internal master 2022-11-18 14:21:05 +08:00
mulin.lyh
90a5efa1c2 [to #46106568]feat: parallel run ci case 2022-11-17 08:51:23 +08:00
wenmeng.zwm
9f2de967c5 merge master and fix conflict 2022-11-16 18:10:39 +08:00
mulin.lyh
c6e575458a [to #45878116]feat: run linter test first 
提高ci效率，先跑linter，linter错误先报漏出来，不用合并release
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10596907

    * [to #45878116]feat: run linter test first
 2022-11-11 16:19:14 +08:00
wenmeng.zwm
266851bbbb Merge branch 'master-gitlab' into master-github 2022-11-08 18:06:20 +08:00
mulin.lyh
87fcd28c4c add ci tag to cicases 2022-11-04 09:11:51 +08:00
wenmeng.zwm
c638acbeda fix merge conflict 2022-10-26 17:12:25 +08:00
mulin.lyh
384377b8f5 * [to #45486649]feat: modelscope model version use model repo tag, unsupport branch or commit it, client user-agent header unified 2022-10-26 13:55:51 +08:00
wenmeng.zwm
ca7189e77a add dummy git config 2022-10-10 17:38:03 +08:00
wenmeng.zwm
e7b37b09d4 do linter test only in alibaba internal env 2022-10-10 12:03:05 +08:00
wenmeng.zwm
fabb4716d4 [to #44610931] fix: add device usage when device is None or empty 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10039848

    * add device usage when device is None or empty

    * update docker env
 2022-09-06 21:47:59 +08:00
mulin.lyh
83dbf71302 [to #44702084]fix: ci pip install domain in single commands, find with requirement install failed is complicated. 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10019738

    * [to #44702084]fix: ci pip install domain in single commands, find with requirement install failed is complicated.
 2022-09-05 17:38:05 +08:00
mulin.lyh
042cff7d68 [to #44702084]fix: ci pip install domain in single commands, find with requirement install failed is complicated. 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10014958

    * [to #44702084]fix: ci pip install domain in single commands, find with requirement install failed is complicated.
 2022-09-05 16:08:50 +08:00
mulin.lyh
9e14d6727b [to #44571845]fix: ci support multiple image 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9974293
 2022-09-02 11:02:43 +08:00
mulin.lyh
12698b31a0 [to #44340132] fix: ci case run out of gpu memory 2022-08-30 17:59:15 +08:00
xingjun.wxj
44033290d4 [to #42322933]MsDataset 支持上传数据集压缩包和meta 
1. MsDataset支持upload数据文件(压缩包)
2. MsDataset支持clone和upload meta data
3. 使用MsDataset.load()下载数据集，支持web端显示数据集下载计数
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9831232
 2022-08-25 22:28:10 +08:00
mulin.lyh
01bb751425 [to #43653669]feat: auto build docker images 
auto build docker images
aone 任务： https://test.aone.alibaba-inc.com/jobs/1824567?buildId=143470479
修改任务分支
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9566518

    * [to #43653669]feat: auto build docker images
 2022-08-22 16:01:42 +08:00
mulin.lyh
3272b2755c [to #43909247]fix: access token in code, move to environment 
move access token to environment variable.
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9681287

    * [to #43909247]fix: access token in code, move to environment
 2022-08-09 17:46:56 +08:00
mulin.lyh
d2a5a63fd8 [to #43859920]fix: ci test TEST_LEVEL is no effort 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9662219

    * [to #43859920]fix: ci test TEST_LEVEL is no effort

* [to #43859920]fix: ci test TEST_LEVEL is no effort
 2022-08-07 15:31:10 +08:00
wenmeng.zwm
c539811deb [to #43115513] remove repeated cv task 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9617409
 2022-08-03 11:49:12 +08:00
mulin.lyh
47dda0a5f9 [to #43698011]feat: login with access token 2022-08-02 18:16:28 +08:00
Yingda Chen
f581ad2eb4 [to #43525415]fix: fix docker ci performance issue 2022-07-25 12:41:44 +08:00
mulin.lyh
69047b99ae [to #43387011]feat: ci test to new host and running in docker 
ci 测试迁移新的机器，并且在容器中运行，减小互相干扰的可能
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9427096

    * add docker ci script
 2022-07-20 11:49:05 +08:00
wenmeng.zwm
9f1ad5da80 fix several small problems for v0.2 
* rename name of whl to modelscope
* auto install all requirements when running citest
* auto download dynamic lib for aec pipeline
* fix setup.py  audio extras not set
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9297825
 2022-07-07 16:40:11 +08:00
wenmeng.zwm
8e51a073a6 [to #42966122] requirements enchanment and self-host repo support 
* add self-hosted repo:
* add extra requirements for different field and reduce necessary requirements
* update docker file with so required by audio
* add requirements checker which will be used later when implement lazy import
* remove repeated requirements and replace opencv-python-headless with opencv-python

example usage:
```shell
pip install model_scope[all] -f https://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/release/maas/repo.html
pip install model_scope[cv] -f https://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/release/maas/repo.html
pip install model_scope[nlp] -f https://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/release/maas/repo.html
pip install model_scope[audio] -f https://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/release/maas/repo.html
pip install model_scope[multi-modal] -f https://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/release/maas/repo.html

```
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9211383
 2022-07-01 16:38:06 +08:00
wenmeng.zwm
1f6b376599 [to #42373878] refactor maaslib to modelscope 
1.  refactor maaslib to modelscope
2.  fix UT error
3.  support pipeline which does not register default model

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8988388
 2022-06-09 20:16:26 +08:00
wenmeng.zwm
db4a8be9c5 [to #41669377] docs and tools refinement and release 
1. add build_doc linter script
2. add sphinx-docs support
3. add development doc and api doc
4. change version to 0.1.0 for the first internal release version

Link: https://code.aone.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8775307
 2022-05-20 16:51:34 +08:00
wenmeng.zwm
5e469008fd [to #41401401] add preprocessor, model and pipeline 
* add preprocessor module
 * add model base and builder
 * update task constant
 * add load image preprocessor and its dependency
 * add pipeline interface and UT covered
 * support default pipeline for task
 * add image matting pipeline
 * refine nlp tokenize interface
 * add nlp pipeline 
 * fix UT failed
 * add test for Compose

Link: https://code.aone.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8769235

* add preprocessor module

* add test for Compose

* fix citest error

* fix abs class error

* add model base and builder

* update task constant

* add load image preprocessor and its dependency

* add pipeline interface and UT covered

* support default pipeline for task

* refine models and pipeline interface

* add pipeline folder structure

* add image matting pipeline

* refine nlp tokenize interface

* add nlp pipeline 

1.add preprossor model pipeline for nlp text classification
2. add corresponding test

Link: https://code.aone.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8757371

* new nlp pipeline

* format pre-commit code

* update easynlp pipeline

* update model_name for easynlp pipeline; add test for maas_lib/utils/typeassert.py

* update test_typeassert.py

* refactor code

1. rename typeassert to type_assert
2. use lazy import to make easynlp dependency optional
3. refine image matting UT

* fix linter test failed

* update requirements.txt

* fix UT failed

* fix citest script to update requirements
 2022-05-19 22:18:35 +08:00