Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2025-12-25 20:49:37 +01:00
Code Issues Packages Projects Releases Wiki Activity
1,729 Commits 263 Branches 106 Tags
b6c40aaa87d7f79ea1f970451d740407ffe063f4
Commit Graph

956 Commits

Author SHA1 Message Date
mulin.lyh
698c794070 [to #50537864]fix: fix select case issue 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13055098
 2023-06-25 22:44:29 +08:00
chenyafeng.cyf
29062d9f94 eres2net_aug v2 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13046524

* eres2net_aug v2
 2023-06-25 18:07:04 +08:00
tongmu.wh
f03c93cda5 add speaker diarization pipeline and improve some speaker pipelines 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12891685

* add new speaker diarization pipeline. improve the capability of speaker pipelines
 2023-06-21 17:56:05 +08:00
xingjun.wxj
0db0ec5586 Merge code from github 
1. Merge(add) daily regression from github PR (daily_regression.yaml)
2. Add lora stable diffusion from github PR
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13010802
* fix: device arg not work, rename device to ngpu (#272)

* Correcting the lora stable diffusion example script (#300)

* add vad model and punc model in README.md 

add vad model and punc model

* Merge pull request #302 from modelscope/langgz-patch-1

add vad model and punc model in README.md

* add 1.6

* modify ignore

* Merge pull request #307 from modelscope/dev_rs_16

Merge release 1.6

* undo datetime to 2099

* Merge pull request #311 from modelscope/fix_master_version

undo datetime to 2099

* add daily regression workflow

* modify workflow name

* fix cron format issue

* lora trainer

* Merge pull request #315 from liuyhwangyh/add_regression_workflow

add daily regression workflow
 2023-06-21 10:22:06 +08:00
xingjun.wxj
cc3c384d5e Fix issues for downloading mplug-youku dataset 
1. Optimize downloading meta-csv files for large-scale dataset like mPLUG-youku (> 1GB for meta csv mapping)
2. Add head and overall progress bar for NativeIterableDataset
3. Modify the try-catch info for oss_utils
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12952842
 2023-06-15 15:42:21 +08:00
hemu.zp
96c2d42f09 Add StreamingMixin 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12445731
* StreamingMixin poc

* update design

* Merge branch 'master' into feat/StreamingMixin

* add dicstr

* make postprocessor input consistent
 2023-06-08 19:40:14 +08:00
mulin.lyh
25a7398aea docker file py38 and py37 compatible merge 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12722824
* debug

* add missing deps

* add stanza

* fix numpy issue

* numpy version

* tf case

* numpy <= 1.22.0

* fix tf 1.15.5 case

* add python38 and python37 compatible docker file

* cv add fvcore

* add easycv

* remove debug code

* fix style issue

* fix python3.7 python3.8 compatible issue

* comment

* modify run config

* scipy<=1.7.3

* revert scipy<=1.7.3

* fix compatible bug

* fix compatible bug

* add easycv

* update version: 1.6.1rc0
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12754942
fix python3.8 and python3.8 compatible issue

* numpy version change

* numpy version

* numpy version

* fix numpy version

* fix numpy version

* update version: 1.6.1rc0
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12754942
fix python3.8 and python3.8 compatible issue

* restore setup.py

* restore setup.py

* add build base image

* fix style

* numpy version fix

* optimize build image, split base and modelscope image

* fix style issue

* modify example path

* modify examples folder

* add SETUPTOOLS_USE_DISTUTILS=stdlib comments
 2023-06-08 10:01:58 +08:00
xixing.tj
1b7e0f50f4 add ocr detection new model db-nas 
新增5M的DB-NAS ocr detection 文字检测模型
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12760623
* add ocr detection new model db-nas

* add comment
 2023-05-31 21:32:46 +08:00
yuanzhi.zyz
10c39b5ce1 add new ocr recognition model (LightweightEdge) and some functions 
1. 增加了新轻量化端侧识别模型 LightweightEdge，并把原来CRNN和ConvNextViT的代码整理了
2. 增加batch inference支持
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12787905
 2023-05-31 21:16:22 +08:00
yuze.zyz
003c33648a Fix CI: test merge dataset failed 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12771691
* fix CI

* add teardown
 2023-05-29 10:32:33 +08:00
xingjun.wxj
5b99bb24fe update version: 1.6.1rc0 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12754942
 2023-05-25 20:28:36 +08:00
lee.lcy
a524e01e47 feat: support onnx export for domain_specific_object_detection 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12682688
 2023-05-25 10:44:09 +08:00
xingjun.wxj
f2640a5a12 fix private dataset auth issue 
1. Fix private datasets auth issue
2. Add arg: token (optional) in MsDataset.load() for FlexTrain
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12721569
 2023-05-24 19:48:20 +08:00
mulin.lyh
6abdce929d fix py38 issue 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12660340
 2023-05-17 22:11:23 +08:00
chenyafeng.cyf
f6ea3eadea eres2net 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12640199
 2023-05-16 22:28:20 +08:00
yeqinghao.yqh
b9c8c99776 Support mPLUG-Owl model. 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12610417
 2023-05-15 16:32:46 +08:00
wenmeng.zwm
a631c17353 Merge branch dev/merge_github_master_0512 into master 
Title: Merge github master 

Merge github master 0512

1. https://github.com/modelscope/modelscope/pull/297
2. https://github.com/modelscope/modelscope/pull/289
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12609811
 2023-05-15 09:39:28 +08:00
lingcai.wl
0aaaaa13ad [to #49275037] remove demo check and fix service decoder 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12432683
 2023-05-14 23:41:40 +08:00
xingjun.wang
98e899b8c2 merge master 2023-05-14 02:09:02 +08:00
xingjun.wang
bf2cd23aa2 cancel a ut 2023-05-14 02:06:11 +08:00
yuze.zyz
febc0365de Support FlexTrain and update the structure of trainer 
1. Refactor training_args
2. Refactor hooks
3. Add train_id for push_to_hub
4. Support both output_dir/output_sub_dir for checkpoint_hooks
5. Support copy when hardlink fails when checkpointing
6. Support mixed dataset config file as a CLI argument
7. Add eval txt in output folder
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12384253
* support the ignorance of file pattern
 2023-05-13 12:12:04 +08:00
xingjun.wang
41a40edef3 ok Merge branch 'master' of gitlab.alibaba-inc.com:Ali-MaaS/MaaS-lib into dev/merge_github_master_0512 2023-05-12 19:05:22 +08:00
xingjun.wang
9be87e6754 ok Merge branch 'dev/merge_github_master_0512' of gitlab.alibaba-inc.com:Ali-MaaS/MaaS-lib into dev/merge_github_master_0512 2023-05-12 18:06:26 +08:00
tongmu.wh
ca0cf2b5cf merge master ,加入speaker change locating pipeline 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12601179
 2023-05-12 17:53:56 +08:00
hejunjie.hjj
1cd67ef060 [to #42322933] add cvpr23 fastinst model 2023-05-12 17:51:00 +08:00
xingjun.wang
92270f5f23 ok Merge branch 'dev/merge_github_master_0512' of gitlab.alibaba-inc.com:Ali-MaaS/MaaS-lib into dev/merge_github_master_0512 2023-05-12 17:32:46 +08:00
xingjun.wxj
e630621599 Virgo SDK supports odps data source 
1. Support ODPS datasource for virgo sdk.
2. Adapt "inner_url" parser for single-modal and multi-modal datasets.

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12583695

* add virgo sdk odps data

* refine virgo sdk args and odps fetch batch data pipeline

* add inner import for odps

* del import VirgoDataset in MsDataset

* fix import VirgoDataset

* support inner url downloading

* refine dataset.py and maxcompute_utils.py

* add ut for virgo odps data batch

* reset unifold ut level as 1

* refine virgo batch data
 2023-05-12 17:27:19 +08:00
xingjun.wxj
cafc90da21 Fix master UT on level0 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12599393

* reset ut level for test_run_with_model_name_for_unite_large and test_run_with_model_name_for_unite_base

* fix master UT for level0

* update EasyCVFace2DKeypointsPipelineTest skip info
 2023-05-12 17:25:32 +08:00
xingjun.wang
0cf0ac2112 merge master 2023-05-12 17:13:10 +08:00
wangmeng.xwm
f0d519219f add clip_interrogator 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12388571

* init commit of clip_interrogator

* fix format

* fix environment dependencies

* fix backend.mps

* fix test image path

* add model introduction and paper link

* add comment example for pipeline, fix device control
 2023-05-12 16:06:52 +08:00
wanyu.wy
58df448182 Add trainer for UniTE 2023-05-11 14:41:08 +08:00
mulin.lyh
7b14a0e11f Pipeline input, output and parameter normalization. 2023-05-11 11:20:01 +08:00
hehong.chh
b80c13036f fix bug: hypotheses did't init in cpu device、make fid_dialogue_test available 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12406082
 2023-05-10 11:10:31 +08:00
Li Tong
626b60e8c2 fix issue: Batch and single are not uniform #215 (#258) 
* fix issue: Batch and single are not uniform #215, https://github.com/modelscope/modelscope/issues/215

* remove extra code

* fix bug

* update test

---------

Co-authored-by: ezeli.lt <ezeli.lt@alibaba-inc.com>
Co-authored-by: hemu <hemu.zp@alibaba-inc.com>
 2023-05-10 10:49:54 +08:00
ly261666
bdd282a127 [to #42322933] add transface model 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12568301
 2023-05-10 01:04:10 +08:00
jiangnana.jnn
46072898da remove easycv codes, plugin access 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11965727

* remove easycv codes

* fix custome msdatasets import and remove metainfo

* fix pipeline imports

* fix pre-check

* fix models import

* fix pre-check

* merge master
 2023-05-09 17:58:01 +08:00
bin.xue
6d68f0ea64 [to #42322933] add ONNX exporter for ans dfsmn 2023-04-28 10:33:32 +08:00
chenyafeng.cyf
93f73a26e7 add rdino model 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12406691
 2023-04-26 11:18:28 +08:00
yuanzhi.zyz
b19cc13ca7 update ocr_reco 
1. 增加本地lmdb输入的接口；
2. finetune后模型的keys有prefix，导致不能直接读取，已在model初始化时统一替换；
3. 添加各个模型的cpu和gpu测试。
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12256669
* add local_lmdb read
 2023-04-21 15:05:16 +08:00
xingjun.wxj
92f7c66d12 Add progress bar and fix cache dir issue 
1. Add progress bar for dataframe apply when downloading data files.
2. Fix cache data files issue
3. Refine _prepare_and_download() func

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12408412
* add progress bar and fix cache dir

* add progress bar and fix cache dir

* refine code
 2023-04-21 13:49:17 +08:00
bin.xue
dd4299e0c8 [to #42322933] add DFSMN model to far field KWS 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12334627
* feat: set stream mode in constructor

* fix: commit model file

* doc: add more information about DFSMN

* fix: seperation model evaluate returns None in speechbrain whose version < 0.5.12
 2023-04-20 22:45:07 +08:00
mulin.lyh
8ea82e6f6d * add lfs append 2023-04-18 20:32:11 +08:00
xingjun.wxj
976bdf4ebc Add virgo hub 
1. Add Hubs.virgo in MsDataset.load()
2. Add VirgoDownloader pipeline
3. Add auth for virgo hub.
4. Add other utils.
5. cherry-pick from https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12352198?file=8c317278a40b19fe76825fb749e768fdbfd6a711

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12351633
 2023-04-18 17:01:43 +08:00
hemu.zp
4f300c8632 Fix generate for ModelForTextGeneration 
Separate the `generate` function, no longer use the implementation in the transformers library to avoid error due to transformers version upgrades.
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12340643
 2023-04-16 02:05:41 +08:00
yuze.zyz
2650d37094 Fix the log level and return value of push_to_hub 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12338218
 2023-04-15 16:42:35 +08:00
zhangzhicheng.zzc
04e8ddc41e fix update ast not remove origin information 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12319197
 2023-04-13 16:10:07 +08:00
hehong.chh
10101e08aa skip FidDialogueTest temporarily, due to the model is set to non-public 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12315621
 2023-04-13 13:27:25 +08:00
hemu
672f25266f Merge branch 'master-github' into master-merge-github-0413 2023-04-13 10:32:01 +08:00
slin000111
92d7eae5b9 add token classification example and gpt3 one layer test (#268) 2023-04-13 10:21:00 +08:00
hemu.zp
7b4a0ea02c Merge branch master-merge-github-0412 into master 
Title: Merge branch 'master-merge-github-0412' into master 
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12304662
 2023-04-12 19:16:11 +08:00