Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2026-02-24 20:19:51 +01:00
Code Issues Packages Projects Releases Wiki Activity
1,740 Commits 267 Branches 107 Tags
84070cc2f5dcf7a42bfbfc68577e27174da17841
Commit Graph

1740 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
wenmeng zhou
84070cc2f5 Merge pull request #336 from modelscope/master_merge_github_0619 
Master merge GitHub 0619 from internal
 2023-06-25 17:00:51 +08:00
wenmeng.zwm
3724697b65 update test data 2023-06-21 17:30:19 +08:00
wenmeng.zwm
54132e9299 update data/test to newest version 2023-06-21 16:35:10 +08:00
xingjun.wang
cb73bd14c0 merge internal master 2023-06-21 10:32:08 +08:00
xingjun.wxj
0db0ec5586 Merge code from github 
1. Merge(add) daily regression from github PR (daily_regression.yaml)
2. Add lora stable diffusion from github PR
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13010802
* fix: device arg not work, rename device to ngpu (#272)

* Correcting the lora stable diffusion example script (#300)

* add vad model and punc model in README.md 

add vad model and punc model

* Merge pull request #302 from modelscope/langgz-patch-1

add vad model and punc model in README.md

* add 1.6

* modify ignore

* Merge pull request #307 from modelscope/dev_rs_16

Merge release 1.6

* undo datetime to 2099

* Merge pull request #311 from modelscope/fix_master_version

undo datetime to 2099

* add daily regression workflow

* modify workflow name

* fix cron format issue

* lora trainer

* Merge pull request #315 from liuyhwangyh/add_regression_workflow

add daily regression workflow
 2023-06-21 10:22:06 +08:00
lanjinpeng.ljp
bc5c16aa10 support third_party key in pipeline 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12940228

* support third_party key in pipeline
 2023-06-20 19:25:24 +08:00
mulin.lyh
fa7562fd96 [to #50384694]fix: ci hang when build torch extension 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12976382
 2023-06-19 17:56:04 +08:00
hemu.zp
ad5d1aeb62 Fix gpt3 finetune nan 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12981998
* fix gpt3 finetune nan
 2023-06-19 11:25:27 +08:00
xingjun.wxj
cc3c384d5e Fix issues for downloading mplug-youku dataset 
1. Optimize downloading meta-csv files for large-scale dataset like mPLUG-youku (> 1GB for meta csv mapping)
2. Add head and overall progress bar for NativeIterableDataset
3. Modify the try-catch info for oss_utils
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12952842
 2023-06-15 15:42:21 +08:00
suluyan.sly
de491cfe6b [to #48855014] llama finetune + deepspeed 
1. llama base finetune：使用trainer从llama finetune至alpaca，效果确认并提供example
2. deepspeed通用性完善：mpu解耦；deepspeed的训练信息支持从ms log透出（目前ms打印的log有误）；支持从modelscope configuration.json 进行 deepspeed config 配置；deepspeed optimizer和lr_scheduler 初始化支持；解决deepspeed和ddp同时使用报错；解决保存ckpt时报错
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12651323
 2023-06-14 10:00:23 +08:00
zhifu.gzf
8b4e9dcdfb ngpu bug and rm easyasr 
修复ngpu指定无效的问题；移除easyasr，全部涉及模型都下架了；将funasr版本限制为>=0.6.0
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12933049
* ngpu bug and rm easyasr
 2023-06-13 16:36:21 +08:00
mulin.lyh
d6bf9e80fb [to #50129142]feat: support pre build torch extension build image, first extension megatron_util 2023-06-12 11:03:28 +08:00
hemu.zp
96c2d42f09 Add StreamingMixin 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12445731
* StreamingMixin poc

* update design

* Merge branch 'master' into feat/StreamingMixin

* add dicstr

* make postprocessor input consistent
 2023-06-08 19:40:14 +08:00
wucong.lyb
a77d390238 fix extra_args 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12839071
* fix extra_args

* update extra_args
 2023-06-08 19:39:05 +08:00
hejunjie.hjj
ae672b73df bug fix for single human parsing 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12869114
* fix bug: single human parsing, 当src_h, src_w相等时，new_h, new_w = 0, 0, 出现bug
 2023-06-08 19:36:18 +08:00
zsl01670416
75c90d8a98 add model revision in training_args and modify dataset loading in finetune text classification 
1.add parameter model_revision in training_args.py.
2.add parameter model_revision in kwargs for finetune_text_classification.py and finetune_text_generation.py.
3.modify dataset loading in finetune_text_classification.py for flex training.
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12869552
* add model revision in training_args and modify dataset loading in finetune text classification
 2023-06-08 19:36:05 +08:00
mulin.lyh
25a7398aea docker file py38 and py37 compatible merge 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12722824
* debug

* add missing deps

* add stanza

* fix numpy issue

* numpy version

* tf case

* numpy <= 1.22.0

* fix tf 1.15.5 case

* add python38 and python37 compatible docker file

* cv add fvcore

* add easycv

* remove debug code

* fix style issue

* fix python3.7 python3.8 compatible issue

* comment

* modify run config

* scipy<=1.7.3

* revert scipy<=1.7.3

* fix compatible bug

* fix compatible bug

* add easycv

* update version: 1.6.1rc0
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12754942
fix python3.8 and python3.8 compatible issue

* numpy version change

* numpy version

* numpy version

* fix numpy version

* fix numpy version

* update version: 1.6.1rc0
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12754942
fix python3.8 and python3.8 compatible issue

* restore setup.py

* restore setup.py

* add build base image

* fix style

* numpy version fix

* optimize build image, split base and modelscope image

* fix style issue

* modify example path

* modify examples folder

* add SETUPTOOLS_USE_DISTUTILS=stdlib comments
 2023-06-08 10:01:58 +08:00
pangda
73f8d39897 Add ner example for flextrain 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12835095
* update named_entity_recognition example
 2023-06-05 15:17:45 +08:00
liuyhwangyh
66fc614e15 Merge pull request #315 from liuyhwangyh/add_regression_workflow 
add daily regression workflow
 2023-06-03 06:14:52 +08:00
liuyhwangyh
2498bb60b6 fix cron format issue 2023-06-02 16:01:36 +08:00
liuyhwangyh
d61529dfa5 modify workflow name 2023-06-01 14:25:02 +08:00
mulin.lyh
8dc5246fdb add daily regression workflow 2023-06-01 09:36:06 +08:00
xixing.tj
1b7e0f50f4 add ocr detection new model db-nas 
新增5M的DB-NAS ocr detection 文字检测模型
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12760623
* add ocr detection new model db-nas

* add comment
 2023-05-31 21:32:46 +08:00
hemu.zp
898e3a42eb Fix bug for DeepspeedHook.register_processor 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12686921
 2023-05-31 21:29:36 +08:00
yuanzhi.zyz
10c39b5ce1 add new ocr recognition model (LightweightEdge) and some functions 
1. 增加了新轻量化端侧识别模型 LightweightEdge，并把原来CRNN和ConvNextViT的代码整理了
2. 增加batch inference支持
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12787905
 2023-05-31 21:16:22 +08:00
Xingjun.Wang
5634a18a58 Merge pull request #311 from modelscope/fix_master_version 
undo datetime to 2099
 2023-05-30 14:27:16 +08:00
xingjun.wang
6611445bab undo datetime to 2099 2023-05-30 14:04:12 +08:00
zhangzhicheng.zzc
c7aede97ba add pattern to skip easycv.thirdparty 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12785287

* add pattern to remove

* update pattern to remove

* avoid import with build
 2023-05-30 11:16:39 +08:00
yuze.zyz
003c33648a Fix CI: test merge dataset failed 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12771691
* fix CI

* add teardown
 2023-05-29 10:32:33 +08:00
xingjun.wxj
5b99bb24fe update version: 1.6.1rc0 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12754942
 2023-05-25 20:28:36 +08:00
yuze.zyz
e2a20fd817 fix merge_hook 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12718423
* fix merge_hook
 2023-05-25 20:05:04 +08:00
yuze.zyz
a7a3eb5dc5 Support flex train feature 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12737668
 2023-05-25 19:39:24 +08:00
yuze.zyz
c08b924968 Fix ner tokenizer which won't accept kwargs 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12664244
* fix slow tokenizer
 2023-05-25 18:55:47 +08:00
feiyu.gfy
f82361d71b 修正lineless_table_recognition功能遇到空白图片崩溃的bug 
用户反馈lineless_table_recognition功能遇到空白图片会发生崩溃。
增加处理机制以防止崩溃。

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12741201
* fix bug
 2023-05-25 18:28:05 +08:00
lee.lcy
a524e01e47 feat: support onnx export for domain_specific_object_detection 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12682688
 2023-05-25 10:44:09 +08:00
xingjun.wxj
f2640a5a12 fix private dataset auth issue 
1. Fix private datasets auth issue
2. Add arg: token (optional) in MsDataset.load() for FlexTrain
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12721569
 2023-05-24 19:48:20 +08:00
Xingjun.Wang
92e12b5f19 Merge pull request #307 from modelscope/dev_rs_16 
Merge release 1.6
 2023-05-24 11:12:18 +08:00
xingjun.wang
d3fa166fe0 modify ignore 2023-05-23 21:21:15 +08:00
xingjun.wang
48c0d2a9af add 1.6 v1.6.0 2023-05-22 10:53:18 +08:00
yuze.zyz
d38fa563a8 Re commit lossing files for flextrain 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12674746
 2023-05-19 10:48:33 +08:00
mulin.lyh
6abdce929d fix py38 issue 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12660340
 2023-05-17 22:11:23 +08:00
chenyafeng.cyf
f6ea3eadea eres2net 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12640199
 2023-05-16 22:28:20 +08:00
yuze.zyz
c6ef9024d7 Fix distributed hook to lazyimport; fix an import bug 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12638653
 2023-05-16 21:37:12 +08:00
yuze.zyz
38fa4cceed fix transformer example and fix some bugs 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12626375
 2023-05-16 15:27:17 +08:00
zsl01670416
71a80173e5 modify examples text_classification, text_generation, token_classification for improved trainer 
1.text_classification/run_train.sh
2.text_generation/run_train_mt5.sh, run_train_palm.sh
3.token_classification/finetune_token_classification.py, run_train_mgeo.sh, run_train_structbert.sh
above files were modified to adapt to improved trainer
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12629010
 2023-05-16 14:32:43 +08:00
hemu.zp
5804ad2dc1 update multi_modal_embedding example 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12626062
 2023-05-16 14:31:26 +08:00
zhifu.gzf
6c8c3a53f8 fix bug in **kwargs duplicated 
修复**kwargs中参数重复的bug
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12633542
 2023-05-16 13:49:01 +08:00
wenmeng.zwm
3ac31290f2 add description for modelscope 2023-05-15 19:26:21 +08:00
yuze.zyz
e325862be8 add pop for base class parameters 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12622643
 2023-05-15 17:46:32 +08:00
yeqinghao.yqh
b9c8c99776 Support mPLUG-Owl model. 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12610417
 2023-05-15 16:32:46 +08:00