Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2025-12-24 03:59:23 +01:00
Code Issues Packages Projects Releases Wiki Activity
1,730 Commits 263 Branches 106 Tags
6ab85256af546ff2b6e9331d18f9e71dfaa3d52e
Commit Graph

1730 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
wenmeng.zwm
6ab85256af to [#50529030] fix easycv lr hook error 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13069804
 2023-06-26 21:54:01 +08:00
suluyan.sly
b6c40aaa87 [to #50538422] feat: make diff & recover for llama based weights 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13055390
 2023-06-26 19:41:19 +08:00
mulin.lyh
698c794070 [to #50537864]fix: fix select case issue 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13055098
 2023-06-25 22:44:29 +08:00
hemu.zp
1421629392 Support device_map for transformers model 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13048856

* support device_map

* support device_map for T5

* fix bug
 2023-06-25 20:15:11 +08:00
chenyafeng.cyf
29062d9f94 eres2net_aug v2 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13046524

* eres2net_aug v2
 2023-06-25 18:07:04 +08:00
wenmeng.zwm
674c67ce48 move data to lfs 2023-06-21 17:57:34 +08:00
tongmu.wh
f03c93cda5 add speaker diarization pipeline and improve some speaker pipelines 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12891685

* add new speaker diarization pipeline. improve the capability of speaker pipelines
 2023-06-21 17:56:05 +08:00
xingjun.wxj
0db0ec5586 Merge code from github 
1. Merge(add) daily regression from github PR (daily_regression.yaml)
2. Add lora stable diffusion from github PR
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13010802
* fix: device arg not work, rename device to ngpu (#272)

* Correcting the lora stable diffusion example script (#300)

* add vad model and punc model in README.md 

add vad model and punc model

* Merge pull request #302 from modelscope/langgz-patch-1

add vad model and punc model in README.md

* add 1.6

* modify ignore

* Merge pull request #307 from modelscope/dev_rs_16

Merge release 1.6

* undo datetime to 2099

* Merge pull request #311 from modelscope/fix_master_version

undo datetime to 2099

* add daily regression workflow

* modify workflow name

* fix cron format issue

* lora trainer

* Merge pull request #315 from liuyhwangyh/add_regression_workflow

add daily regression workflow
 2023-06-21 10:22:06 +08:00
lanjinpeng.ljp
bc5c16aa10 support third_party key in pipeline 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12940228

* support third_party key in pipeline
 2023-06-20 19:25:24 +08:00
mulin.lyh
fa7562fd96 [to #50384694]fix: ci hang when build torch extension 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12976382
 2023-06-19 17:56:04 +08:00
hemu.zp
ad5d1aeb62 Fix gpt3 finetune nan 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12981998
* fix gpt3 finetune nan
 2023-06-19 11:25:27 +08:00
xingjun.wxj
cc3c384d5e Fix issues for downloading mplug-youku dataset 
1. Optimize downloading meta-csv files for large-scale dataset like mPLUG-youku (> 1GB for meta csv mapping)
2. Add head and overall progress bar for NativeIterableDataset
3. Modify the try-catch info for oss_utils
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12952842
 2023-06-15 15:42:21 +08:00
suluyan.sly
de491cfe6b [to #48855014] llama finetune + deepspeed 
1. llama base finetune：使用trainer从llama finetune至alpaca，效果确认并提供example
2. deepspeed通用性完善：mpu解耦；deepspeed的训练信息支持从ms log透出（目前ms打印的log有误）；支持从modelscope configuration.json 进行 deepspeed config 配置；deepspeed optimizer和lr_scheduler 初始化支持；解决deepspeed和ddp同时使用报错；解决保存ckpt时报错
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12651323
 2023-06-14 10:00:23 +08:00
zhifu.gzf
8b4e9dcdfb ngpu bug and rm easyasr 
修复ngpu指定无效的问题；移除easyasr，全部涉及模型都下架了；将funasr版本限制为>=0.6.0
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12933049
* ngpu bug and rm easyasr
 2023-06-13 16:36:21 +08:00
mulin.lyh
d6bf9e80fb [to #50129142]feat: support pre build torch extension build image, first extension megatron_util 2023-06-12 11:03:28 +08:00
hemu.zp
96c2d42f09 Add StreamingMixin 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12445731
* StreamingMixin poc

* update design

* Merge branch 'master' into feat/StreamingMixin

* add dicstr

* make postprocessor input consistent
 2023-06-08 19:40:14 +08:00
wucong.lyb
a77d390238 fix extra_args 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12839071
* fix extra_args

* update extra_args
 2023-06-08 19:39:05 +08:00
hejunjie.hjj
ae672b73df bug fix for single human parsing 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12869114
* fix bug: single human parsing, 当src_h, src_w相等时，new_h, new_w = 0, 0, 出现bug
 2023-06-08 19:36:18 +08:00
zsl01670416
75c90d8a98 add model revision in training_args and modify dataset loading in finetune text classification 
1.add parameter model_revision in training_args.py.
2.add parameter model_revision in kwargs for finetune_text_classification.py and finetune_text_generation.py.
3.modify dataset loading in finetune_text_classification.py for flex training.
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12869552
* add model revision in training_args and modify dataset loading in finetune text classification
 2023-06-08 19:36:05 +08:00
mulin.lyh
25a7398aea docker file py38 and py37 compatible merge 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12722824
* debug

* add missing deps

* add stanza

* fix numpy issue

* numpy version

* tf case

* numpy <= 1.22.0

* fix tf 1.15.5 case

* add python38 and python37 compatible docker file

* cv add fvcore

* add easycv

* remove debug code

* fix style issue

* fix python3.7 python3.8 compatible issue

* comment

* modify run config

* scipy<=1.7.3

* revert scipy<=1.7.3

* fix compatible bug

* fix compatible bug

* add easycv

* update version: 1.6.1rc0
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12754942
fix python3.8 and python3.8 compatible issue

* numpy version change

* numpy version

* numpy version

* fix numpy version

* fix numpy version

* update version: 1.6.1rc0
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12754942
fix python3.8 and python3.8 compatible issue

* restore setup.py

* restore setup.py

* add build base image

* fix style

* numpy version fix

* optimize build image, split base and modelscope image

* fix style issue

* modify example path

* modify examples folder

* add SETUPTOOLS_USE_DISTUTILS=stdlib comments
 2023-06-08 10:01:58 +08:00
pangda
73f8d39897 Add ner example for flextrain 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12835095
* update named_entity_recognition example
 2023-06-05 15:17:45 +08:00
xixing.tj
1b7e0f50f4 add ocr detection new model db-nas 
新增5M的DB-NAS ocr detection 文字检测模型
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12760623
* add ocr detection new model db-nas

* add comment
 2023-05-31 21:32:46 +08:00
hemu.zp
898e3a42eb Fix bug for DeepspeedHook.register_processor 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12686921
 2023-05-31 21:29:36 +08:00
yuanzhi.zyz
10c39b5ce1 add new ocr recognition model (LightweightEdge) and some functions 
1. 增加了新轻量化端侧识别模型 LightweightEdge，并把原来CRNN和ConvNextViT的代码整理了
2. 增加batch inference支持
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12787905
 2023-05-31 21:16:22 +08:00
zhangzhicheng.zzc
c7aede97ba add pattern to skip easycv.thirdparty 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12785287

* add pattern to remove

* update pattern to remove

* avoid import with build
 2023-05-30 11:16:39 +08:00
yuze.zyz
003c33648a Fix CI: test merge dataset failed 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12771691
* fix CI

* add teardown
 2023-05-29 10:32:33 +08:00
xingjun.wxj
5b99bb24fe update version: 1.6.1rc0 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12754942
 2023-05-25 20:28:36 +08:00
yuze.zyz
e2a20fd817 fix merge_hook 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12718423
* fix merge_hook
 2023-05-25 20:05:04 +08:00
yuze.zyz
a7a3eb5dc5 Support flex train feature 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12737668
 2023-05-25 19:39:24 +08:00
yuze.zyz
c08b924968 Fix ner tokenizer which won't accept kwargs 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12664244
* fix slow tokenizer
 2023-05-25 18:55:47 +08:00
feiyu.gfy
f82361d71b 修正lineless_table_recognition功能遇到空白图片崩溃的bug 
用户反馈lineless_table_recognition功能遇到空白图片会发生崩溃。
增加处理机制以防止崩溃。

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12741201
* fix bug
 2023-05-25 18:28:05 +08:00
lee.lcy
a524e01e47 feat: support onnx export for domain_specific_object_detection 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12682688
 2023-05-25 10:44:09 +08:00
xingjun.wxj
f2640a5a12 fix private dataset auth issue 
1. Fix private datasets auth issue
2. Add arg: token (optional) in MsDataset.load() for FlexTrain
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12721569
 2023-05-24 19:48:20 +08:00
yuze.zyz
d38fa563a8 Re commit lossing files for flextrain 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12674746
 2023-05-19 10:48:33 +08:00
mulin.lyh
6abdce929d fix py38 issue 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12660340
 2023-05-17 22:11:23 +08:00
chenyafeng.cyf
f6ea3eadea eres2net 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12640199
 2023-05-16 22:28:20 +08:00
yuze.zyz
c6ef9024d7 Fix distributed hook to lazyimport; fix an import bug 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12638653
 2023-05-16 21:37:12 +08:00
yuze.zyz
38fa4cceed fix transformer example and fix some bugs 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12626375
 2023-05-16 15:27:17 +08:00
zsl01670416
71a80173e5 modify examples text_classification, text_generation, token_classification for improved trainer 
1.text_classification/run_train.sh
2.text_generation/run_train_mt5.sh, run_train_palm.sh
3.token_classification/finetune_token_classification.py, run_train_mgeo.sh, run_train_structbert.sh
above files were modified to adapt to improved trainer
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12629010
 2023-05-16 14:32:43 +08:00
hemu.zp
5804ad2dc1 update multi_modal_embedding example 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12626062
 2023-05-16 14:31:26 +08:00
zhifu.gzf
6c8c3a53f8 fix bug in **kwargs duplicated 
修复**kwargs中参数重复的bug
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12633542
 2023-05-16 13:49:01 +08:00
wenmeng.zwm
3ac31290f2 add description for modelscope 2023-05-15 19:26:21 +08:00
yuze.zyz
e325862be8 add pop for base class parameters 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12622643
 2023-05-15 17:46:32 +08:00
yeqinghao.yqh
b9c8c99776 Support mPLUG-Owl model. 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12610417
 2023-05-15 16:32:46 +08:00
hemu.zp
1d28c5b730 mv mmdet to cv 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12614719
 2023-05-15 11:48:29 +08:00
wenmeng.zwm
a631c17353 Merge branch dev/merge_github_master_0512 into master 
Title: Merge github master 

Merge github master 0512

1. https://github.com/modelscope/modelscope/pull/297
2. https://github.com/modelscope/modelscope/pull/289
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12609811
 2023-05-15 09:39:28 +08:00
lingcai.wl
0aaaaa13ad [to #49275037] remove demo check and fix service decoder 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12432683
 2023-05-14 23:41:40 +08:00
xingjun.wang
98e899b8c2 merge master 2023-05-14 02:09:02 +08:00
xingjun.wang
bf2cd23aa2 cancel a ut 2023-05-14 02:06:11 +08:00
yuze.zyz
febc0365de Support FlexTrain and update the structure of trainer 
1. Refactor training_args
2. Refactor hooks
3. Add train_id for push_to_hub
4. Support both output_dir/output_sub_dir for checkpoint_hooks
5. Support copy when hardlink fails when checkpointing
6. Support mixed dataset config file as a CLI argument
7. Add eval txt in output folder
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12384253
* support the ignorance of file pattern
 2023-05-13 12:12:04 +08:00