Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2025-12-25 20:49:37 +01:00
Code Issues Packages Projects Releases Wiki Activity
1,493 Commits 263 Branches 106 Tags
de67aa28e6d0cbc12e91f2436d025c2147542f44
Commit Graph

1493 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
bin.xue
de67aa28e6 [to #42322933] feat: optimize kws pipeline and training conf 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11897822
 2023-03-09 00:37:51 +08:00
liugao.lg
c28fd09d42 add more cli tools 
提供独立repo库快速接入modelscope的小工具，模板创建，模型上传，一键导入、版本管理等命令

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11757517
 2023-03-09 00:17:11 +08:00
wucong.lyb
9825c9d469 modify audio input types && punc postprocess 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11899315

* modify punctuation postprocess

* modify audio input types

* add timestamp args
 2023-03-09 00:08:19 +08:00
mulin.lyh
a8365bb17f [to #48269383]fix: onnx protobuf version conflict 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11924445
 2023-03-08 21:58:43 +08:00
zeyinzi.jzyz
bf3a2b6c09 support vision efficient tuning finetune 
## 查看改动点 ↓↓↓
### vision efficient tuning finetune
- Model模块改造成适配训练的
- Model模块在支持训练同时向下兼容之前发布的modecard
- Pipline兼容modelcard加载的preprocessor或直接定义的
- 添加 ImageClassificationPreprocessor （非mmcv版本）
- 添加 VisionEfficientTuningTrainer
- ~~添加 opencv_transforms==0.0.6~~ (以源代码引入必要)

### Modelcard
- test pipeline和trainer合并到一起
- 新增3个模型的test
- 新增demo service

### 公共组件
- ms_dataset.py: fix warning, [UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or xxx]
- preprocessor添加common：ToNumpy、Rename、Identity
- preprocessor common对于dict进行key判断再取值。
- ~~修复learning rate在iter级别变化的逻辑。~~ （本次不做了）
- ~~修复非dist状态下train data没有进行shuffle的bug。~~ (Master已有人改了)
- 修复训练时调用util中非cv包的异常 zhconv。

### 其他
- 为防止新引入的preprocessor模块在config中被原代码加载，导致在其他人做CI时会报错；所以暂时没有添加新的tag，等CR完成后，会进行打tag再rerun CI。
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11762108

* support vision efficient tuning finetune

* update test case

* update shuffle on IterableDataset

* update bitfit & sidetuning

* compatible with base trainer
 2023-03-08 16:42:23 +08:00
xingjun.wxj
8298a3c31d add force redownload for nerf dataset in ut 
add force redownload for nerf dataset in ut

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11915271
 2023-03-08 16:23:58 +08:00
mulin.lyh
e993438f86 [to #48269383]fix: onnx protobuf version conflict 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11914565

    * [to #48269383]fix: onnx protobuf version conflict
 2023-03-08 14:49:22 +08:00
xianzhe.xxz
691443257c add onnx exporter for damoyolo 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11889858
 2023-03-07 23:01:03 +08:00
bin.xue
b4c90d8160 [to #42322933] feat: add dingding denoise model 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11731594

* feat: add dingding denoise

* refactor: delete duplicated unidfsmn class

* refactor: delete empty lines

* refactor: make some methods inline

* style: add license and optimize imports

* style: comments style
 2023-03-07 21:55:10 +08:00
neo.dzh
0503c40919 re-format output for single input in audio/speaker_verification pipeline 
针对单条输入，格式为用户更友好的输出形式

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11897502
 2023-03-07 20:29:48 +08:00
xuanjie.wxb
ebb50d7887 add maoe unit test 2023-03-07 15:24:37 +08:00
pangda
798aa93cba [to #42322933] add onnx/torchscript exporter for token classification models 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11895085
 2023-03-07 14:12:23 +08:00
wendi.hwd
99fa2fe909 update license and revert support camouflaged-detection from latest master 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11892922

    * update license and revert support camouflaged-detection from latest master
 2023-03-07 11:41:10 +08:00
shichen.fsc
04f63ec71f [to #42322933] add new kws models in UT and update requirements/audio_kws.txt 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11830273

* [Add] add new models for kws UT

* [Update] update requirements

* pre-commit
 2023-03-06 23:14:45 +08:00
zhangzhicheng.zzc
4a72209e07 [to #48217480]bug fixed for ast scan funcitondef 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11882379
 2023-03-06 16:48:08 +08:00
fubang.zfb
dfd1058d45 siamese uie finetune support 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11706625
 2023-03-02 20:14:31 +08:00
wenmeng.zwm
77ff1d94d6 Revert "Add single-camera depth estimation model bts" 
This reverts commit feb4123327.
 2023-03-02 10:56:24 +08:00
wenmeng.zwm
f2f48dd305 Revert "support camouflaged-detection " 
This reverts commit ff55bd9436.
 2023-03-02 10:52:36 +08:00
ashui.cbh
329231fe9d ImagePaintbyexamplePipeline demo service 支持 
修改pipeline输出格式为numpy array 供demo service使用

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11841517
 2023-03-01 20:34:24 +08:00
xingjun.wxj
99e58c6ea3 [to #42322933] fix some feedback issues 
1. Fix the conflict between local path and remote dataset name in the form of dataset_name='namespace/dataset_name' in MsDataset.load() function.
2. Modify the obj_key.startswith value in get_split_objects_map function to adapt to dir name 'xxx/' format.

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11820290

* fix the conflict between local path and namespace/dataset_name of the dataset_name

* fix function: get_split_objects_map

* add UT for loading local csv file

* add new test case for test_load_local_csv function
 2023-03-01 20:13:31 +08:00
ryan.yy
4197f72124 nerf重建加速模型 加入trainer训练模块 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11794296
 2023-03-01 12:05:57 +08:00
yuxiang.tyx
09a178d171 support onnx export for SCRFD model 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11826666
 2023-03-01 10:02:49 +08:00
lee.lcy
8c39eefeff feat(trainer): support load_from for easycv trainer 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11794432
 2023-03-01 09:54:24 +08:00
hemu.zp
ceeb85f10f [to #42322933] Support multi-machine data and tensor parallel finetuning 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11682479
 2023-02-28 18:47:17 +08:00
myf272609
51cf0b16a8 [to #42322933] add fintune support for cartoon task 
人像卡通化模型增加训练支持

 Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11675597

* add fintune support for cartoon
 2023-02-28 17:01:34 +08:00
jiangyu.xzy
9f655da220 add speaker diarization pipeline 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11808124

* add speaker diarization pipeline
 2023-02-28 16:19:30 +08:00
yuxiang.tyx
3e5dbd2997 set num_workers=0 to prevent create subprocess which may cause error in demo-service 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11802831
 2023-02-28 14:12:50 +08:00
yuze.zyz
7181e667f6 Refactor hooks 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11651547
 2023-02-28 13:51:01 +08:00
zhicheng.sc
d11fff0c0a add ControlNet for controllable image generation 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11697239

* add ControlNet for scribble2image

* update code comments

* support scribble input

* update scribble input for demo service

* support all models of ControlNet

* add requirements

* fix code style bug

* update model id
 2023-02-28 13:39:12 +08:00
lee.lcy
a0877216a4 fix(damoyolo): fix FileNotFoundError when using trainer.evaluate() && add work_dir and exp_name to kwargs 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11793714

    * fix(damoyolo): fix FileNotFoundError when using trainer.evaluate() && add work_dir and exp_name to kwargs

* style(damoyolo): add code annotation to ImageDetectionDamoyoloTrainer
 2023-02-27 19:26:12 +08:00
wenmeng.zwm
99892d0812 Merge branch merge_master_github_0224 into master 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11782747
 2023-02-24 17:16:38 +08:00
yuanzhi.zyz
eca75bc6af add ocr_recognition bugfix 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11785977
 2023-02-24 17:02:24 +08:00
zhongning.hzn
8411645524 bad_image_detecting模型支持在数据集上validation功能 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11761935
 2023-02-24 14:23:57 +08:00
wenmeng.zwm
6a13557c19 Merge branch 'master-github' into merge_master_github_0224 2023-02-24 11:16:35 +08:00
jiaqi.sjq
058f772d16 [to #41669377] fix pipeline output wav is 32-bit float expect 16-bit int 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11765670
 2023-02-24 11:00:40 +08:00
maojialiang.mjl
97e35a040c [to #42322933] fix: remove restriction on image resolution in the preprocessing stage 
去掉预处理阶段resize 到720x1280的限制，支持输出与原始输入图像分辨率对应的结果

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11762135
 2023-02-24 09:49:02 +08:00
klayzhang.zb
4ca52a0934 [to #42322933] fix for text error correction: generate <unk> when input contains upper letter 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11762322

* [fix] fix: generate <unk> when input contains upper letter

* compatible new fairseq version
 2023-02-24 09:45:28 +08:00
Ikko Eltociear Ashimine
70d3799b26 fix typo in adv_utils.py (#133) 
occured -> occurred
 2023-02-23 20:37:01 +08:00
mulin.lyh
0cf0af9d62 [to #47521140]feat: tools statastics collection 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11565214
 2023-02-23 10:14:10 +08:00
jiangyu.xzy
0697f969a1 asr infer change vad/lm/punc input logic 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11747596

* change vad/punc/lm model input logic

* asr infer extra kwargs

* fix format

* new funasr version
 2023-02-22 21:05:58 +08:00
Anderson
7d15025302 update librosa.resample() parameters for version after 0.9 (#128) 
librosa.resample() changed parameters after 0.9

* Update ans_pipeline.py
 2023-02-22 20:28:24 +08:00
zhk1425734486
99d2177bb0 landing AI: tbs image detection pull request (#84) 
https://modelscope.cn/models/landingAI/LD_CytoBrainCerv/summary

兰丁宫颈细胞AI辅助诊断模型

Co-authored-by: QZStudio <1643006760@qq.com>
Co-authored-by: wenmeng.zwm <wenmeng.zwm@alibaba-inc.com>
 2023-02-22 20:25:37 +08:00
wenmeng.zwm
c5be16950f fix: fail to run ut in local because TEST_TOKEN required to set 2023-02-22 14:21:06 +08:00
wenmeng.zwm
f32fc32e16 update gif 2023-02-22 12:21:55 +08:00
chenxujun
adb4de753d Fix some words (#129) 2023-02-22 10:01:18 +08:00
zhangyanzhao.zyz
2ebb6db260 fix bug of modelscope.trainers.nlp.sentence_embedding_trainer.get_data_collator 
修正sentence_embedding_trainer 初始化bug

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11725227
 2023-02-21 22:42:13 +08:00
zhangzhicheng.zzc
fb9c41092a [to #42322933] update preprocessor for deberta v2 
相关改动已经在 model repo中进行了改动

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11714125

* update preprocessor for deberta
 2023-02-21 22:41:49 +08:00
zhangzhicheng.zzc
3e4ee5958a [to #47939677] load only backbone with weights 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11679242

* load backbone with weights directly
 2023-02-21 22:41:14 +08:00
xixing.tj
a06297e8c4 add TorchModel for vlpt and dbnet for ocr detection 
把vlpt和dbnet相关模型集成到model模块，通过configuration.json控制模型和后处理

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11712249
 2023-02-21 22:40:00 +08:00
mulin.lyh
386be89f3d [to #47947101]fix: fix diff based case not process deleted file 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11729541
 2023-02-21 20:50:03 +08:00