Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2025-12-25 12:39:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
1,555 Commits 263 Branches 106 Tags
b9bfbb70bc862da46e6b4a3a8b1fbe4469a4d089
Commit Graph

1555 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
wenmeng zhou
b9bfbb70bc Merge pull request #199 from modelscope/master-merge-internal20230315 
Master merge internal20230315
 2023-03-18 12:50:41 +08:00
yzhao
2a32822224 update version to 1.4.1 2023-03-15 21:59:38 +08:00
wenmeng.zwm
f7bbb33835 Merge branch master-merge-github0315-2 into master 
Title: Merge github master to gitlab 

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12008714
 2023-03-15 21:57:06 +08:00
yzhao
e6b73f3af6 Merge branch 'master-github' into master-merge-github0315-2 2023-03-15 21:34:38 +08:00
yuze.zyz
35744958f8 specify lower version 0.9.2 for librosa to avoid compatibility problem 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12008222
 2023-03-15 21:30:33 +08:00
zhk1425734486
9c2ca892ce Landing AI: tbs detetion pull request (revise image_open) (#184) 
* open image using load_image

* format code

---------

Co-authored-by: QZStudio <1643006760@qq.com>
Co-authored-by: wenmeng.zwm <wenmeng.zwm@alibaba-inc.com>
 2023-03-15 21:21:24 +08:00
xingjun.wxj
4e77f654f5 backward compatible with to_task_dataset function in adaseq repo 
1. backward compatible with to_task_dataset function for DefaultTrainer in adaseq repo
2. fix registry issue for RedsImageDeblurringDataset and GoproImageDeblurringDataset
3. add ut TestCustomDatasetsCompatibility
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11981956
 2023-03-14 18:34:44 +08:00
yuze.zyz
4a0cb67e66 fix numpy dependency 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11978446
 2023-03-14 13:31:22 +08:00
yuze.zyz
e1760bf846 fix onnxsim dependency 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11972894
 2023-03-13 21:24:53 +08:00
yuze.zyz
30be0b8bb0 fix import 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11969536
 2023-03-13 20:41:14 +08:00
shimin.ysm
79fd3f4696 bts 图片加载切换 LoadImage 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11969289
 2023-03-13 18:20:07 +08:00
yuanzhi.zyz
352f670bec fix cuda bug 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11968695
 2023-03-13 17:49:45 +08:00
xingjun.wxj
7659b64cdc downward compatible with v1.3.x and lower version for task_datasets, to fix import errors: 
1. gopro_image_deblurring_dataset
2. reds_image_deblurring_dataset
3. sidd_image_denoising
4. video_summarization_dataset
5. ASRDataset
 2023-03-13 16:00:33 +08:00
wucong.lyb
8e88ff46fc change funasr version to 0.3.0 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11956032
 2023-03-13 11:24:41 +08:00
lee.lcy
104213e4bf feat(thirdparty): add ADADET && add thirdparty arg for damoyolo trainer. 
feat(thirdparty): add ADADET && add thirdparty arg for damoyolo trainer.
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11957959

    * feat(thirdparty): add ADADET && add thirdparty arg for damoyolo trainer.
 2023-03-13 11:20:26 +08:00
lllcho.lc
38bcd54ee4 fix bug for install detecron2 
从github安装库有时会因为网络问题导致失败，故改成gitee
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11945563
 2023-03-10 15:14:21 +08:00
wucong.lyb
2286e7be08 fix bug：modify function generate_scp_from_url 
对于多通道url音频，目前maas-lib解析出来的bytes 每个通道的数据都混到了一起，因此做出修改：用户输入url音频，maas-lib负责下载到本地，然后把本地音频路径传给funasr，然后由funasr做音频解析
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11948914

    * modify function generate_scp_from_url
 2023-03-10 15:14:07 +08:00
hemu.zp
eecdd90412 add finetune_text_generation 
1. Add TrainingArgs and cli call form for text generation task
2. Fix dp+tp finetune bug
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11929345
 2023-03-10 15:08:56 +08:00
yzhao
bd12695d47 Update modelscope version to 1.4.0 2023-03-10 15:03:00 +08:00
yzhao
c505c6027e Merge branch 'master-merge-github20230310' 2023-03-10 14:58:34 +08:00
yzhao
f8a6362a22 pre-commit passed 2023-03-10 14:06:38 +08:00
kangxiaoyang.kxy
681afac803 add finetune for ddcolor image colorization 
图像上色模型添加finetune支持，本地测试已通过：python tests/test_image_colorization_trainer.py

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11874339
 2023-03-10 14:00:48 +08:00
yzhao
99e94bc2c2 Merge branch 'master-github' into master-merge-github20230310 2023-03-10 13:52:31 +08:00
james.wjg
1ce28bc4f9 cv/video_instance_segmentation 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11891175
 2023-03-10 10:04:03 +08:00
hemu.zp
429cfee826 Support run text generation pipeline with args 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11937122
 2023-03-10 09:48:10 +08:00
xingjun.wxj
e02a260c93 Refactor the task_datasets module 
Refactor the task_datasets module:

1. Add new module modelscope.msdatasets.dataset_cls.custom_datasets.
2. Add new function: modelscope.msdatasets.ms_dataset.MsDataset.to_custom_dataset().
2. Add calling to_custom_dataset() func in MsDataset.load() to adapt new custom_datasets module.
3. Refactor the pipeline for loading custom dataset: 
	1) Only use MsDataset.load() function to load the custom datasets.
	2) Combine MsDataset.load() with class EpochBasedTrainer.
4. Add new entry func for building datasets in EpochBasedTrainer: see modelscope.trainers.trainer.EpochBasedTrainer.build_dataset()
5. Add new func to build the custom dataset from model configuration, see: modelscope.trainers.trainer.EpochBasedTrainer.build_dataset_from_cfg()
6. Add new registry function for building custom datasets, see: modelscope.msdatasets.dataset_cls.custom_datasets.builder.build_custom_dataset()
7. Refine the class SiameseUIETrainer to adapt the new custom_datasets module.
8. Add class TorchCustomDataset as a superclass for custom datasets classes.
9. To move modules/classes/functions:
	1) Move module msdatasets.audio to custom_datasets
	2) Move module msdatasets.cv to custom_datasets
	3) Move module bad_image_detecting to custom_datasets
	4) Move module damoyolo to custom_datasets
	5) Move module face_2d_keypoints to custom_datasets
	6) Move module hand_2d_keypoints to custom_datasets
	7) Move module human_wholebody_keypoint to custom_datasets
	8) Move module image_classification to custom_datasets
	9) Move module image_inpainting to custom_datasets
	10) Move module image_portrait_enhancement to custom_datasets
	11) Move module image_quality_assessment_degradation to custom_datasets
	12) Move module image_quality_assmessment_mos to custom_datasets
	13) Move class LanguageGuidedVideoSummarizationDataset to custom_datasets
	14) Move class MGeoRankingDataset to custom_datasets
	15) Move module movie_scene_segmentation custom_datasets
	16) Move module object_detection to custom_datasets
	17) Move module referring_video_object_segmentation to custom_datasets
	18) Move module sidd_image_denoising to custom_datasets
	19) Move module video_frame_interpolation to custom_datasets
	20) Move module video_stabilization to custom_datasets
	21) Move module video_super_resolution to custom_datasets
	22) Move class GoproImageDeblurringDataset to custom_datasets
	23) Move class EasyCVBaseDataset to custom_datasets
	24) Move class ImageInstanceSegmentationCocoDataset to custom_datasets
	25) Move class RedsImageDeblurringDataset to custom_datasets
	26) Move class TextRankingDataset to custom_datasets
	27) Move class VecoDataset to custom_datasets
	28) Move class VideoSummarizationDataset to custom_datasets
10. To delete modules/functions/classes:
	1) Del module task_datasets
	2) Del to_task_dataset() in EpochBasedTrainer
	3) Del build_dataset() in EpochBasedTrainer and renew a same name function.
11. Rename class Datasets to CustomDatasets in metainfo.py

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11872747
 2023-03-10 09:03:32 +08:00
yanwen.pyl
fc7daea9c2 add soonet for video temporal grounding 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11810444
 2023-03-10 09:02:39 +08:00
zhangzhicheng.zzc
8a19e9645d [to #47860410]plugin with cli tool 
1. 支持 plugin方式接入外部 repo、github repo，本地repo，并进行外部插件管理
2. 支持allow_remote方式接入modelhub repo，该类型属于model 范畴不做额外插件管理
3. 支持cli 安装plugin相关

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11775456
 2023-03-09 23:07:13 +08:00
hemu.zp
2b1af959d5 Convert cfg during training 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11900238
 2023-03-09 22:27:44 +08:00
jinmao.yk
4078abf488 add human reconstruction task 
单图人体重建任务 

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11778199

 * add human reconstruction task
 2023-03-09 21:58:48 +08:00
dave.ma
f493e33720 vidt模型代码评审 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11873585

* vidt_v0
 2023-03-09 21:52:18 +08:00
wucong.lyb
d9b34daa79 add task: speech_timestamp && fix vad bugs(audio_fs is invalid) 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11940358

* modify punctuation postprocess

* modify audio input types

* add timestamp args

* add task: speech_timestamp

* modidy timestamp outputs

* add args:seg_dict&split_with_space

* feat: Fix conflict, auto commit by WebIDE
 2023-03-09 21:51:04 +08:00
yinyueqin.yyq
b5fe11fea0 upload disco guided diffusion 
把https://huggingface.co/IDEA-CCNL/Taiyi-Diffusion-532M-Cyberpunk-Chinese和https://huggingface.co/IDEA-CCNL/Taiyi-Diffusion-532M-Nature-Chinese迁移到MaaS-lib上。该project基于disco diffusion+guided diffusion。

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11818412

* upload disco guided diffusion
 2023-03-09 21:45:33 +08:00
yuze.zyz
a0bc5549a1 trainer support parallel_groups 
Design doc: https://yuque.alibaba-inc.com/suluyan.sly/yh1rvu/yx0owblyebpa2b3l?singleDoc#flU3s

1. Add parallel_group field in trainer to support DP, TP, PP.
2. Move the construction of common hooks(except optimizer/lrscheduler hook) to trainer's init method to support after_init stage.
	after_init is to support DP, TP, PP's initializing
         https://aone.alibaba-inc.com/v2/workitem#viewIdentifier=1c46ee8637e0c978f115b6f7&openWorkitemIdentifier=48099986
3. Add before_eval/after_eval stage to support model wrapping.
	to solve the order problem of apex amp & ddp wrapping.
         https://aone.alibaba-inc.com/v2/workitem#viewIdentifier=1c46ee8637e0c978f115b6f7&openWorkitemIdentifier=48099986
4. Exporter supports lazy importing.
	https://aone.alibaba-inc.com/v2/workitem#viewIdentifier=1c46ee8637e0c978f115b6f7&openWorkitemIdentifier=48122780
5. Fold all megatron imports to megatron hook.
         https://aone.alibaba-inc.com/v2/workitem#viewIdentifier=1c46ee8637e0c978f115b6f7&openWorkitemIdentifier=48099986
6. Add compile method to TorchModel ,Pipeline,Trainer to support torch2.0
	https://aone.alibaba-inc.com/v2/workitem#viewIdentifier=1c46ee8637e0c978f115b6f7&openWorkitemIdentifier=46869415
7. Fix bug: Lrscheduler builder does not support torch2.0
8. Add callbacks for trainer
	https://aone.alibaba-inc.com/v2/workitem#viewIdentifier=1c46ee8637e0c978f115b6f7&openWorkitemIdentifier=48210342
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11849932
 2023-03-09 21:33:35 +08:00
yuanzhi.zyz
2ce41d4974 add ocr_reco_crnntrain 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11824395
 2023-03-09 21:14:09 +08:00
bin.xue
281d3c630e [to #42322933] feat: kws support continue training from a checkpoint 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11940446

* feat: kws support continue training from a checkpoint

* log: add loading model log
 2023-03-09 21:07:34 +08:00
yuze.zyz
a2bed42fe1 fix a CI case 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11940845
 2023-03-09 19:55:22 +08:00
lllcho.lc
3e70ecfb37 add action detection finetune 
添加action-detection模型finetune支持

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11873651
 2023-03-09 16:54:19 +08:00
Louis
8abfffc7e5 import math to fix NameError (#147) 2023-03-09 15:35:32 +08:00
neo.dzh
86d5c2c5a8 fixbug for speaker_verification_pipeline and speaker_diarization_pipeline: re-write the default config with configure.json 
修复speaker_verification_pipeline和speaker_diarization_pipeline中configuration.json配置不生效的bug：
使用configuration.json中有的参数覆盖default

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11932226
 2023-03-09 15:20:53 +08:00
mulin.lyh
62f769877f [to #48291018]fix: fix data releate case failed bug 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11925637
 2023-03-09 15:18:50 +08:00
xixing.tj
46799325c8 add ocr_detection_db training module 
新增ocr_detection dbnet训练代码

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11892455
 2023-03-09 15:11:27 +08:00
feiyu.gfy
f534fd3ae0 add lore lineness table recognition 
读光无线表格识别（lineless-table-recognition）

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11772358
 2023-03-09 15:09:53 +08:00
tastelikefeet
cf8d2d574f add conversion script for checkpoint files before version 1.3.1. (#161) 
* add conversion script for older checkpoint files.

* fix format
 2023-03-09 14:58:21 +08:00
jiaqi.sjq
8099cc9f60 [to #41669377] Refactoring by using kantts as third party wheel and add ptts finetune&infer support 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11913929
 2023-03-09 11:32:44 +08:00
Patrick von Platen
2bf4ead3fb Correct DDIM sampling on GPU (#173) 2023-03-09 11:21:56 +08:00
ru.he
9307b071b7 Add the code for PEER model 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11873170
 2023-03-09 01:26:27 +08:00
qize.yqz
0efc892196 add smoke and fire detection model using damoyolo 
新增垂域目标检测模型-基于DAMO_YOLO的烟火检测模型

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11880693
 2023-03-09 01:25:06 +08:00
lingchen.zlm
a5b9d3378f [to #42322933] add generative multimodal embedding model RLEG 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11867878

* add generative multimodal embedding model RLEG

* remove useless import in rleg model
 2023-03-09 01:17:58 +08:00
gongbiao.gb
13752fa0c0 add vop_se for text video retrival 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11719262
 2023-03-09 01:14:47 +08:00