Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2026-02-24 12:10:09 +01:00
Code Issues Packages Projects Releases Wiki Activity
1,558 Commits 267 Branches 107 Tags
chatglm_6b
Commit Graph

1558 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Zhicheng Zhang
d117f2c007 add test and merge release 2023-03-18 16:31:44 +08:00
Zhicheng Zhang
148a43d526 Merge remote-tracking branch 'github/release/1.4' into chatglm_6b 2023-03-18 10:54:58 +08:00
shuaigezhu
ffa7975ba2 init 2023-03-17 14:05:58 +08:00
yzhao
060f7ba540 update version to 1.4.1 v1.4.1 2023-03-15 22:01:02 +08:00
yuze.zyz
334e7d91b4 specify lower version 0.9.2 for librosa to avoid compatibility problem 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12008222
(cherry picked from commit 35744958f8)
 2023-03-15 22:00:12 +08:00
zhk1425734486
ab07c44681 Landing AI: tbs detetion pull request (revise image_open) (#184) 
* open image using load_image

* format code

---------

Co-authored-by: QZStudio <1643006760@qq.com>
Co-authored-by: wenmeng.zwm <wenmeng.zwm@alibaba-inc.com>
(cherry picked from commit 9c2ca892ce)
 2023-03-15 22:00:03 +08:00
yzhao
a434b3be73 update to official version v1.4.0 2023-03-15 11:23:46 +08:00
xingjun.wxj
1d6734a909 backward compatible with to_task_dataset function in adaseq repo 
1. backward compatible with to_task_dataset function for DefaultTrainer in adaseq repo
2. fix registry issue for RedsImageDeblurringDataset and GoproImageDeblurringDataset
3. add ut TestCustomDatasetsCompatibility
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11981956

(cherry picked from commit 4e77f654f5)
 2023-03-14 18:35:11 +08:00
yzhao
24ae36767c change release time 2023-03-14 17:47:14 +08:00
yuze.zyz
4aa6d7ad6d fix numpy dependency 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11978446

(cherry picked from commit 4a0cb67e66)
 2023-03-14 13:31:49 +08:00
yuze.zyz
05109d836e fix onnxsim dependency 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11972894

(cherry picked from commit e1760bf846)
 2023-03-13 21:25:12 +08:00
yuze.zyz
ea939517ce fix import 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11969536

(cherry picked from commit 30be0b8bb0)
 2023-03-13 20:42:21 +08:00
shimin.ysm
e295e0dcee bts 图片加载切换 LoadImage 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11969289
(cherry picked from commit 79fd3f4696)
 2023-03-13 19:14:18 +08:00
yuanzhi.zyz
e639a97928 fix cuda bug 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11968695

(cherry picked from commit 352f670bec)
 2023-03-13 17:51:12 +08:00
xingjun.wxj
3c97cda6fb downward compatible with v1.3.x and lower version for task_datasets, to fix import errors: 
1. gopro_image_deblurring_dataset
2. reds_image_deblurring_dataset
3. sidd_image_denoising
4. video_summarization_dataset
5. ASRDataset

(cherry picked from commit 7659b64cdc)
 2023-03-13 16:01:09 +08:00
wucong.lyb
6e11983cb1 change funasr version to 0.3.0 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11956032

(cherry picked from commit 8e88ff46fc)
 2023-03-13 11:26:12 +08:00
lee.lcy
9a84f4bb4b feat(thirdparty): add ADADET && add thirdparty arg for damoyolo trainer. 
feat(thirdparty): add ADADET && add thirdparty arg for damoyolo trainer.
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11957959

    * feat(thirdparty): add ADADET && add thirdparty arg for damoyolo trainer.

(cherry picked from commit 104213e4bf)
 2023-03-13 11:26:04 +08:00
lllcho.lc
87d015fb55 fix bug for install detecron2 
从github安装库有时会因为网络问题导致失败，故改成gitee
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11945563

(cherry picked from commit 38bcd54ee4)
 2023-03-10 15:15:15 +08:00
wucong.lyb
2b19c3f705 fix bug：modify function generate_scp_from_url 
对于多通道url音频，目前maas-lib解析出来的bytes 每个通道的数据都混到了一起，因此做出修改：用户输入url音频，maas-lib负责下载到本地，然后把本地音频路径传给funasr，然后由funasr做音频解析
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11948914

    * modify function generate_scp_from_url

(cherry picked from commit 2286e7be08)
 2023-03-10 15:15:03 +08:00
hemu.zp
8c13f9107f add finetune_text_generation 
1. Add TrainingArgs and cli call form for text generation task
2. Fix dp+tp finetune bug
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11929345

(cherry picked from commit eecdd90412)
 2023-03-10 15:09:32 +08:00
yzhao
80bac9a7f6 update release version to 1.4.0rc0 2023-03-10 15:06:30 +08:00
yzhao
bd12695d47 Update modelscope version to 1.4.0 2023-03-10 15:03:00 +08:00
yzhao
c505c6027e Merge branch 'master-merge-github20230310' 2023-03-10 14:58:34 +08:00
yzhao
f8a6362a22 pre-commit passed 2023-03-10 14:06:38 +08:00
kangxiaoyang.kxy
681afac803 add finetune for ddcolor image colorization 
图像上色模型添加finetune支持，本地测试已通过：python tests/test_image_colorization_trainer.py

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11874339
 2023-03-10 14:00:48 +08:00
yzhao
99e94bc2c2 Merge branch 'master-github' into master-merge-github20230310 2023-03-10 13:52:31 +08:00
james.wjg
1ce28bc4f9 cv/video_instance_segmentation 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11891175
 2023-03-10 10:04:03 +08:00
hemu.zp
429cfee826 Support run text generation pipeline with args 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11937122
 2023-03-10 09:48:10 +08:00
xingjun.wxj
e02a260c93 Refactor the task_datasets module 
Refactor the task_datasets module:

1. Add new module modelscope.msdatasets.dataset_cls.custom_datasets.
2. Add new function: modelscope.msdatasets.ms_dataset.MsDataset.to_custom_dataset().
2. Add calling to_custom_dataset() func in MsDataset.load() to adapt new custom_datasets module.
3. Refactor the pipeline for loading custom dataset: 
	1) Only use MsDataset.load() function to load the custom datasets.
	2) Combine MsDataset.load() with class EpochBasedTrainer.
4. Add new entry func for building datasets in EpochBasedTrainer: see modelscope.trainers.trainer.EpochBasedTrainer.build_dataset()
5. Add new func to build the custom dataset from model configuration, see: modelscope.trainers.trainer.EpochBasedTrainer.build_dataset_from_cfg()
6. Add new registry function for building custom datasets, see: modelscope.msdatasets.dataset_cls.custom_datasets.builder.build_custom_dataset()
7. Refine the class SiameseUIETrainer to adapt the new custom_datasets module.
8. Add class TorchCustomDataset as a superclass for custom datasets classes.
9. To move modules/classes/functions:
	1) Move module msdatasets.audio to custom_datasets
	2) Move module msdatasets.cv to custom_datasets
	3) Move module bad_image_detecting to custom_datasets
	4) Move module damoyolo to custom_datasets
	5) Move module face_2d_keypoints to custom_datasets
	6) Move module hand_2d_keypoints to custom_datasets
	7) Move module human_wholebody_keypoint to custom_datasets
	8) Move module image_classification to custom_datasets
	9) Move module image_inpainting to custom_datasets
	10) Move module image_portrait_enhancement to custom_datasets
	11) Move module image_quality_assessment_degradation to custom_datasets
	12) Move module image_quality_assmessment_mos to custom_datasets
	13) Move class LanguageGuidedVideoSummarizationDataset to custom_datasets
	14) Move class MGeoRankingDataset to custom_datasets
	15) Move module movie_scene_segmentation custom_datasets
	16) Move module object_detection to custom_datasets
	17) Move module referring_video_object_segmentation to custom_datasets
	18) Move module sidd_image_denoising to custom_datasets
	19) Move module video_frame_interpolation to custom_datasets
	20) Move module video_stabilization to custom_datasets
	21) Move module video_super_resolution to custom_datasets
	22) Move class GoproImageDeblurringDataset to custom_datasets
	23) Move class EasyCVBaseDataset to custom_datasets
	24) Move class ImageInstanceSegmentationCocoDataset to custom_datasets
	25) Move class RedsImageDeblurringDataset to custom_datasets
	26) Move class TextRankingDataset to custom_datasets
	27) Move class VecoDataset to custom_datasets
	28) Move class VideoSummarizationDataset to custom_datasets
10. To delete modules/functions/classes:
	1) Del module task_datasets
	2) Del to_task_dataset() in EpochBasedTrainer
	3) Del build_dataset() in EpochBasedTrainer and renew a same name function.
11. Rename class Datasets to CustomDatasets in metainfo.py

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11872747
 2023-03-10 09:03:32 +08:00
yanwen.pyl
fc7daea9c2 add soonet for video temporal grounding 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11810444
 2023-03-10 09:02:39 +08:00
zhangzhicheng.zzc
8a19e9645d [to #47860410]plugin with cli tool 
1. 支持 plugin方式接入外部 repo、github repo，本地repo，并进行外部插件管理
2. 支持allow_remote方式接入modelhub repo，该类型属于model 范畴不做额外插件管理
3. 支持cli 安装plugin相关

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11775456
 2023-03-09 23:07:13 +08:00
hemu.zp
2b1af959d5 Convert cfg during training 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11900238
 2023-03-09 22:27:44 +08:00
jinmao.yk
4078abf488 add human reconstruction task 
单图人体重建任务 

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11778199

 * add human reconstruction task
 2023-03-09 21:58:48 +08:00
dave.ma
f493e33720 vidt模型代码评审 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11873585

* vidt_v0
 2023-03-09 21:52:18 +08:00
wucong.lyb
d9b34daa79 add task: speech_timestamp && fix vad bugs(audio_fs is invalid) 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11940358

* modify punctuation postprocess

* modify audio input types

* add timestamp args

* add task: speech_timestamp

* modidy timestamp outputs

* add args:seg_dict&split_with_space

* feat: Fix conflict, auto commit by WebIDE
 2023-03-09 21:51:04 +08:00
yinyueqin.yyq
b5fe11fea0 upload disco guided diffusion 
把https://huggingface.co/IDEA-CCNL/Taiyi-Diffusion-532M-Cyberpunk-Chinese和https://huggingface.co/IDEA-CCNL/Taiyi-Diffusion-532M-Nature-Chinese迁移到MaaS-lib上。该project基于disco diffusion+guided diffusion。

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11818412

* upload disco guided diffusion
 2023-03-09 21:45:33 +08:00
yuze.zyz
a0bc5549a1 trainer support parallel_groups 
Design doc: https://yuque.alibaba-inc.com/suluyan.sly/yh1rvu/yx0owblyebpa2b3l?singleDoc#flU3s

1. Add parallel_group field in trainer to support DP, TP, PP.
2. Move the construction of common hooks(except optimizer/lrscheduler hook) to trainer's init method to support after_init stage.
	after_init is to support DP, TP, PP's initializing
         https://aone.alibaba-inc.com/v2/workitem#viewIdentifier=1c46ee8637e0c978f115b6f7&openWorkitemIdentifier=48099986
3. Add before_eval/after_eval stage to support model wrapping.
	to solve the order problem of apex amp & ddp wrapping.
         https://aone.alibaba-inc.com/v2/workitem#viewIdentifier=1c46ee8637e0c978f115b6f7&openWorkitemIdentifier=48099986
4. Exporter supports lazy importing.
	https://aone.alibaba-inc.com/v2/workitem#viewIdentifier=1c46ee8637e0c978f115b6f7&openWorkitemIdentifier=48122780
5. Fold all megatron imports to megatron hook.
         https://aone.alibaba-inc.com/v2/workitem#viewIdentifier=1c46ee8637e0c978f115b6f7&openWorkitemIdentifier=48099986
6. Add compile method to TorchModel ,Pipeline,Trainer to support torch2.0
	https://aone.alibaba-inc.com/v2/workitem#viewIdentifier=1c46ee8637e0c978f115b6f7&openWorkitemIdentifier=46869415
7. Fix bug: Lrscheduler builder does not support torch2.0
8. Add callbacks for trainer
	https://aone.alibaba-inc.com/v2/workitem#viewIdentifier=1c46ee8637e0c978f115b6f7&openWorkitemIdentifier=48210342
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11849932
 2023-03-09 21:33:35 +08:00
yuanzhi.zyz
2ce41d4974 add ocr_reco_crnntrain 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11824395
 2023-03-09 21:14:09 +08:00
bin.xue
281d3c630e [to #42322933] feat: kws support continue training from a checkpoint 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11940446

* feat: kws support continue training from a checkpoint

* log: add loading model log
 2023-03-09 21:07:34 +08:00
yuze.zyz
a2bed42fe1 fix a CI case 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11940845
 2023-03-09 19:55:22 +08:00
lllcho.lc
3e70ecfb37 add action detection finetune 
添加action-detection模型finetune支持

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11873651
 2023-03-09 16:54:19 +08:00
Louis
8abfffc7e5 import math to fix NameError (#147) 2023-03-09 15:35:32 +08:00
neo.dzh
86d5c2c5a8 fixbug for speaker_verification_pipeline and speaker_diarization_pipeline: re-write the default config with configure.json 
修复speaker_verification_pipeline和speaker_diarization_pipeline中configuration.json配置不生效的bug：
使用configuration.json中有的参数覆盖default

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11932226
 2023-03-09 15:20:53 +08:00
mulin.lyh
62f769877f [to #48291018]fix: fix data releate case failed bug 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11925637
 2023-03-09 15:18:50 +08:00
xixing.tj
46799325c8 add ocr_detection_db training module 
新增ocr_detection dbnet训练代码

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11892455
 2023-03-09 15:11:27 +08:00
feiyu.gfy
f534fd3ae0 add lore lineness table recognition 
读光无线表格识别（lineless-table-recognition）

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11772358
 2023-03-09 15:09:53 +08:00
tastelikefeet
cf8d2d574f add conversion script for checkpoint files before version 1.3.1. (#161) 
* add conversion script for older checkpoint files.

* fix format
 2023-03-09 14:58:21 +08:00
jiaqi.sjq
8099cc9f60 [to #41669377] Refactoring by using kantts as third party wheel and add ptts finetune&infer support 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11913929
 2023-03-09 11:32:44 +08:00
Patrick von Platen
2bf4ead3fb Correct DDIM sampling on GPU (#173) 2023-03-09 11:21:56 +08:00
ru.he
9307b071b7 Add the code for PEER model 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11873170
 2023-03-09 01:26:27 +08:00