Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2025-12-25 12:39:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
1,162 Commits 263 Branches 106 Tags
1ec601aea2333d2e667b2c1728ea12edd683080b
Commit Graph

668 Commits

Author SHA1 Message Date
mulin.lyh
1ec601aea2 skip tests/trainers/test_dialog_intent_trainer.py for list model file 500 error 2023-01-06 09:15:50 +08:00
ni.chongjia
ac53ce3e36 modify format of itn_pipeline 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11257394

    * dev for asr itn inference pipeline

* add task interface

* add pipeline input

* add modemodelscope/pipelines/audio/itn_inference_pipeline.py

* add modelscope/pipelines/audio/itn_inference_pipeline.py

* modelscope/pipelines/audio/itn_inference_pipeline.py

* update modelscope/pipelines/audio/itn_inference_pipeline.py

* modify itn_inference_pipeline.py

* modify itn_inference_pipeline.py

* modify itn_inference_pipeline.py

* remove itn.py

* modify some names

* add modify itn_inference_pipeline.py

* modify itn_inference_pipeline.py

* modify itn_inference_pipeline.py

* modify itn_inference_pipeline.py

* modify itn

* add tests/pipelines/test_inverse_text_processing.py

* modify asr_inference_pipeline.py for the original files

* modify format

* add commits files

* Merge remote-tracking branch 'origin' into remotes/origin/asr/itn_nichongjia

* Merge remote-tracking branch 'origin' into remotes/origin/asr/itn_nichongjia

* modify the pipelines

* Merge branch 'master' into remotes/origin/asr/itn_nichongjia

* [to #47031187]fix: hub test suites can not parallel 
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11276872

    * [to #47031187]fix: hub test suites can not parallel

* google style docs and selected file generator 

ref: https://yuque.alibaba-inc.com/pai/rwqgvl/go8sc8tqzeqqfmsz
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11150212

    * google style docs and selected file generator

* merge

* Merge remote-tracking branch 'origin' into remotes/origin/asr/itn_nichongjia

* Merge branch 'master' into remotes/origin/asr/itn_nichongjia

* add requirements for fun_text_processing
 2023-01-05 16:36:17 +08:00
kangxiaoyang.kxy
2260dd45fa 1230-image-colorization 
submit new algorithm for image colorization and corresponding pipeline.
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11179627
 2023-01-05 15:02:49 +08:00
kaisong.sks
2d68c6772a 用户满意度分析推理 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11235766

    * add

* add

* Merge remote-tracking branch 'origin' into nlp_user-satisfaction-estimation

* add

* add

* add

* Merge branch 'master' into nlp_user-satisfaction-estimation

* add

* Merge branch 'master' into nlp_user-satisfaction-estimation

* add

* add

* Merge branch 'master' into nlp_user-satisfaction-estimation

* Merge branch 'master' into nlp_user-satisfaction-estimation

* Merge branch 'master' into nlp_user-satisfaction-estimation

* Merge remote-tracking branch 'origin' into nlp_user-satisfaction-estimation

* add

* Merge branch 'master' into nlp_user-satisfaction-estimation
 2023-01-05 10:58:36 +08:00
changxu.ccx
60bd40742a [to #42322933] Add vldoc to maas lib 
Test
```python
python tests/pipelines/test_document_vl_embedding.py
```
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11199555
 2023-01-04 19:11:30 +08:00
caorongyu.cry
72c39fb161 add space-t trainer 
1. 增加fine-tuning流程
2. 增加evalution流程
3. 关联数据集nlp_convai_text2sql_pretrain_cn_trainset
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11276053

    * add space-t trainer

* revise for trainer

* Merge branch 'master' into dev/tableqa_finetune

* revise for trainer

* Merge remote-tracking branch 'origin' into dev/tableqa_finetune
 2023-01-04 09:46:37 +08:00
pangda
2dbc93a931 [to #42322933] add UT for chunking model 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11289061
 2023-01-04 04:41:00 +08:00
mulin.lyh
ab07dc5b5a google style docs and selected file generator 
ref: https://yuque.alibaba-inc.com/pai/rwqgvl/go8sc8tqzeqqfmsz
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11150212

    * google style docs and selected file generator
 2023-01-03 16:27:29 +08:00
mulin.lyh
0675bd5c88 [to #47031187]fix: hub test suites can not parallel 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11276872

    * [to #47031187]fix: hub test suites can not parallel
 2023-01-03 16:26:59 +08:00
bin.xue
0fdf37312f [to #42322933] feat:add speech separation pipeline 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11255740
 2023-01-03 13:18:44 +08:00
dadong.gxd
01c498cd14 add cv_casmvs_multi-view-depth-esimation_general 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11204285
 2023-01-03 08:24:41 +08:00
shouzhou.bx
4698051fa5 update task of hand detect 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11271647
 2023-01-02 12:14:49 +08:00
james.wjg
a12ef720a6 全景分割easycv接入支持finetune（mask2former-r50）- 12.30 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11179263
 2023-01-02 11:27:11 +08:00
mulin.lyh
41cd220e01 temp skip failed case 2022-12-30 19:24:19 +08:00
lee.lcy
ed28b849eb [to #42322933] add domain specific object detection models 
添加垂类目标检测模型。
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11265502
 2022-12-30 14:19:16 +08:00
hejunjie.hjj
d560291525 [to #42322933] add maskdino model (1230) 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11187240
 2022-12-30 07:43:57 +08:00
wenmeng.zwm
b8ec677739 add training args support and image classification fintune example 
design doc: https://yuque.antfin.com/pai/rwqgvl/khy4uw5dgi39s6ke

usage:
```python

    from modelscope.trainers.training_args import (ArgAttr, MSArgumentParser,
                                               training_args)


    training_args.topk = ArgAttr(cfg_node_name=['train.evaluation.metric_options.topk',
                                                'evaluation.metric_options.topk'],
                                 default=(1,), help='evaluation using topk, tuple format, eg (1,), (1,5)')
    training_args.train_data = ArgAttr(type=str, default='tany0699/cats_and_dogs', help='train dataset')
    training_args.validation_data = ArgAttr(type=str, default='tany0699/cats_and_dogs', help='validation dataset')
    training_args.model_id = ArgAttr(type=str, default='damo/cv_vit-base_image-classification_ImageNet-labels', help='model name')

    parser = MSArgumentParser(training_args)
    cfg_dict = parser.get_cfg_dict()
    args = parser.args
    
    train_dataset = create_dataset(args.train_data, split='train')
    val_dataset = create_dataset(args.validation_data, split='validation')

    def cfg_modify_fn(cfg):
        cfg.merge_from_dict(cfg_dict)
        return cfg

    kwargs = dict(
        model=args.model_id,          # model id
        train_dataset=train_dataset,  # training dataset
        eval_dataset=val_dataset,     # validation dataset
        cfg_modify_fn=cfg_modify_fn     # callback to modify configuration
    )

    trainer = build_trainer(name=Trainers.image_classification, default_args=kwargs)
    # start to train
    trainer.train()
```
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11225071
 2022-12-30 07:35:15 +08:00
hemu.zp
fd0099c92d [to #42322933] Refactor megatron-util 
Rename import lib 'megatron' to 'megatron_util' and add error message for users when import failed.

Use initialize_megatron as a unified initialization entry in megatron-util, which can accept configuration input of ConfigDict in MaaS-lib.

Wrap the initialization process into the utils/megatron_utils.py file, add default parameters for the existing large model to be compatible with the uploaded configuration file.

The version of megatron_cfg currently supports v3 (default), v1 and moe.
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11231840
 2022-12-29 15:01:45 +08:00
ly261666
6583e6f398 [to #42322933] Add FLIR Face Liveness Model 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11250177
 2022-12-29 14:55:51 +08:00
pengteng.spt
cddebf567f add kws nearfield finetune 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11179425

* add kws nearfield finetune

* work on rank-0 only if evaluating

* split kaldi relevant code into runtime utils

* add evaluate but not files checking

* test evaluate on cpu

* add default value for cmvn_file
 2022-12-29 10:14:41 +08:00
dadong.gxd
42557b0867 add cv_pointnet2_sceneflow-estimation_general 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11201880
 2022-12-29 08:09:57 +08:00
yeqinghao.yqh
f7a7504782 Add HiTeA model for VideoQA and Caption (12.30) 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11201652
 2022-12-29 08:06:34 +08:00
hemu.zp
f58060b140 [to #42322933] add GPT-2 model 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11194200
 2022-12-29 07:59:40 +08:00
lee.lcy
e8a354d226 [to #42322933] add real-time human detection model 
add real-time human detection model
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11191113
 2022-12-29 07:58:45 +08:00
yichang.zyc
0c79b57fcc support batch infer 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11170755
 2022-12-28 12:17:36 +08:00
qianmu.ywh
59b7f411b8 add video depth estimation pipeline 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11189483
 2022-12-28 06:34:02 +08:00
wjq264216
e57424eaf0 [to #42322933] Add ocr-detection-vlpt-pipeline to maas lib 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11213036
 2022-12-28 06:26:15 +08:00
huizheng.hz
cb9f1bfb8d 1230-video-super-resolution 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11090960
 2022-12-28 06:18:50 +08:00
wenmeng.zwm
4ee2ad3617 skip unifold ut due to MMseqs2 api error 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11218838
 2022-12-26 22:37:45 +08:00
wenmeng.zwm
a01e29ceb0 fix check video type cv2.VideoCapture and add unittest 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11181951
 2022-12-26 09:59:59 +08:00
shuying.shu
048207d79b fix memory leak bug in eval 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11195714
 2022-12-25 08:40:19 +08:00
jiaqi.sjq
147da4b3d9 [to #41669377] tts pipeline change infer result to wav 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11191925
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11191925
 2022-12-24 09:36:02 +08:00
Yingda Chen
c4d116ab53 disable keyword spotting test 2022-12-22 09:49:37 -08:00
hannah.yh
b36bb72869 add image skychange 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10947701
 2022-12-21 17:40:46 +08:00
hooks.hl
0d205c8322 add support for cv_rdevos_video-object-segmentation 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11066863
 2022-12-21 07:37:27 +08:00
jiaqi.sjq
8896087034 [to #42322933] support kantts infer and finetune 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11111331#tab=detail
 2022-12-20 10:45:34 +08:00
caorongyu.cry
b992bf278c add bi model 
1. 更新模型文件，支持BI能力
2. 更新代码，支持BI能力
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11115454
 2022-12-20 01:51:00 +08:00
jerry.lp
906fa673b4 add gpt-moe model for modelscope finetune 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11085918
 2022-12-17 05:52:57 +08:00
shouzhou.bx
95ede6378e [to #42322933] 1230: add hand detection 2022-12-16 13:24:02 +08:00
wenmeng.zwm
a1688f5775 enhance interface standard and refactor card_detection and face detection pipeline 
1. build preprocessor for pipeline automatically if preprocessor is configed in configuration.json
2. refactor scrfd_detect.py as a standard cv model code
3. refacotr card_detection_pipeline  face_detection_pipeline as standard pipeline code

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11057557
 2022-12-15 14:31:50 +08:00
yuanzheng.yuanzhen
93f2c52303 fix multimer input for science/protein_structure 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10608748
 2022-12-13 14:18:39 +08:00
wenmeng.zwm
c8dcdd93da broadcase metric values across all workers for distribution 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10980488
 2022-12-08 10:22:47 +08:00
wanggui.hwg
3ce1866224 [to #42322933] Fix bugs for UniTE 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11011725
 2022-12-07 17:23:52 +08:00
shiyi.zxh
c3a494e46d [to #42322933] 
enable finetune of ofa-mmspeech 
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10981972
 2022-12-06 20:58:49 +08:00
xuanjie.wxb
a3a942352e support prompt ner 
修改preprocessor增加对prompt模型的支持。
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10972542
 2022-12-06 10:39:37 +08:00
ly261666
ed23d460d5 [to #42322933] Add facial landmark confidence model 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10780109
 2022-12-05 21:07:35 +08:00
ly261666
262f738460 [to #42322933] add FairFace face attribute model 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10726376
 2022-12-05 17:20:48 +08:00
ly261666
9d43823f36 [to #42322933] add TinyMogFace face detector 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10792564
 2022-12-05 12:01:26 +08:00
baiguan.yt
ce0480f7ed update image-portait-enhancement trainer 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10717891
 2022-12-05 11:43:52 +08:00
hemu.zp
941dbe75cf [to #42322933] Add GPT-3 tensor parallel finetuning 
Add GPT-3 tensor parallel finetuning, adjust some distributed codes to make tensor and data parallel compatible.
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10949507
 2022-12-05 10:01:32 +08:00