Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2025-12-25 04:29:22 +01:00
Code Issues Packages Projects Releases Wiki Activity
1,160 Commits 263 Branches 106 Tags
2260dd45fa6fed0f86ecef9b1ca46bdc20457b35
Commit Graph

1160 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
kangxiaoyang.kxy
2260dd45fa 1230-image-colorization 
submit new algorithm for image colorization and corresponding pipeline.
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11179627
 2023-01-05 15:02:49 +08:00
kaisong.sks
2d68c6772a 用户满意度分析推理 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11235766

    * add

* add

* Merge remote-tracking branch 'origin' into nlp_user-satisfaction-estimation

* add

* add

* add

* Merge branch 'master' into nlp_user-satisfaction-estimation

* add

* Merge branch 'master' into nlp_user-satisfaction-estimation

* add

* add

* Merge branch 'master' into nlp_user-satisfaction-estimation

* Merge branch 'master' into nlp_user-satisfaction-estimation

* Merge branch 'master' into nlp_user-satisfaction-estimation

* Merge remote-tracking branch 'origin' into nlp_user-satisfaction-estimation

* add

* Merge branch 'master' into nlp_user-satisfaction-estimation
 2023-01-05 10:58:36 +08:00
liugao.lg
6dd0bea98d ofa-finetune 支持训练推理配置文件一体化 
ofa的finetune的配置文档教程冗余，用户使用copy内容太多，修改复杂，提供简洁的finetune代码流程
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11282842
 2023-01-05 07:25:47 +08:00
changxu.ccx
60bd40742a [to #42322933] Add vldoc to maas lib 
Test
```python
python tests/pipelines/test_document_vl_embedding.py
```
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11199555
 2023-01-04 19:11:30 +08:00
caorongyu.cry
72c39fb161 add space-t trainer 
1. 增加fine-tuning流程
2. 增加evalution流程
3. 关联数据集nlp_convai_text2sql_pretrain_cn_trainset
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11276053

    * add space-t trainer

* revise for trainer

* Merge branch 'master' into dev/tableqa_finetune

* revise for trainer

* Merge remote-tracking branch 'origin' into dev/tableqa_finetune
 2023-01-04 09:46:37 +08:00
pangda
2dbc93a931 [to #42322933] add UT for chunking model 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11289061
 2023-01-04 04:41:00 +08:00
mulin.lyh
ab07dc5b5a google style docs and selected file generator 
ref: https://yuque.alibaba-inc.com/pai/rwqgvl/go8sc8tqzeqqfmsz
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11150212

    * google style docs and selected file generator
 2023-01-03 16:27:29 +08:00
mulin.lyh
0675bd5c88 [to #47031187]fix: hub test suites can not parallel 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11276872

    * [to #47031187]fix: hub test suites can not parallel
 2023-01-03 16:26:59 +08:00
bin.xue
0fdf37312f [to #42322933] feat:add speech separation pipeline 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11255740
 2023-01-03 13:18:44 +08:00
dadong.gxd
01c498cd14 add cv_casmvs_multi-view-depth-esimation_general 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11204285
 2023-01-03 08:24:41 +08:00
shouzhou.bx
4698051fa5 update task of hand detect 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11271647
 2023-01-02 12:14:49 +08:00
james.wjg
a12ef720a6 全景分割easycv接入支持finetune（mask2former-r50）- 12.30 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11179263
 2023-01-02 11:27:11 +08:00
mulin.lyh
41cd220e01 temp skip failed case 2022-12-30 19:24:19 +08:00
lee.lcy
ed28b849eb [to #42322933] add domain specific object detection models 
添加垂类目标检测模型。
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11265502
 2022-12-30 14:19:16 +08:00
hejunjie.hjj
d560291525 [to #42322933] add maskdino model (1230) 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11187240
 2022-12-30 07:43:57 +08:00
wenmeng.zwm
b8ec677739 add training args support and image classification fintune example 
design doc: https://yuque.antfin.com/pai/rwqgvl/khy4uw5dgi39s6ke

usage:
```python

    from modelscope.trainers.training_args import (ArgAttr, MSArgumentParser,
                                               training_args)


    training_args.topk = ArgAttr(cfg_node_name=['train.evaluation.metric_options.topk',
                                                'evaluation.metric_options.topk'],
                                 default=(1,), help='evaluation using topk, tuple format, eg (1,), (1,5)')
    training_args.train_data = ArgAttr(type=str, default='tany0699/cats_and_dogs', help='train dataset')
    training_args.validation_data = ArgAttr(type=str, default='tany0699/cats_and_dogs', help='validation dataset')
    training_args.model_id = ArgAttr(type=str, default='damo/cv_vit-base_image-classification_ImageNet-labels', help='model name')

    parser = MSArgumentParser(training_args)
    cfg_dict = parser.get_cfg_dict()
    args = parser.args
    
    train_dataset = create_dataset(args.train_data, split='train')
    val_dataset = create_dataset(args.validation_data, split='validation')

    def cfg_modify_fn(cfg):
        cfg.merge_from_dict(cfg_dict)
        return cfg

    kwargs = dict(
        model=args.model_id,          # model id
        train_dataset=train_dataset,  # training dataset
        eval_dataset=val_dataset,     # validation dataset
        cfg_modify_fn=cfg_modify_fn     # callback to modify configuration
    )

    trainer = build_trainer(name=Trainers.image_classification, default_args=kwargs)
    # start to train
    trainer.train()
```
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11225071
 2022-12-30 07:35:15 +08:00
xiaodongdeng.dxd
1e4a3dcbe5 add_api_docs_for_ofa 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11097614
 2022-12-30 01:13:21 +08:00
hemu.zp
fd0099c92d [to #42322933] Refactor megatron-util 
Rename import lib 'megatron' to 'megatron_util' and add error message for users when import failed.

Use initialize_megatron as a unified initialization entry in megatron-util, which can accept configuration input of ConfigDict in MaaS-lib.

Wrap the initialization process into the utils/megatron_utils.py file, add default parameters for the existing large model to be compatible with the uploaded configuration file.

The version of megatron_cfg currently supports v3 (default), v1 and moe.
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11231840
 2022-12-29 15:01:45 +08:00
ly261666
6583e6f398 [to #42322933] Add FLIR Face Liveness Model 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11250177
 2022-12-29 14:55:51 +08:00
pengteng.spt
cddebf567f add kws nearfield finetune 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11179425

* add kws nearfield finetune

* work on rank-0 only if evaluating

* split kaldi relevant code into runtime utils

* add evaluate but not files checking

* test evaluate on cpu

* add default value for cmvn_file
 2022-12-29 10:14:41 +08:00
dadong.gxd
42557b0867 add cv_pointnet2_sceneflow-estimation_general 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11201880
 2022-12-29 08:09:57 +08:00
yeqinghao.yqh
f7a7504782 Add HiTeA model for VideoQA and Caption (12.30) 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11201652
 2022-12-29 08:06:34 +08:00
hemu.zp
f58060b140 [to #42322933] add GPT-2 model 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11194200
 2022-12-29 07:59:40 +08:00
lee.lcy
e8a354d226 [to #42322933] add real-time human detection model 
add real-time human detection model
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11191113
 2022-12-29 07:58:45 +08:00
xingjun.wxj
6423445697 [to #42322933] Modify object_exists in oss utils download. 
1. Move self.bucket.object_exists to try-except block. 
        
CR Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11245983
 2022-12-28 19:59:02 +08:00
yichang.zyc
0c79b57fcc support batch infer 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11170755
 2022-12-28 12:17:36 +08:00
hemu.zp
addda1f613 [to #42322933] Fix saved checkpoint can't run with pipeline for gpt3 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11225426
 2022-12-28 06:37:54 +08:00
qianmu.ywh
59b7f411b8 add video depth estimation pipeline 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11189483
 2022-12-28 06:34:02 +08:00
wjq264216
e57424eaf0 [to #42322933] Add ocr-detection-vlpt-pipeline to maas lib 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11213036
 2022-12-28 06:26:15 +08:00
huizheng.hz
cb9f1bfb8d 1230-video-super-resolution 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11090960
 2022-12-28 06:18:50 +08:00
wenmeng.zwm
4ee2ad3617 skip unifold ut due to MMseqs2 api error 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11218838
 2022-12-26 22:37:45 +08:00
wenmeng.zwm
a01e29ceb0 fix check video type cv2.VideoCapture and add unittest 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11181951
 2022-12-26 09:59:59 +08:00
suluyan.sly
086f629fbc [to #42322933]fix a bug for plug inference 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11199968

]
 2022-12-26 02:58:17 +08:00
shuying.shu
048207d79b fix memory leak bug in eval 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11195714
 2022-12-25 08:40:19 +08:00
jiaqi.sjq
147da4b3d9 [to #41669377] tts pipeline change infer result to wav 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11191925
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11191925
 2022-12-24 09:36:02 +08:00
xingjun.wxj
85b9d656eb [to #42322933] Add auto reload sts token and bigdata args 
1. Add auto-reload sts token for oss download function.
2. Improve big-data (e.g. Uni-Fold-Data) downloading performance.

        
CR Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11188490
 2022-12-23 14:40:37 +08:00
shuying.shu
b680691c2c adjust pipeline output 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11136535
 2022-12-23 01:53:17 +08:00
Yingda Chen
c4d116ab53 disable keyword spotting test 2022-12-22 09:49:37 -08:00
hannah.yh
b36bb72869 add image skychange 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10947701
 2022-12-21 17:40:46 +08:00
mulin.lyh
02edb1ab15 [to #46852519]fix: fix useragent string and a trainer invokedby 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11164288

    * [to #46852519]fix: fix useragent string and a trainer invokedby
 2022-12-21 08:28:40 +08:00
hooks.hl
0d205c8322 add support for cv_rdevos_video-object-segmentation 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11066863
 2022-12-21 07:37:27 +08:00
mulin.lyh
9a67e0bb48 [to #46852519]fix: statistics header not correct set 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11160051

    * [to #46852519]fix: statistics header not correct set
 2022-12-20 22:28:28 +08:00
jiaqi.sjq
8896087034 [to #42322933] support kantts infer and finetune 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11111331#tab=detail
 2022-12-20 10:45:34 +08:00
caorongyu.cry
b992bf278c add bi model 
1. 更新模型文件，支持BI能力
2. 更新代码，支持BI能力
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11115454
 2022-12-20 01:51:00 +08:00
jerry.lp
906fa673b4 add gpt-moe model for modelscope finetune 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11085918
 2022-12-17 05:52:57 +08:00
wenmeng.zwm
070ec00720 remove opencv-python from framework requirements and remove easynlp from nlp default requirements 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11119626
 2022-12-16 14:27:23 +08:00
shouzhou.bx
95ede6378e [to #42322933] 1230: add hand detection 2022-12-16 13:24:02 +08:00
guanhu.wgh
82c903121e add license_header to some cv files 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10337361
 2022-12-15 22:24:29 +08:00
wenmeng.zwm
a1688f5775 enhance interface standard and refactor card_detection and face detection pipeline 
1. build preprocessor for pipeline automatically if preprocessor is configed in configuration.json
2. refactor scrfd_detect.py as a standard cv model code
3. refacotr card_detection_pipeline  face_detection_pipeline as standard pipeline code

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11057557
 2022-12-15 14:31:50 +08:00
xingjun.wxj
9e9d06cc9a [to #42322933] Fix timeout issue for uni-fold list_oss_objects api 
1. 下载uni-fold数据集，因为包含百万以上的objects，在调用list_objects接口时需要至少300s，所以全量下载数据集时 有用户反馈报超时错误
2. 目前hub api中因为采用了统一的self.session.get接口，默认timeout被设置为了60s
3. 修改timeout=1800

link:  https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11097756
 2022-12-14 21:21:01 +08:00