Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2025-12-25 04:29:22 +01:00
Code Issues Packages Projects Releases Wiki Activity
1,148 Commits 263 Branches 106 Tags
41cd220e01162b79eddd13820448cf37f69caf34
Commit Graph

1148 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
mulin.lyh
41cd220e01 temp skip failed case 2022-12-30 19:24:19 +08:00
lee.lcy
ed28b849eb [to #42322933] add domain specific object detection models 
添加垂类目标检测模型。
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11265502
 2022-12-30 14:19:16 +08:00
hejunjie.hjj
d560291525 [to #42322933] add maskdino model (1230) 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11187240
 2022-12-30 07:43:57 +08:00
wenmeng.zwm
b8ec677739 add training args support and image classification fintune example 
design doc: https://yuque.antfin.com/pai/rwqgvl/khy4uw5dgi39s6ke

usage:
```python

    from modelscope.trainers.training_args import (ArgAttr, MSArgumentParser,
                                               training_args)


    training_args.topk = ArgAttr(cfg_node_name=['train.evaluation.metric_options.topk',
                                                'evaluation.metric_options.topk'],
                                 default=(1,), help='evaluation using topk, tuple format, eg (1,), (1,5)')
    training_args.train_data = ArgAttr(type=str, default='tany0699/cats_and_dogs', help='train dataset')
    training_args.validation_data = ArgAttr(type=str, default='tany0699/cats_and_dogs', help='validation dataset')
    training_args.model_id = ArgAttr(type=str, default='damo/cv_vit-base_image-classification_ImageNet-labels', help='model name')

    parser = MSArgumentParser(training_args)
    cfg_dict = parser.get_cfg_dict()
    args = parser.args
    
    train_dataset = create_dataset(args.train_data, split='train')
    val_dataset = create_dataset(args.validation_data, split='validation')

    def cfg_modify_fn(cfg):
        cfg.merge_from_dict(cfg_dict)
        return cfg

    kwargs = dict(
        model=args.model_id,          # model id
        train_dataset=train_dataset,  # training dataset
        eval_dataset=val_dataset,     # validation dataset
        cfg_modify_fn=cfg_modify_fn     # callback to modify configuration
    )

    trainer = build_trainer(name=Trainers.image_classification, default_args=kwargs)
    # start to train
    trainer.train()
```
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11225071
 2022-12-30 07:35:15 +08:00
xiaodongdeng.dxd
1e4a3dcbe5 add_api_docs_for_ofa 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11097614
 2022-12-30 01:13:21 +08:00
hemu.zp
fd0099c92d [to #42322933] Refactor megatron-util 
Rename import lib 'megatron' to 'megatron_util' and add error message for users when import failed.

Use initialize_megatron as a unified initialization entry in megatron-util, which can accept configuration input of ConfigDict in MaaS-lib.

Wrap the initialization process into the utils/megatron_utils.py file, add default parameters for the existing large model to be compatible with the uploaded configuration file.

The version of megatron_cfg currently supports v3 (default), v1 and moe.
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11231840
 2022-12-29 15:01:45 +08:00
ly261666
6583e6f398 [to #42322933] Add FLIR Face Liveness Model 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11250177
 2022-12-29 14:55:51 +08:00
pengteng.spt
cddebf567f add kws nearfield finetune 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11179425

* add kws nearfield finetune

* work on rank-0 only if evaluating

* split kaldi relevant code into runtime utils

* add evaluate but not files checking

* test evaluate on cpu

* add default value for cmvn_file
 2022-12-29 10:14:41 +08:00
dadong.gxd
42557b0867 add cv_pointnet2_sceneflow-estimation_general 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11201880
 2022-12-29 08:09:57 +08:00
yeqinghao.yqh
f7a7504782 Add HiTeA model for VideoQA and Caption (12.30) 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11201652
 2022-12-29 08:06:34 +08:00
hemu.zp
f58060b140 [to #42322933] add GPT-2 model 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11194200
 2022-12-29 07:59:40 +08:00
lee.lcy
e8a354d226 [to #42322933] add real-time human detection model 
add real-time human detection model
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11191113
 2022-12-29 07:58:45 +08:00
xingjun.wxj
6423445697 [to #42322933] Modify object_exists in oss utils download. 
1. Move self.bucket.object_exists to try-except block. 
        
CR Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11245983
 2022-12-28 19:59:02 +08:00
yichang.zyc
0c79b57fcc support batch infer 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11170755
 2022-12-28 12:17:36 +08:00
hemu.zp
addda1f613 [to #42322933] Fix saved checkpoint can't run with pipeline for gpt3 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11225426
 2022-12-28 06:37:54 +08:00
qianmu.ywh
59b7f411b8 add video depth estimation pipeline 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11189483
 2022-12-28 06:34:02 +08:00
wjq264216
e57424eaf0 [to #42322933] Add ocr-detection-vlpt-pipeline to maas lib 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11213036
 2022-12-28 06:26:15 +08:00
huizheng.hz
cb9f1bfb8d 1230-video-super-resolution 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11090960
 2022-12-28 06:18:50 +08:00
wenmeng.zwm
4ee2ad3617 skip unifold ut due to MMseqs2 api error 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11218838
 2022-12-26 22:37:45 +08:00
wenmeng.zwm
a01e29ceb0 fix check video type cv2.VideoCapture and add unittest 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11181951
 2022-12-26 09:59:59 +08:00
suluyan.sly
086f629fbc [to #42322933]fix a bug for plug inference 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11199968

]
 2022-12-26 02:58:17 +08:00
shuying.shu
048207d79b fix memory leak bug in eval 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11195714
 2022-12-25 08:40:19 +08:00
jiaqi.sjq
147da4b3d9 [to #41669377] tts pipeline change infer result to wav 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11191925
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11191925
 2022-12-24 09:36:02 +08:00
xingjun.wxj
85b9d656eb [to #42322933] Add auto reload sts token and bigdata args 
1. Add auto-reload sts token for oss download function.
2. Improve big-data (e.g. Uni-Fold-Data) downloading performance.

        
CR Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11188490
 2022-12-23 14:40:37 +08:00
shuying.shu
b680691c2c adjust pipeline output 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11136535
 2022-12-23 01:53:17 +08:00
Yingda Chen
c4d116ab53 disable keyword spotting test 2022-12-22 09:49:37 -08:00
hannah.yh
b36bb72869 add image skychange 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10947701
 2022-12-21 17:40:46 +08:00
mulin.lyh
02edb1ab15 [to #46852519]fix: fix useragent string and a trainer invokedby 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11164288

    * [to #46852519]fix: fix useragent string and a trainer invokedby
 2022-12-21 08:28:40 +08:00
hooks.hl
0d205c8322 add support for cv_rdevos_video-object-segmentation 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11066863
 2022-12-21 07:37:27 +08:00
mulin.lyh
9a67e0bb48 [to #46852519]fix: statistics header not correct set 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11160051

    * [to #46852519]fix: statistics header not correct set
 2022-12-20 22:28:28 +08:00
jiaqi.sjq
8896087034 [to #42322933] support kantts infer and finetune 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11111331#tab=detail
 2022-12-20 10:45:34 +08:00
caorongyu.cry
b992bf278c add bi model 
1. 更新模型文件，支持BI能力
2. 更新代码，支持BI能力
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11115454
 2022-12-20 01:51:00 +08:00
jerry.lp
906fa673b4 add gpt-moe model for modelscope finetune 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11085918
 2022-12-17 05:52:57 +08:00
wenmeng.zwm
070ec00720 remove opencv-python from framework requirements and remove easynlp from nlp default requirements 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11119626
 2022-12-16 14:27:23 +08:00
shouzhou.bx
95ede6378e [to #42322933] 1230: add hand detection 2022-12-16 13:24:02 +08:00
guanhu.wgh
82c903121e add license_header to some cv files 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10337361
 2022-12-15 22:24:29 +08:00
wenmeng.zwm
a1688f5775 enhance interface standard and refactor card_detection and face detection pipeline 
1. build preprocessor for pipeline automatically if preprocessor is configed in configuration.json
2. refactor scrfd_detect.py as a standard cv model code
3. refacotr card_detection_pipeline  face_detection_pipeline as standard pipeline code

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11057557
 2022-12-15 14:31:50 +08:00
xingjun.wxj
9e9d06cc9a [to #42322933] Fix timeout issue for uni-fold list_oss_objects api 
1. 下载uni-fold数据集，因为包含百万以上的objects，在调用list_objects接口时需要至少300s，所以全量下载数据集时 有用户反馈报超时错误
2. 目前hub api中因为采用了统一的self.session.get接口，默认timeout被设置为了60s
3. 修改timeout=1800

link:  https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11097756
 2022-12-14 21:21:01 +08:00
Yingda Chen
f6dcfb11a0 fix card-detection model unregistered error and fix log warning 2022-12-13 13:32:52 -08:00
Yingda Chen
0bf94e5c2d fix build break 2022-12-13 12:46:07 -08:00
yuanzheng.yuanzhen
93f2c52303 fix multimer input for science/protein_structure 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10608748
 2022-12-13 14:18:39 +08:00
james.wjg
9172637ec8 fix demo service && copy license for cv/language_guided_video_summarization 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10985657

    * fix demo service && copy license
 2022-12-13 14:17:09 +08:00
jiangyu.xzy
7aab9ca313 fix demo service dataloader bug 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11049894

    * fix dataloader bug

* fix dataloader bug
 2022-12-10 01:02:36 +08:00
wenmeng.zwm
c8dcdd93da broadcase metric values across all workers for distribution 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10980488
 2022-12-08 10:22:47 +08:00
jinmao.yk
8284d2d366 fix log format to avoid misunderstanding 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11013023
 2022-12-07 19:04:09 +08:00
zhangzhicheng.zzc
92c5abb076 [to #46619305] add kwargs in init method to allow additional kwargs 2022-12-07 18:42:29 +08:00
pangda
f59f9146de fix save_pretrained & load_checkpoint bug in DDP mode 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11012439

    * fix save_pretrained & load_checkpoint bug in DDP mode
 2022-12-07 18:33:00 +08:00
wanggui.hwg
3ce1866224 [to #42322933] Fix bugs for UniTE 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11011725
 2022-12-07 17:23:52 +08:00
mulin.lyh
f1a7ee91c6 [to #46604161]fix: git repository initialize lfs bug 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11005012
 2022-12-07 13:14:22 +08:00
wenmeng.zwm
f0d6d58b17 add project description 2022-12-07 12:37:42 +08:00