Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2025-12-25 12:39:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
830 Commits 263 Branches 106 Tags
06abae4dc6d68e99cba56608c857de5cdabd16b0
Commit Graph

830 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
zhangzhicheng.zzc
06abae4dc6 [to #42322933]add token-cls test cases and bug fix 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10585502
 2022-11-01 09:56:15 +08:00
mulin.lyh
5302259a0a [to #45854437]fix: add user name to user-agent 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10584797
 2022-10-31 22:46:17 +08:00
Yingda Chen
3464324f6b [to #42322933] limit datasets version for now 2022-10-31 22:15:37 +08:00
zhangzhicheng.zzc
0d3b7b0df2 [to #42322933]fix bugs relate to token cls 
1.修复token classification preprocessor finetune结果错误问题
2.修复word segmentation output 无用属性
3. 修复nlp preprocessor传use_fast错误
4. 修复torch model exporter bug
5. 修复文档撰写过程中发现trainer相关bug
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10573269
 2022-10-31 20:52:27 +08:00
shouzhou.bx
e72988c2ba add face detection to face_2d_keypoints_pipeline 2022-10-31 20:46:49 +08:00
xiaodongdeng.dxd
64868bf2ad [to #42322933]多模态预训练模型OFA增加支持6b模型的feature 
多模态预训练模型OFA增加支持6b模型的feature
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10574571
 2022-10-31 20:42:56 +08:00
yuanzheng.yuanzhen
ce08cfbea8 [to #42322933]Add licenses 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10580553

    * Add licenses
 2022-10-31 18:47:06 +08:00
yichang.zyc
e2d35fbb14 [to #42322933]clip支持finetune 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10572842
 2022-10-30 21:51:11 +08:00
Yingda Chen
9f7b8b86a3 [to #42322933] disble 2dkeypoints training since face_2d_keypoints_dataset is set to be private 2022-10-30 13:59:12 +08:00
Yingda Chen
902019c2e0 [to #42322933] disble vgg19_fer 2022-10-30 13:55:49 +08:00
Yingda Chen
29448c0f57 [to #42322933] disble vit 2022-10-30 11:15:52 +08:00
wenmeng.zwm
e07f3cdbf5 remove fasttext 2022-10-29 15:05:26 +08:00
wenmeng.zwm
ae55fed216 bumpy version to 1.0.0 2022-10-29 14:37:56 +08:00
mulin.lyh
3791ee7ad2 [to #45821936]fix: fix block user specify revision after release_datetime 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10572162
 2022-10-29 13:44:47 +08:00
yuze.zyz
4b7e8e89aa [to #42322933] Fix some bugs when downgrade the version of some dependencies 
1. Fix bug in model exporting
2. Skip some long trainings in test level 2
3. Refine some comments
4. Fix a bug that mode is not correct when saving checkpoints
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10564716
 2022-10-28 21:44:33 +08:00
Yufeng
261c04b8b5 add Mglm (#5) 
* mglm init

* add mglm requirements

Co-authored-by: Yufeng <zhuyufeng@gmail.com>
Co-authored-by: wenmeng.zwm <wenmeng.zwm@alibaba-inc.com>
 2022-10-28 17:12:47 +08:00
lingcai.wl
84ed59d857 [to #44834022] add service utils for model deploy 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10529621
 2022-10-28 16:10:50 +08:00
pangda
303ae2ff36 [to #42322933] fix bug for text logger 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10560149
 2022-10-28 15:26:17 +08:00
Yingda Chen
46cfa177aa [to #42322933]skip timeconsuming test 2022-10-28 09:34:29 +08:00
hemu.zp
53e9f02561 [to #42322933] Fix bug for bleu in text generation metrics. 
修复了使用错误算法导致 BLEU-4 值计算结果偏小的问题
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10558494
 2022-10-28 09:28:15 +08:00
xianzhe.xxz
88e8d4291a [to #42322933]"fix: set the eps and momentum of BN consistent with training" 
To keep consistent between training and evaluation, change the eps and momentum of BN.
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10554451
 2022-10-28 09:27:55 +08:00
menrui.mr
c7b0787049 修复初始化过程参数未生效问题 
此前文生图模型没有加载configuration.json中的参数 影响默认配置
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10558026
 2022-10-27 23:29:08 +08:00
hemu.zp
fa415d8720 [to #42322933] Fix bug for bloom and gpt_neo 
1. 修复 bloom 和 gpt_neo 模型更新 transformers 4.23 后后处理报错的问题
2. 统一使用 ModelOutput 作为模型输出
3. gpt_neo checkpoint 已上线，修改 ut 为 level2
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10553103
 2022-10-27 23:27:28 +08:00
zhangzhicheng.zzc
b713e3de1c [to #42322933]fix token classification bugs 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10550136
 2022-10-27 22:53:16 +08:00
zhangzhicheng.zzc
9df3f5c41f override pipeline by tasks name after finetune done, avoid case like fill mask pipeline with a text cls task 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10554512
 2022-10-27 22:52:29 +08:00
xingjun.wxj
78f29cf999 [to #42322933] Add delete datasets files and upload mode. 
1. Add : MsDataset.delete() , support delete dataset file or dir.

2. Add: upload mode,  MsDataset.upload(xx, upload_mode=UploadMode.FORCE_UPLOAD), or  MsDataset.upload(xx, upload_mode=UploadMode.APPEND_UPLOAD)
     if upload_mode = UploadMode.APPEND_UPLOAD, then skip object in case of this object exists.

3. Add: support reload sts token automatically to avoid expire. (current expiration: 24h)

4. Fix: add cookies in api.py for downloading private datasets.
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10524449
 2022-10-27 20:30:35 +08:00
Yingda Chen
374fd3090e [to #42322933]skip referring video tests since model is private 2022-10-27 20:23:51 +08:00
yuze.zyz
212cf53318 [to #42322933] Fix some bugs 
1. Add F1 score to sequence classification metric
2. Fix a bug that the evaluate method in trainer does not support a pure pytorch_model.bin
3. Fix a bug in evaluation of veco trainer 
4. Add some tips if lr_scheduler in the trainer needs a higher version torch
5. Add some comments
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10532230
 2022-10-27 19:49:21 +08:00
shuying.shu
ddcb57440d [to #42322933]add fine-tune code for referring video object segmentation 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10539423
 2022-10-27 19:43:54 +08:00
mulin.lyh
3b75623be4 [to #45773874]fix: get_model revision=None bug, and hub case occasionally delete test model failed 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10549680
 2022-10-27 17:06:18 +08:00
yichang.zyc
bdadea9791 [to #42322933]fix image_open func, support url 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10548014

    * fix image_open func
 2022-10-27 16:02:44 +08:00
eniac.xcw
8886c3c1ae [to #42322933]fine tune team on caltech-101 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10525413
 2022-10-27 12:00:14 +08:00
yingda.chen
de708dd518 add basic remap column wrapper 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10539917

    * add basic remap column wrapper
 2022-10-27 10:12:05 +08:00
hemu.zp
69104c0f8a [to #42322933] Refactor text generation model outputs and fix some bugs 
1. 将 single_gpu_test 与 multi_gpu_test 中的 model.forward 部分分离为 EpochBasedTrainer 中的 evaluation_step，为部分 evaluation 阶段不调用 forward 的模型提供更好的灵活性
2. 重构代码将文本生成模型 Model 层的输入输出统一为 Tensor，Tensor 到 str 的 decode 过程移动到 pipeline 中完成
3. pipeline 后处理添加对中文和中文标点与英文混杂时空格的处理，使 decode 后中英文混杂输出正确
4. 添加 TextGenerationTrainer 修复了部分模型 evaluation 过程 forward 输出单个 token 计算 metrics 的问题
5. 修复了 rouge 无法接收空字符串的问题
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10473768
 2022-10-27 09:52:05 +08:00
yuze.zyz
f9e12669ba [to #42322933]add default mapping for preprocessors 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10536603
 2022-10-27 09:33:19 +08:00
liugao.lg
0605376135 [to #42322933]add ofa finetune 
新增ofa的finetune能力
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10540701
 2022-10-27 09:29:06 +08:00
hemu.zp
d0f8547e7e [to #42322933] Fix gpt3 loading checkpoint after finetuning. 
1. 修复GPT-3模型无法加载finetune保存的checkpoint的问题
2. 为GPT-3诗词生成模型添加 ut
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10537209
 2022-10-26 20:58:00 +08:00
jiaqi.sjq
7b84adc914 [to #42322933]Fix remove files in local model not take effect to remote repo after push_model 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10533214
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10533214
 2022-10-26 19:15:43 +08:00
hemu.zp
e4a0e046f9 [to #42322933] Add ut for mplug and bloom 
为新上线的 langboat/bloom-1b4-zh，damo/mplug_visual-question-answering_coco_base_zh，damo/mplug_image-captioning_coco_base_zh 三个模型添加 ut，test_level 设置为 2
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10524221
 2022-10-26 16:19:20 +08:00
wenshen.xws
2c994ed760 [to #42322933]fix tokenizer for faq 
多语言faq，Tokenizer新增类型判别
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10530690
 2022-10-26 16:18:27 +08:00
caorongyu.cry
3b8fb92c13 [to #42322933] debug header ids and header names 
修复header_ids和header_names命名反了的问题
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10516557
 2022-10-26 16:04:14 +08:00
ran.zhou
13f7e9ceca [to #42322933]SEA multilingual NLP (NER & word segmentation) 
添加东南亚小语种NLP支持，包括：
1. 针对泰语，越南语NER的预处理
2. 基于XLMR-CRF架构的分词模型和pipeline
3. 针对泰语分词的预处理

添加了相应pipeline的unittest
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10492404
 2022-10-26 14:52:22 +08:00
mulin.lyh
384377b8f5 * [to #45486649]feat: modelscope model version use model repo tag, unsupport branch or commit it, client user-agent header unified 2022-10-26 13:55:51 +08:00
jiaqi.sjq
5190c7de11 [to #41669377] tts using default master revision model in UT 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10526747
 2022-10-26 11:53:52 +08:00
zhangyanzhao.zyz
781fe49d63 [to #42322933]修正finetune text ranking bugs 
之前的finetune代码当dataset最后长度不足制定batch size时会出错，现已修正
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10524066
 2022-10-26 09:44:25 +08:00
yuanzheng.yuanzhen
bab54bbce8 [to #42322933]support uni fold 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10481410
 2022-10-25 22:59:19 +08:00
siyang.ssy
ba3db0f552 [to #42322933] fix video embedding output 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10525516
 2022-10-25 22:56:14 +08:00
tingwei.gtw
d40cc98994 [to #42322933] update IO for demo services 
修改了I/O的代码，以支持modelscope的demo services
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10518318
 2022-10-25 22:49:15 +08:00
yuze.zyz
c2da44b371 [to #42322933] remove dev model inference and fix some bugs 
1. Change structbert dev revision to master revision
2. Fix bug:  Sample code failed because the updating of model configuration
3. Fix bug: Continue training regression failed
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10519992
 2022-10-25 22:38:49 +08:00
lllcho.lc
41b35619e8 [to #42322933] Fix bug for demo service 
在demo service场景，同时调用同一个视频文件，会导致ffmpeg处理同名视频的冲突。通过uuid生成唯一的文件名解决这个冲突。
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10518178
 2022-10-25 20:31:53 +08:00