Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2026-02-24 20:19:51 +01:00
Code Issues Packages Projects Releases Wiki Activity
979 Commits 267 Branches 107 Tags
fix/linter_failed
Commit Graph

139 Commits

Author SHA1 Message Date
hemu.zp
0f0fdcae6f [to #42322933] Fix bug for mplug evaluation 
修复了 mplug evaluation 使用了错误的 metrics 的问题，将部分中文处理代码独立到 utils 中，为 mplug 添加 trainer
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10612875
 2022-11-08 17:58:03 +08:00
翎航
fd4276ad1a add five finetune task & merge master 2022-11-08 10:57:22 +08:00
翎航
3534970709 add five finetune task & merge master 2022-11-08 10:39:59 +08:00
翎航
a02b2409d8 add five finetune task & merge master 2022-11-08 10:39:45 +08:00
翎航
eb82ba9c6f add finetune & merge master 2022-11-07 20:30:18 +08:00
翎航
0418786cbe add five task finetune 2022-11-07 20:23:17 +08:00
liugao.lg
40b6770956 [to #42322933]fix ocr prepreocess & conflict 
修复ocr预处理逻辑不一致问题
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10581697
 2022-11-01 10:22:11 +08:00
zhangzhicheng.zzc
0d3b7b0df2 [to #42322933]fix bugs relate to token cls 
1.修复token classification preprocessor finetune结果错误问题
2.修复word segmentation output 无用属性
3. 修复nlp preprocessor传use_fast错误
4. 修复torch model exporter bug
5. 修复文档撰写过程中发现trainer相关bug
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10573269
 2022-10-31 20:52:27 +08:00
翎航
11cebb8d64 fix ocr prepreocess & conflict 2022-10-31 17:03:18 +08:00
翎航
3b21ff10ec fix ocr prepreocess 2022-10-31 16:57:49 +08:00
yichang.zyc
e2d35fbb14 [to #42322933]clip支持finetune 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10572842
 2022-10-30 21:51:11 +08:00
Yingda Chen
9f7b8b86a3 [to #42322933] disble 2dkeypoints training since face_2d_keypoints_dataset is set to be private 2022-10-30 13:59:12 +08:00
yuze.zyz
4b7e8e89aa [to #42322933] Fix some bugs when downgrade the version of some dependencies 
1. Fix bug in model exporting
2. Skip some long trainings in test level 2
3. Refine some comments
4. Fix a bug that mode is not correct when saving checkpoints
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10564716
 2022-10-28 21:44:33 +08:00
Yingda Chen
46cfa177aa [to #42322933]skip timeconsuming test 2022-10-28 09:34:29 +08:00
Yingda Chen
374fd3090e [to #42322933]skip referring video tests since model is private 2022-10-27 20:23:51 +08:00
yuze.zyz
212cf53318 [to #42322933] Fix some bugs 
1. Add F1 score to sequence classification metric
2. Fix a bug that the evaluate method in trainer does not support a pure pytorch_model.bin
3. Fix a bug in evaluation of veco trainer 
4. Add some tips if lr_scheduler in the trainer needs a higher version torch
5. Add some comments
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10532230
 2022-10-27 19:49:21 +08:00
shuying.shu
ddcb57440d [to #42322933]add fine-tune code for referring video object segmentation 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10539423
 2022-10-27 19:43:54 +08:00
eniac.xcw
8886c3c1ae [to #42322933]fine tune team on caltech-101 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10525413
 2022-10-27 12:00:14 +08:00
yingda.chen
de708dd518 add basic remap column wrapper 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10539917

    * add basic remap column wrapper
 2022-10-27 10:12:05 +08:00
hemu.zp
69104c0f8a [to #42322933] Refactor text generation model outputs and fix some bugs 
1. 将 single_gpu_test 与 multi_gpu_test 中的 model.forward 部分分离为 EpochBasedTrainer 中的 evaluation_step，为部分 evaluation 阶段不调用 forward 的模型提供更好的灵活性
2. 重构代码将文本生成模型 Model 层的输入输出统一为 Tensor，Tensor 到 str 的 decode 过程移动到 pipeline 中完成
3. pipeline 后处理添加对中文和中文标点与英文混杂时空格的处理，使 decode 后中英文混杂输出正确
4. 添加 TextGenerationTrainer 修复了部分模型 evaluation 过程 forward 输出单个 token 计算 metrics 的问题
5. 修复了 rouge 无法接收空字符串的问题
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10473768
 2022-10-27 09:52:05 +08:00
liugao.lg
0605376135 [to #42322933]add ofa finetune 
新增ofa的finetune能力
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10540701
 2022-10-27 09:29:06 +08:00
翎航
2299f8fa65 fix conflict 2022-10-26 22:41:13 +08:00
翎航
022fa4948a fix ocr-finetune acc 2022-10-26 19:44:54 +08:00
翎航
90d47832c0 add ocr-finetune ned 2022-10-26 11:45:50 +08:00
翎航
c077dea072 add ocr-finetune 2022-10-26 10:52:10 +08:00
zhangyanzhao.zyz
781fe49d63 [to #42322933]修正finetune text ranking bugs 
之前的finetune代码当dataset最后长度不足制定batch size时会出错，现已修正
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10524066
 2022-10-26 09:44:25 +08:00
yuze.zyz
c2da44b371 [to #42322933] remove dev model inference and fix some bugs 
1. Change structbert dev revision to master revision
2. Fix bug:  Sample code failed because the updating of model configuration
3. Fix bug: Continue training regression failed
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10519992
 2022-10-25 22:38:49 +08:00
yichang.zyc
6ddafb3218 [to #42322933]caption finetune done, add belu metric 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10318299
 2022-10-25 12:55:41 +08:00
yuze.zyz
605cd7f44a [to #42322933] NLP 1030 Refactor 
Features:
1. Refactor the directory structure of nlp models. All model files are placed into either the model folder or the task_model folder
2. Refactor all the comments to google style
3. Add detail comments to important tasks and nlp models, to list the description of the model, and its preprocessor&trainer
4. Model Exporting now supports a direct all to TorchModelExporter(no need to derive from it)
5. Refactor model save_pretrained method to support direct running(independent from trainer)
6. Remove the judgement of Model in the pipeline base class, to support outer register models running in our pipelines
7. Nlp trainer now has a NLPTrainingArguments class , user can pass arguments into the dataclass, and use it as a normal cfg_modify_fn, to simplify the operation of modify cfg.
8. Merge the BACKBONES and the MODELS, so user can get a backbone with the Model.from_pretrained call
9. Model.from_pretrained now support a task argument, so user can use a backbone and load it with a specific task class.
10. Support Preprocessor.from_pretrained method
11. Add standard return classes to important nlp tasks, so some of the pipelines and the models are independent now, the return values of the models will always be tensors, and the pipelines will take care of the conversion to numpy and the following stuffs.
12. Split the file of the nlp preprocessors, to make the dir structure more clear.

Bugs Fixing:
1. Fix a bug that lr_scheduler can be called earlier than the optimizer's step
2. Fix a bug that the direct call of Pipelines (not from pipeline(xxx)) throws error
3. Fix a bug that the trainer will not call the correct TaskDataset class
4. Fix a bug that the internal loading of dataset will throws error in the trainer class
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10490585
 2022-10-25 12:26:25 +08:00
bin.xue
525fa3ea89 [to #42322933]test: use 'master' branch in training test 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10509580
 2022-10-25 12:10:07 +08:00
行嗔
cc8b78eac8 update rdrop 2022-10-25 11:55:37 +08:00
行嗔
0c64d3fca5 Merge remote-tracking branch 'origin/ofa/finetune_loss' into ofa/finetune 
# Conflicts:
#	tests/trainers/test_ofa_trainer.py
 2022-10-25 11:48:03 +08:00
翎航
1bb1eeec77 fix ut 2022-10-25 10:55:24 +08:00
行嗔
2288a0fdf3 fix all comments 2022-10-25 10:18:33 +08:00
行嗔
df5bd86048 fix a ut bug 2022-10-25 10:15:06 +08:00
行嗔
9e3f035fa7 fix a ut bug 2022-10-25 10:13:48 +08:00
行嗔
d5b2dabaf5 Merge remote-tracking branch 'origin/master' into ofa/finetune 2022-10-25 10:13:02 +08:00
huizheng.hz
a1738690c9 [to #42322933]test_image_denoise_trainer 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10465138
 2022-10-25 10:08:57 +08:00
yingda.chen
de7b6a06e9 [to #42322933] remove revision usage for face detection 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10507910

    * [to #42322933] remove revision usage for face detection
 2022-10-25 09:28:01 +08:00
行嗔
cb9a2b9d10 Merge remote-tracking branch 'origin/master' into ofa/finetune 2022-10-25 00:52:52 +08:00
行嗔
85a7832d57 fix a typo 2022-10-25 00:52:35 +08:00
zhangyanzhao.zyz
c4dbb69d65 [to #42322933]增加对text-ranking任务中文模型的单元测试，以方便得到官方模型打标。 
增加对text-ranking任务中文模型的单元测试，以方便得到官方模型打标。
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10492754
 2022-10-24 23:41:20 +08:00
行嗔
46c3bdcfe8 fix a bug 2022-10-24 23:19:23 +08:00
翎航
73469b8400 fix loss&log 2022-10-24 19:44:04 +08:00
行嗔
1682ea7dec fix local path 2022-10-24 18:07:18 +08:00
行嗔
55fb3b05a9 format finetune code, and ut case 2022-10-23 21:29:17 +08:00
行嗔
44026e0bf8 Merge remote-tracking branch 'origin/master' into ofa/finetune 2022-10-23 10:36:46 +08:00
yuze.zyz
707cbef013 [to #42322933]Fix bug in daily UT 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10491891
 2022-10-22 23:25:18 +08:00
zhangyanzhao.zyz
824ee8232c [to #42322933]更新语义相关性任务英文名称为text ranking，修改对应变量名和类名 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10491951
 2022-10-22 17:12:48 +08:00
Yingda Chen
dee93c40e2 [to #42322933] force download dataset for portraint enhancement 2022-10-22 16:28:30 +08:00