行嗔
5d83f62312
mnli finetune done
2022-09-21 15:52:10 +08:00
wenmeng.zwm
6808e9a301
[to #44902099 ] add license for framework files
...
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10189613
2022-09-20 17:49:31 +08:00
yuze.zyz
4cdd0c23eb
[to #42322933 ] Refactor and fix some bugs
...
1. Fix a bug in trainer's progress bar
2. Fix a bug that trainer does not support dataset in config file
3. Add feature: support go on training via checkpoint file
4. Add feature: support fixed filename when saving best checkpoint
5. Fix a bug that no id2label in config file after finetune of nlp models
6. Fix some other bugs
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10138906
2022-09-19 17:05:35 +08:00
wenmeng.zwm
7fb25d7bbb
[to #42322933 ]fix UT error for 830 version
...
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10142442
2022-09-16 22:42:39 +08:00
jiangnana.jnn
adee5d10ae
update EasyCV MsDataset
...
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10103248
* update EasyCV MSDataset
2022-09-15 18:11:03 +08:00
dingkun.ldk
54e1a6d88b
[to #42322933 ]830NLP 篇章排序/文本表示模型代码check
...
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9856179
2022-09-10 15:59:56 +08:00
ly119399
b41b10f897
[to #42322933 ] space finetune on generation task
...
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10061562
2022-09-09 14:27:08 +08:00
jiangnana.jnn
5e176da3a1
adapt to msdataset for EasyCV
...
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9935664
* adapt to msdataset for EasyCV
2022-09-09 10:01:51 +08:00
cyc385202
d4759e4c24
[to #42322933 ] 加入space模型在banking数据集上的finetune代码
...
加入space模型在banking数据集上的微调代码
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10006792
2022-09-08 13:45:14 +08:00
yuze.zyz
01e768503c
[to #42322933 ] Fix random seed for trainer
...
1. Fix random seed for trainer and init it at the first line of init
2. Add a regress test for fixed training
3. Change the dataset 'dureader_robust_qg' to 'DuReader_robust-QG'
4. Change some datasets from loading hf.datasets to loading msdataset.load
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10029509
2022-09-06 15:13:14 +08:00
行嗔
5cf6910bed
Merge remote-tracking branch 'origin/master' into ofa/finetune
2022-09-05 20:48:41 +08:00
hemu.zp
3d3f9b4537
[to #42322933 ] fix checkpoint format
...
1. 修复 palm，gpt3，mplug 模型存在的 finetune 后保存 checkpoint 与原有 checkpoint key 字段存在区别无法使用 from_pretrained 导入的问题
2. 调整 test_finetune_mplug.py 为只保存训练结束时的 checkpoint，减少 ci 耗时
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10016517
2022-09-05 17:51:22 +08:00
bin.xue
b870e4eed5
[to #42322933 ] test: use custom config to reduce test time
...
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10011826
、
2022-09-05 13:26:30 +08:00
yuze.zyz
a9c14e4ead
[to #42322933 ] Support saving the best checkpoint for inference
...
1. Support saving the best checkpoint for inference
2. Fix a bug that _max_iters field does not exist in trainer
3. Fix a bug that function in lambda_lr field cannot be saved to file
4. Fix a bug that save_pretrained would not be called by iterating
5. Fix a bug that interval is not passed from BestCkptHook's init
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9972765
2022-09-05 11:07:48 +08:00
行嗔
db534fe946
add hf dataset
2022-09-04 16:08:25 +08:00
Yingda Chen
39a309b655
[to #42322933 ] reduce train epoch from 3 to w
2022-09-03 12:18:52 +08:00
jiangnana.jnn
1a22fa0222
fix trainer unittest
...
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9970626
* fix trainer unittest
2022-09-02 14:06:08 +08:00
行嗔
c546f2a8b9
Merge remote-tracking branch 'origin/master' into ofa/finetune
...
# Conflicts:
# modelscope/preprocessors/multi_modal.py
2022-09-01 18:08:58 +08:00
行嗔
a3aee4bec2
test finetune
2022-09-01 18:02:07 +08:00
shuying.shu
a9deb3895c
[to #42322933 ] movie scene segmentation模型接入
...
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9872869
2022-08-31 20:54:20 +08:00
mulin.lyh
12698b31a0
[to #44340132 ] fix: ci case run out of gpu memory
2022-08-30 17:59:15 +08:00
feiwu.yfw
2b64cf2bb6
[to #42322933 ]支持从dataset json文件中获取参数
...
* dataset json file add args
2022-08-30 15:15:15 +08:00
yuze.zyz
88d0804dcd
[to #42322933 ] Add S4: child-tuning
...
1. add child-tuning optimizer and ut
2. fix a training bug which can cause interruption after cross-evaluation
3. move model.params from cfg to default args in build_optimizer to prevent the saving of params in save_pretrained
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9891963
2022-08-30 10:52:07 +08:00
hemu.zp
a9089570e5
[to #42322933 ] Add mplug retrieval pipeline and finetune
...
支持 MPLUG 模型 image-text-retrieval 任务的 pipeline 和 finetune
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9919955
2022-08-30 10:50:52 +08:00
bin.xue
3e66244c0d
[to #42322933 ] Add ANS trainer
...
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9892528
2022-08-30 10:23:22 +08:00
feiwu.yfw
39485426e7
[to #42322933 ]:fix msdataset
...
* 修复了zip文件不同打包模式下返回路径错误问题。
* 修复了替换了数据集文件重新下载时校验失败问题。
* 修复dataset oss文件在 REUSE 模式下重复下载的问题。
* 修复了csv数据集的meta json文件中某个split的meta和file字段都为''时加载所有split失败的问题。
* 修复了不同版本datasets路径不一致的问题。
2022-08-26 22:41:13 +08:00
hemu.zp
20a935d406
[to #42322933 ] add gpt3 base finetune
...
添加 gpt3 中小模型单机单卡下的 finetune 代码
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9899004
2022-08-26 14:54:45 +08:00
jiangnana.jnn
930d55d9ad
support EasyCV framework and add Segformer model
...
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9781849
* support EasyCV
2022-08-26 13:58:50 +08:00
hemu.zp
b92e2ca0a0
[to #42322933 ] add vqa and caption finetuning for mplug
...
添加 mplug 模型 caption 及 vqa 任务的 finetuning 支持
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9858028
2022-08-25 21:26:51 +08:00
zhangzhicheng.zzc
b94bb74f66
[to #42322933 ]Add model.save_pretrained method and allow finetune results used by pipeline
2022-08-24 21:39:08 +08:00
james.wjg
5f326e46f3
cv/video_summarization
...
新增video summarization模型的inference和finetune
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9840532
2022-08-24 13:41:47 +08:00
feiwu.yfw
35548bd492
[to #43875101 ]
...
msdataset add coco dataset
unify taskdataset and ms dataset
fix hf datasets
2022-08-17 22:51:22 +08:00
jiangnana.jnn
cfc3d1eed7
fix trainer about iters_per_epoch
...
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9791200
* fix trainer about iters_per_epoch
2022-08-17 20:06:25 +08:00
ya235025
4c08bd752a
[to #42322933 ]更新中文CLIP模型inference代码
...
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9763330
2022-08-17 18:40:55 +08:00
jiangnana.jnn
76482cc3ea
[to #43850241 ] fix processor and collate_fn
...
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9644184
* fix ditributed training and eval
2022-08-16 12:04:07 +08:00
yichang.zyc
5cbd9ff8fd
[to #42322933 ]replace clip modelid
...
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9691512
2022-08-09 17:31:30 +08:00
zhangzhicheng.zzc
0874089f6c
change sentiment-classification branch
...
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9662406
* bug fix for nlp backbone-head trainers
2022-08-06 22:00:26 +08:00
xiangpeng.wxp
a41de5e80e
[to #42322933 ]Merge request from 鹏程:nlp_translation_finetune
...
* csanmt finetune wxp
2022-08-05 23:48:46 +08:00
jiangnana.jnn
6f5b864735
[to #43850241 ] fix unittest
...
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9660779
* fix unittest
2022-08-05 18:39:59 +08:00
yuze.zyz
794e277270
[to #42322933 ] Fix problem in ws
...
1. remove comments
2. fix a bug that ws assert failure for english input
3. add an english input test for ws
3. remove a test case which the dataset can not be visited by outer website
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9657140
2022-08-05 16:18:29 +08:00
zhangzhicheng.zzc
9d0b38b4e4
[to #42322933 ] lazy load on trainer
2022-08-04 14:07:14 +08:00
hemu.zp
75da5fd344
[to #42322933 ] Remove beta revision
...
移除测试中 damo/nlp_palm2.0_text-generation_english-base beta 分支的使用
2022-08-04 10:54:28 +08:00
yuze.zyz
064f1041a9
[to #42322933 ] Refine the nlp examples of finetuning
...
1. Refine the nlp finetune examples
2. Remove some useless code
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9631158
2022-08-03 20:31:10 +08:00
yuze.zyz
21fa71baf0
[to #42322933 ] add/refactor nlp models source code and finetune
...
1. add sbert,veco,palm,space source code
2. support sbert sequence classification, token classification finetune
3. support veco sequence classification finetune
4. support palm nlg finetune
evaluation result: https://sheet.alibaba-inc.com/#/sheet/f7fdcc7f22bd5105 sheet:Maas
5. add ut for finetunes
6. add veco's taskdataset processor
7. add a common trainer for nlp, and a specific trainer for veco
8. merge some duplicate codes of models, preprocessors, pipelines
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9574105
* add basic class of hook&metrics
* pre-commit passed
* change some comments
* pre commit passed
* 1. remove accuracy's groups 2. remove useless hooks 3. simplify priorities
* pre-commit passed
* fix a comment
* Merge branch 'master' into finetune_hooks_metrics
# Conflicts:
# modelscope/metainfo.py
* pre-commit passed
* add basic class of hook&metrics
* pre-commit passed
* change some comments
* pre commit passed
* 1. remove accuracy's groups 2. remove useless hooks 3. simplify priorities
* pre-commit passed
* fix a comment
* Merge branch 'feat/finetune' of gitlab.alibaba-inc.com:Ali-MaaS/MaaS-lib into feat/finetune
* mv hooks related to modelscope/trainers/hooks
* mv priority back
* add torch mdoel base and test
* update hooks, trainer, import_util
* add torch epoch based trainer and dis utils
* add hooks
* fix warmup
* format code stype and fix warmup and add warmup unittest
* fix impls
* pre-commit check passed
* update hook and add EpochBasedTrainer
* add trainer unittest
* Merge branch 'feat/add_hooks' into feat/add_task
# Conflicts:
# modelscope/models/base_torch.py
# modelscope/trainers/hooks/hook.py
# modelscope/trainers/trainer.py
* update unittest name
* rewrite taskdataset to trainer
* fix trainer and add unittest
* add unittest
* code: run to forward
* run through... but ugly code
* arrange some cls
* fix some errs
* revert some mistakes
* init check in
* Merge branch 'feat/add_hooks' into feat/add_task
# Conflicts:
# modelscope/trainers/trainer.py
* test with bigger epoch and size
* add the default metrics class
* move build metrics code to a method
* merge add_task
* merge origin add_task
* add device initialization
* remove preprocessor arg for bool
* add task models
* move metric collect logic to metrics class
* pre-commit passed
* fix cr comments
* precommit passed
* add task models
* Merge remote-tracking branch 'origin/feat/add_task' into feat/backbone_head
* add comment
* change comment formats.
* fix comments
* fix ut bug
* fix comments
* add wrapper check
* fix comments
* pre commit passed
* fix cr comments
* solve a loop import problem
* fix ut bug
* fix ut errors
* change dummydataset to msdataset
* precommit passed
* merge add task
* backbone-head is build, model is not correctly loaded
* model load states matched
* result matched
* lint
* add veco/palm_v2 code
* merge master
* merge master success running
* add repr model name level
* Merge branch 'feat/veco_palm' into feat/finetune_sbert_veco
* model test for training
* add token-classification metric add formal ut
* fix running bug
* finetune and pipeline are working with backbone-head
* add nli
* add missing code
* finetune and pipeline are working with backbone-head
* Merge branch 'feat/backbone_head' of http://gitlab.alibaba-inc.com/Ali-MaaS/MaaS-lib into feat/backbone_head
* add a test repo for pr
* remove merge conflicted file
* remove merge conflicted file 1
* lint check
* import error
* none type bug fix
* forward input unpacking or dict bug
* move head into models, add build_backbone with registry, no base method
* merge master
* feat: 1. add interleave dataset method 2. support multiple dataset in trainer.build_dataset 3. support 3 sub tasks in sequence_classification task
* unfinished
* update the task model structure in NLP field
* merge master
* update by comments
* keep the default model id as current on production
* unfinished
* unfinished
* veco can run
* Merge remote-tracking branch 'origin/master' into feat/backbone_head
* add taskmodel for module management
* remove forward_input_is_dict
* unfinished
* token classification started
* update base model structure
* move space to backbone
* remove 'type' in build_from_cfg method
* test update
* bug fix
* on tesing, mess code
* Merge branch 'feat/backbone_head' into feat/refactor_nlp_730
# Conflicts:
# modelscope/metrics/builder.py
# modelscope/models/__init__.py
# modelscope/models/nlp/__init__.py
# modelscope/preprocessors/nlp.py
# modelscope/trainers/trainer.py
# requirements/multi-modal.txt
* add missing merge
* add sofa source code
* refactor
* add veco task dataset
* add veco task dataset
* pre-commit passed
* fix bug of log
* add some features
* merge master
* bug fix
* refine nlp models
* fix the training error
* unfinished
* refactor pipeline
* Merge branch 'feat/backbone_head' into feat/refactor_nlp_730
# Conflicts:
# modelscope/metrics/builder.py
# modelscope/models/nlp/__init__.py
# modelscope/models/nlp/backbones/structbert/modeling_sbert.py
# modelscope/models/nlp/palm_v2/palm_for_text_generation.py
# modelscope/preprocessors/base.py
# modelscope/preprocessors/nlp.py
# modelscope/trainers/trainer.py
* Merge commit 'ab04ceafc5453ce7daa9aa09e37a55f703072a10' into feat/refactor_nlp_730
# Conflicts:
# modelscope/metainfo.py
# modelscope/metrics/builder.py
# modelscope/models/__init__.py
# modelscope/models/base/base_torch_model.py
# modelscope/models/nlp/__init__.py
# modelscope/models/nlp/backbones/space/model/intent_unified_transformer.py
# modelscope/models/nlp/backbones/space/model/model_base.py
# modelscope/models/nlp/palm_v2/palm_for_text_generation.py
# modelscope/models/nlp/sbert_for_sequence_classification.py
# modelscope/models/nlp/sequence_classification.py
# modelscope/models/nlp/space/__init__.py
# modelscope/models/nlp/space_for_dialog_intent_prediction.py
# modelscope/models/nlp/space_for_dialog_modeling.py
# modelscope/models/nlp/space_for_dialog_state_tracking.py
# modelscope/models/nlp/task_model.py
# modelscope/pipelines/nlp/sentiment_classification_pipeline.py
# modelscope/preprocessors/base.py
# modelscope/preprocessors/nlp.py
# modelscope/trainers/trainer.py
* revert changes
* unify sentnece classification postprocess
* revert some changes, move some model files
* pipeline first case run through
* ws pipeline passed
* Merge branch 'feat/refactor_nlp_730' into feat/finetune_sbert_veco
* finetune
* revert code
* revert some code
* ws finetune started, only the accuracy is weird
* Merge branch 'feat/veco_taskdataset' into feat/finetune_sbert_veco
# Conflicts:
# modelscope/task_datasets/veco_dataset.py
# tests/taskdataset/test_veco_dataset.py
* veco+nli finetune started
* Merge branch 'master' into feat/finetune_sbert_veco
# Conflicts:
# modelscope/models/nlp/sbert_for_sequence_classification.py
# modelscope/models/nlp/sbert_for_token_classification.py
# modelscope/models/nlp/sbert_for_zero_shot_classification.py
# modelscope/models/nlp/space/space_for_dialog_intent_prediction.py
# modelscope/models/nlp/space/space_for_dialog_modeling.py
# modelscope/trainers/trainer.py
* add trainer for nlp
* trainer: dataset params passed into preprocessor
* test passed by nlptrainer
* fix some bugs
* fix some bugs
* add backbone/head subclass
* fix regression bugs
* fix bug in token-cls finetune
* support cfg modification
* fix bug
* fix bug
* update requirements
* add some comments and fix some t
* add some comments and revert a argument
* split to two test files
* revert code
* fixbug in precessor
(cherry picked from commit 7a648d096ef8500c694d3255dabe29e6f4bfc3e5)
* fix ut bug
* support sbert models
* unfinished
* Merge branch 'feat/finetune_sbert_veco' into sly_tmp_veco_finetune
# Conflicts:
# tests/trainers/test_finetune_sequence_classification.py
* fixbug in veco
* fix bug
* fixbug
* correct running params
* remove useless files
* add palm finetuning with cnn_dailymail dataset
* copy space model from sofa
* Merge branch 'feat/finetune_sbert_veco' of gitlab.alibaba-inc.com:Ali-MaaS/MaaS-lib into feat/finetune_sbert_veco
* Merge branch 'master' into feat/finetune_sbert_veco
# Conflicts:
# modelscope/metrics/__init__.py
# modelscope/models/__init__.py
# modelscope/models/nlp/__init__.py
# modelscope/models/nlp/backbones/__init__.py
# modelscope/models/nlp/backbones/structbert/modeling_sbert.py
# modelscope/models/nlp/heads/__init__.py
# modelscope/models/nlp/masked_language.py
# modelscope/models/nlp/palm_v2/palm_for_text_generation.py
# modelscope/models/nlp/sbert_for_nli.py
# modelscope/models/nlp/sbert_for_sentence_similarity.py
# modelscope/models/nlp/sbert_for_sentiment_classification.py
# modelscope/models/nlp/sbert_for_sequence_classification.py
# modelscope/models/nlp/sbert_for_token_classification.py
# modelscope/models/nlp/sbert_for_zero_shot_classification.py
# modelscope/models/nlp/sequence_classification.py
# modelscope/models/nlp/space/space_for_dialog_intent_prediction.py
# modelscope/models/nlp/space/space_for_dialog_modeling.py
# modelscope/models/nlp/space/space_for_dialog_state_tracking.py
# modelscope/models/nlp/structbert/adv_utils.py
# modelscope/models/nlp/structbert/configuration_sbert.py
# modelscope/models/nlp/task_models/task_model.py
# modelscope/pipelines/__init__.py
# modelscope/pipelines/nlp/__init__.py
# modelscope/pipelines/nlp/fill_mask_pipeline.py
# modelscope/pipelines/nlp/named_entity_recognition_pipeline.py
# modelscope/pipelines/nlp/nli_pipeline.py
# modelscope/pipelines/nlp/sentence_similarity_pipeline.py
# modelscope/pipelines/nlp/sentiment_classification_pipeline.py
# modelscope/pipelines/nlp/text_generation_pipeline.py
# modelscope/pipelines/nlp/word_segmentation_pipeline.py
# modelscope/pipelines/nlp/zero_shot_classification_pipeline.py
# modelscope/preprocessors/nlp.py
# modelscope/task_datasets/__init__.py
# modelscope/trainers/trainer.py
# modelscope/trainers/utils/inference.py
# modelscope/utils/file_utils.py
# requirements/nlp.txt
# tests/pipelines/test_nli.py
# tests/pipelines/test_sentence_similarity.py
# tests/pipelines/test_sentiment_classification.py
* fix imports
* mark backbone in their own modeling
* pre-commit check passed
* pre-commit passed, remove roberta model
* fix a bug in ast import
* skip all finetune uts
* fix bugs
* pre-commit passed
* bug fixed
* bug fixed
* bug fixed
* bug fixed
* fix ut bug
* fix bug
* fix ut bug
* fix bug
* fix bug
* fixbugs
* fixbug
* revert veco
* revert veco because of core dump
* fix palm bug
* revert veco
* revert mistaken code
* add a test print
* pre-commit check
* test exception
* add test code
* for test
* fix bug and test
* remove test code
* remove useless file
* 1. fix some bugs 2. add backbone ut
* Merge branch 'master' into feat/finetune_refactor_730
# Conflicts:
# modelscope/metainfo.py
# modelscope/metrics/sequence_classification_metric.py
# modelscope/models/nlp/__init__.py
# modelscope/models/nlp/task_models/task_model.py
# modelscope/preprocessors/__init__.py
# modelscope/preprocessors/nlp.py
# modelscope/trainers/trainer.py
# modelscope/trainers/utils/inference.py
# modelscope/utils/file_utils.py
# tests/trainers/test_trainer_with_nlp.py
* pre-commit passed
* revert files
* increase test level
* unregister models
* fix bugs
* fix cr comments
* fix bug in backbone-head
* add sbert backbone
* fix bug
* add test for token-cls-metric
* pre-commit passed
* fix ut comments
* revert normal tokenizer to fast tokenizer
* Merge branch 'master' into feat/finetune_refactor_730
# Conflicts:
# modelscope/models/nlp/__init__.py
# modelscope/models/nlp/backbones/__init__.py
# modelscope/models/nlp/backbones/structbert/__init__.py
# modelscope/models/nlp/masked_language.py
# modelscope/models/nlp/palm_v2/palm_for_text_generation.py
# modelscope/models/nlp/sbert_for_sequence_classification.py
# modelscope/models/nlp/sbert_for_token_classification.py
# modelscope/models/nlp/sbert_for_zero_shot_classification.py
# modelscope/pipelines/nlp/text_generation_pipeline.py
# modelscope/preprocessors/nlp.py
# modelscope/trainers/trainer.py
# modelscope/trainers/utils/inference.py
* fix merge bugs
* pre commit passed
* fix bug
* fix bug
* fix bug
* fix bug from master
* add print
* fix ut bug
* fix bug
* Merge branch 'master' into feat/finetune_refactor_730
* skip task model test
2022-08-03 18:38:41 +08:00
baiguan.yt
3f7a5c8279
[to #42322933 ]人像增强
...
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9590794
2022-08-02 21:33:32 +08:00
yuze.zyz
7d348b9ae8
[to #42322933 ] change dummy dataset to msdataset
...
1. change dummy dataset to msdataset
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9586561
* change dummpy dataset to msdataset
* add pre-commit ignore
* Merge commit 'e93339ea877b93fa0c1b9ebfeee8877f78facb0e' into feat/ms_dataset_case
* Merge commit '34840fc5d8a8ee8cd1278efea913d42db522f9c8' into feat/ms_dataset_case
* remove useless ip hosts.
* Merge commit '47dda0a5f9b4b4466177d9acae097a53f8bea8f7' into feat/ms_dataset_case
* Merge commit '21de1e7db035843c6c2caccf382a6e4f7071f96b' into feat/ms_dataset_case
2022-08-02 21:07:47 +08:00
jiangnana.jnn
34840fc5d8
[to #43627720 ] support ReduceLROnPlateau and fix lr scheduler
...
1. Support `ReduceLROnPlateau` lr scheduler, and add `PlateauLrSchedulerHook` for it
2. Support custom `optimizer_hook` and `lr_scheduler_hook`
3. Remove function of save best ckpt from `EvaluationHook`, replace with `BestCkptSaverHook`
4. `evaluation_loop` return metric values directly，move metric computation to `single_gpu_test` and `multi_gpu_test`
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9584322
* [to #43627720 ] support ReduceLROnPlateau and fix lr scheduler
2022-08-02 14:49:48 +08:00
feiwu.yfw
743e876981
[to #43660556 ] msdataset数据集加载
...
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9552632
* load csv dataset from modelscoop
2022-07-29 12:22:48 +08:00
wenmeng.zwm
39c684db24
[to #43115513 ] remove easynlp trainer test
...
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9567392
2022-07-29 11:19:54 +08:00
wenmeng.zwm
b3f4ac8acc
[to #43115042 ] add trainer usage doc
...
1. add trainer doc
2. support local configuration file for trainer
3. update nlp trainer test
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9541239
2022-07-29 10:28:50 +08:00
