7 Commits

Author SHA1 Message Date
hemu.zp
06296c1819 [to #42322933] Fix evaluation oom
Add merge method for all metrics, parallel metrics can be merged when using data parallel. No longer save all data in the evaluation process to avoid oom.

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11399082
2023-01-12 13:02:54 +08:00
hemu.zp
941dbe75cf [to #42322933] Add GPT-3 tensor parallel finetuning
Add GPT-3 tensor parallel finetuning, adjust some distributed codes to make tensor and data parallel compatible.
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10949507
2022-12-05 10:01:32 +08:00
yingda.chen
de708dd518 add basic remap column wrapper
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10539917

    * add basic remap column wrapper
2022-10-27 10:12:05 +08:00
hemu.zp
69104c0f8a [to #42322933] Refactor text generation model outputs and fix some bugs
1. 将 single_gpu_test 与 multi_gpu_test 中的 model.forward 部分分离为 EpochBasedTrainer 中的 evaluation_step,为部分 evaluation 阶段不调用 forward 的模型提供更好的灵活性
2. 重构代码将文本生成模型 Model 层的输入输出统一为 Tensor,Tensor 到 str 的 decode 过程移动到 pipeline 中完成
3. pipeline 后处理添加对中文和中文标点与英文混杂时空格的处理,使 decode 后中英文混杂输出正确
4. 添加 TextGenerationTrainer 修复了部分模型 evaluation 过程 forward 输出单个 token 计算 metrics 的问题
5. 修复了 rouge 无法接收空字符串的问题
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10473768
2022-10-27 09:52:05 +08:00
yuze.zyz
01e768503c [to #42322933] Fix random seed for trainer
1. Fix random seed for trainer and init it at the first line of init
2. Add a regress test for fixed training
3. Change the dataset 'dureader_robust_qg' to 'DuReader_robust-QG'
4. Change some datasets from loading hf.datasets to loading msdataset.load
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10029509
2022-09-06 15:13:14 +08:00
yuze.zyz
a9c14e4ead [to #42322933] Support saving the best checkpoint for inference
1. Support saving the best checkpoint for inference
2. Fix a bug that _max_iters field does not exist in trainer
3. Fix a bug that function in lambda_lr field cannot be saved to file
4. Fix a bug that save_pretrained would not be called by iterating
5. Fix a bug that interval is not passed from BestCkptHook's init
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9972765
2022-09-05 11:07:48 +08:00
hemu.zp
20a935d406 [to #42322933] add gpt3 base finetune
添加 gpt3 中小模型单机单卡下的 finetune 代码
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9899004
2022-08-26 14:54:45 +08:00