Commit Graph

9 Commits

Author SHA1 Message Date
mulin.lyh
7c0f6aa5c5 Merge branch 'master-github' into master-merge-github0829 2023-08-29 21:20:54 +08:00
tastelikefeet
f2e9ea4199 Replace code with swift wheel (#467) 2023-08-29 17:27:18 +08:00
zsl01670416
43a57fe110 support load dataset for llama
support loading dataset for llama:
1.load dataset by MsDataset when parameters train dataset name and val dataset name were set. but there is no suitable dataset in hub.
2.load dataset by MsDataset when only parameter train dataset name was set, and then split into train dataset and validation dataset . 
3.load dataset by MsDataset when user set parameter src_txt, which is a file path such as 'alpaca_data.json', and then split into training dataset and validation dataset.
4.load dataset by build dataset from file in flex training.

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13505335
2023-08-07 19:48:36 +08:00
hemu.zp
ed6e139759 Support llama & lora finetune without deepspeed
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13131145
* support llama + lora without deepspeed

* feat: Fix conflict, auto commit by WebIDE
2023-07-25 17:32:46 +08:00
hemu.zp
bb194f1d96 fix llama bug
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13133641
* fix bug
2023-06-30 20:49:30 +08:00
hemu.zp
30599dc34f fix llama lora bug
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13129620
2023-06-30 19:03:56 +08:00
hemu.zp
f4c90f2adf Support lora for llama
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13080086

* support lora for llama

* update baichuan

* remove work_dir

* fixbug: 1. change ConfigDict to list when hooks key not in config 2. ignore all bin files when preparing output folder

* 1. support device_map 2. remove the operation of to float when using lora

* add inference file

* add comment

* support device_map
2023-06-29 22:05:34 +08:00
suluyan.sly
7e28ca226b [to #50334474] llama tuned model -> pipeline
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13071602
* prepare for inference
2023-06-26 22:49:44 +08:00
suluyan.sly
de491cfe6b [to #48855014] llama finetune + deepspeed
1. llama base finetune:使用trainer从llama finetune至alpaca,效果确认并提供example
2. deepspeed通用性完善:mpu解耦;deepspeed的训练信息支持从ms log透出(目前ms打印的log有误);支持从modelscope configuration.json 进行 deepspeed config 配置;deepspeed optimizer和lr_scheduler 初始化支持;解决deepspeed和ddp同时使用报错;解决保存ckpt时报错
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12651323
2023-06-14 10:00:23 +08:00