Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2025-12-19 09:39:23 +01:00
Code Issues Packages Projects Releases Wiki Activity
Files
remove-warning
modelscope/examples/pytorch/llama/run_train_lora.sh

24 lines
728 B
Bash
Raw Permalink Normal View History

Support lora for llama Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13080086 * support lora for llama * update baichuan * remove work_dir * fixbug: 1. change ConfigDict to list when hooks key not in config 2. ignore all bin files when preparing output folder * 1. support device_map 2. remove the operation of to float when using lora * add inference file * add comment * support device_map
2023-06-29 22:05:34 +08:00
export PYTHONPATH=$PYTHONPATH:./
Support llama & lora finetune without deepspeed Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13131145 * support llama + lora without deepspeed * feat: Fix conflict, auto commit by WebIDE
2023-07-25 17:32:46 +08:00
torchrun examples/pytorch/llama/finetune_llama.py \
Support lora for llama Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13080086 * support lora for llama * update baichuan * remove work_dir * fixbug: 1. change ConfigDict to list when hooks key not in config 2. ignore all bin files when preparing output folder * 1. support device_map 2. remove the operation of to float when using lora * add inference file * add comment * support device_map
2023-06-29 22:05:34 +08:00
--work_dir './tmp' \
--model 'skyline2006/llama-7b' \
support load dataset for llama support loading dataset for llama: 1.load dataset by MsDataset when parameters train dataset name and val dataset name were set. but there is no suitable dataset in hub. 2.load dataset by MsDataset when only parameter train dataset name was set, and then split into train dataset and validation dataset . 3.load dataset by MsDataset when user set parameter src_txt, which is a file path such as 'alpaca_data.json', and then split into training dataset and validation dataset. 4.load dataset by build dataset from file in flex training. Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13505335
2023-08-07 19:48:36 +08:00
--train_dataset_name 'alpaca-gpt4-data-zh' \
--train_subset_name 'default' \
--train_split 'train' \
--train_dataset_namespace 'AI-ModelScope' \
--per_device_train_batch_size 4 \
--per_device_eval_batch_size 4 \
--eval_strategy 'by_epoch' \
--eval_interval 1 \
--eval_metrics 'ppl' \
--lr 2e-5 \
--save_strategy no \
--save_best true \
--metric_for_best_model ppl \
--metric_rule_for_best_model min \
Support lora for llama Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13080086 * support lora for llama * update baichuan * remove work_dir * fixbug: 1. change ConfigDict to list when hooks key not in config 2. ignore all bin files when preparing output folder * 1. support device_map 2. remove the operation of to float when using lora * add inference file * add comment * support device_map
2023-06-29 22:05:34 +08:00
--use_lora 1 \
Support llama & lora finetune without deepspeed Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13131145 * support llama + lora without deepspeed * feat: Fix conflict, auto commit by WebIDE
2023-07-25 17:32:46 +08:00
--device_map 'auto' \
support load dataset for llama support loading dataset for llama: 1.load dataset by MsDataset when parameters train dataset name and val dataset name were set. but there is no suitable dataset in hub. 2.load dataset by MsDataset when only parameter train dataset name was set, and then split into train dataset and validation dataset . 3.load dataset by MsDataset when user set parameter src_txt, which is a file path such as 'alpaca_data.json', and then split into training dataset and validation dataset. 4.load dataset by build dataset from file in flex training. Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13505335
2023-08-07 19:48:36 +08:00
--task 'text-generation' \
--model.type 'llama' \
--max_epochs 3 \
Reference in New Issue Copy Permalink