Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2025-12-23 19:49:24 +01:00
Code Issues Packages Projects Releases Wiki Activity
1,918 Commits 263 Branches 106 Tags
f856c04785fe95a1eee1708b51bc09cc667bdcd3
Commit Graph

83 Commits

Author SHA1 Message Date
wenmeng.zwm
48a39244f1 Merge branch master-merge-github-230728 into master 
Title: Merge branch 'master-github' into master-merge-github-230728 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13456249
 2023-07-31 16:09:07 +08:00
suluyan.sly
d7cd2ce28e Merge branch 'master-github' into master-merge-github-230728 2023-07-29 10:42:54 +08:00
Jintao
312b63fe06 fix checkpoint, same device bug (#427) 2023-07-29 00:06:27 +08:00
zsl01670416
847607ab66 Merge branch 'debug_chatglm6b_json_dataset' fix conflict between hf dataset and to_hf_dataset 
The type of dataset built from file is hf dataset, which can not use function to_hf_dataset.
 2023-07-28 19:12:21 +08:00
suluyan.sly
05e1357c32 Merge branch 'master-github' into master-merge-github-230728 2023-07-28 16:40:34 +08:00
Wang Qiang
34ea2b474a Upgrade stable diffusion version to a more powerful version 2.1 (#415) 2023-07-27 20:40:38 +08:00
suluyan.sly
1c6f5fe775 Merge branch 'master-github' into master-merge-github-230727 
Conflicts:
       examples/pytorch/baichuan/finetune_baichuan.py
       examples/pytorch/chatglm6b/finetune.py
 2023-07-27 17:29:27 +08:00
Wang Qiang
66cf72a75c Merge pull request #376 from XDUWQ/custom_diffusion 
Custom method for finetuning stable diffusion
 2023-07-27 10:41:38 +08:00
Jintao
4ca937d2ba support openbuddy-llama2-13b (#416) 2023-07-26 18:12:55 +08:00
XDUWQ
99aa707995 fix bugs 2023-07-26 16:35:40 +08:00
tastelikefeet
0db3d1d53b Fix bug of amp and device_map (#397) 
* fix amp

* remove useless code

* Fix bug
 2023-07-25 19:28:00 +08:00
Jintao
f03898626e ckpt output directory ignore *.safetensors (#410) 
* ckpt output file ignore *.safetensors

* update
 2023-07-25 19:27:11 +08:00
hemu.zp
fc54593a56 fix baichuan eval and support sequence_length 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13404289
* fix baichuan eval

* support sequence_length and ppl

* fix typo

* fix bug for palm

* fix bug
 2023-07-25 19:10:45 +08:00
zsl01670416
9926ad685b support getting labels from dataset in sbert text classification and building dataset from file in chatglm-6b 
1.Add getting labels from dataset in "text_classificationfinetune_text_classification.py" to simplify user's operation in flex training. Parameters "--num_labels" and "--labels" were removed in "run_train.sh".
2.In "chatglm6b / finetune.py", building dataset from file  is necessary to support flex training.
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13382745
* support getting labels from dataset in sbert text classification and building dataset from file in chatglm-6b

* support getting labels from dataset in sbert text classification and building dataset from file in chatglm-6b

* remove repetitive labels in a concise manner of using set

* reserve parameter labels in finetune_text_classification

* Merge branch 'master' of http://gitlab.alibaba-inc.com/Ali-MaaS/MaaS-lib

reserve parameter labels in finetune_text_classification

* Merge branch 'support_text_cls_labels_chatglm_json'
reserve parameter labels in finetune_text_classification
 2023-07-25 19:02:32 +08:00
hemu.zp
ed6e139759 Support llama & lora finetune without deepspeed 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13131145
* support llama + lora without deepspeed

* feat: Fix conflict, auto commit by WebIDE
 2023-07-25 17:32:46 +08:00
XDUWQ
3412a074c5 precommit 2023-07-25 15:00:28 +08:00
XDUWQ
ffbf77fcf2 update 2023-07-25 14:47:45 +08:00
XDUWQ
8e157cfa15 precommit 2023-07-24 22:10:52 +08:00
XDUWQ
426f55d57b add lora_rank for lora stable diffusion 2023-07-24 19:43:20 +08:00
XDUWQ
bc93e2dc96 add lora_rank for lora stable diffusion 2023-07-24 19:32:04 +08:00
XDUWQ
eb24e23d19 add lora_rank for lora stable diffusion 2023-07-24 19:24:52 +08:00
XDUWQ
6fb340e7f8 add lora_rank for lora stable diffusion 2023-07-24 19:17:49 +08:00
Jintao
ba4b9fc43f Added full parameter sft to llm (#402) 
* Optimized code

* update parse_args

* fix get_logger bug

* update parse_args

* Added full parameter fine-tuning

* Add support_bf16 warning

* Modify the code format and fix bugs
 2023-07-24 15:52:09 +08:00
xingjun.wang
96a5282021 check format 2023-07-21 23:09:24 +08:00
Jintao
2f7c669f33 support llama2 (#393) 
* Unify sft and infer code into a single file

* update llama2 sft infer
 2023-07-19 17:34:27 +08:00
tastelikefeet
12bc1603a9 Fix amp + device_map (#386) 
1. Fix the amp + device_map bug in chatglm2 finetune code 
2. Optional to save optimizer state 
3. Fix the logger double print problem
 2023-07-16 08:45:20 +08:00
Jintao
c6189d68a0 Fix/chatglm2 (#384) 2023-07-15 09:59:53 +08:00
XDUWQ
bcf443c672 custom diffusion 2023-07-12 15:32:48 +08:00
XDUWQ
0e99c27d54 custom diffusion 2023-07-12 15:27:08 +08:00
XDUWQ
d6368b2617 custom diffusion 2023-07-12 15:02:43 +08:00
XDUWQ
0dc57f8dec custom diffusion 2023-07-12 14:56:14 +08:00
Xingjun.Wang
0f36b081ef Merge pull request #371 from foocker/master 
ASRDataset for download_mode parameters
 2023-07-12 14:47:32 +08:00
tastelikefeet
544f6c0410 Fea/chatglm6b v2 new version (#368) 
* upgrade code

* add chatglm2 ptuning

* pre-commit passed
 2023-07-12 14:44:09 +08:00
Wang Qiang
04b940f600 Merge branch 'modelscope:master' into custom_diffusion 2023-07-12 10:08:11 +08:00
gg
49c6d8bcf6 pre-commit 2023-07-12 09:43:24 +08:00
XDUWQ
1caa45422c custom diffusion 2023-07-11 20:46:32 +08:00
gg
574b4568ff flake8 2023-07-11 18:36:54 +08:00
Jintao
d20d033e07 add example/llm (#372) 
* add example/llm

* fix lint test
 2023-07-11 17:35:11 +08:00
fq
d47684de5b Optimize comments 2023-07-11 17:12:59 +08:00
fq
4d77e57769 add download_mode param to params 
maybe set from funasr is better.
 2023-07-11 16:43:37 +08:00
fq
61faedfb15 Update finetune_speech_recognition.py add params.download_mode 
set from params, config
 2023-07-11 15:18:59 +08:00
fq
2073f4fd55 Update finetune_speech_recognition.py 
MsDataset replace by ASRDataset.
 2023-07-11 11:54:05 +08:00
fq
20c15d3aaa Update finetune_speech_recognition.py 
using the newest ASRDataset, and add download_mode for re-download the dataset(dataset is broken and so on)
 2023-07-11 11:33:18 +08:00
Wang Qiang
d49953b943 fix bugs of loading local sd dataset (#357) 2023-07-04 22:01:21 +08:00
Firmament-cyou
423e2ce940 Add lora_inference for baichuan. (#352) 
* add lora_inference.py for baichuan

* fix linttest

* fix linttest

---------

Co-authored-by: hemu <hemu.zp@alibaba-inc.com>
 2023-07-04 18:39:36 +08:00
tastelikefeet
08c71f1f3d Fix/chatglm6b 2 (#354) 2023-07-04 01:58:57 +08:00
tastelikefeet
45cf0035f4 fix chatglm2 evaluation error: hypothesis emtpy (#348) 
* fix evaluation error: hypothesis emtpy

* fix pipeline

* fix bug
 2023-07-03 23:16:38 +08:00
Jintao
432f0ee20f add baichuan/chatglm2 +lora+agent examples (#350) 2023-07-03 16:19:00 +08:00
Wang Qiang
a50342ed5d Add stable diffusion tutorial ipynb (#346) 2023-07-03 13:52:35 +08:00
hemu.zp
bb194f1d96 fix llama bug 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13133641
* fix bug
 2023-06-30 20:49:30 +08:00