Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2025-12-25 12:39:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
1,971 Commits 263 Branches 106 Tags
8ff95a5dc82ae87520f02e8e6d7ca6706984ba4c
Commit Graph

94 Commits

Author SHA1 Message Date
Wang Qiang
a67d339e3b support float16 training and pipeline for stable diffusion (#447) 
* support float16 traing and pipeline for stable diffusion

* pre commit

* fix bugs

* add torch type example

* fix bugs of torch type

* support type float16

* fix bugs of load pipeline

* change type to fp16

* lora rank

---------

Co-authored-by: 翊靖 <yijing.wq@alibaba-inc.com>
 2023-08-15 20:04:32 +08:00
Zhicheng Zhang
d212ced3f3 Update qwen QA example (#470) 2023-08-15 17:25:25 +08:00
Jintao
18d33a4825 fix copytree python37 bug (#464) 
* fix copytree python37 bug

* add copytree_py37 function
 2023-08-14 11:45:33 +08:00
Yingda Chen
b18288cd6b update example 2023-08-13 11:06:41 +08:00
Jintao
c06f47f384 add readme and warning (#462) 
* add readme and warning

* fix bug

* update

* update readme
 2023-08-11 14:55:24 +08:00
Zhicheng Zhang
0e8be51fef add qwen qa example with langchain 2023-08-11 11:53:17 +08:00
Zhicheng Zhang
ad5b134bfe update qwen qa example 2023-08-10 19:34:39 +08:00
Zhicheng Zhang
2baef2110d update QA example 2023-08-10 19:03:20 +08:00
Zhicheng Zhang
9854759a55 add doc search QA with dashscope 2023-08-08 15:05:50 +08:00
lukeming.lkm
bd2f70a6eb add quantization in qwen pipelines and relevant unittests 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13499600

* add quant features

* resolve import

* resolve format

* fix save vocab
 2023-08-02 14:05:13 +08:00
lukeming.lkm
33bd74a7be add qwen 7b base and chat 
添加QWen 7b base模型和chat模型及相关pipelines
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13482235

* add qwen 7b base and chat

* fix logger

* update examples, lint test

* add unittest for qwen base and chat

* rename qwen to qwen-7b

* resolve imports and add a registry to text-generation

* reset load model from pretrained

* fix precheck

* skip qwen test case now

* remove strange file
 2023-08-02 09:25:21 +08:00
wenmeng.zwm
48a39244f1 Merge branch master-merge-github-230728 into master 
Title: Merge branch 'master-github' into master-merge-github-230728 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13456249
 2023-07-31 16:09:07 +08:00
suluyan.sly
d7cd2ce28e Merge branch 'master-github' into master-merge-github-230728 2023-07-29 10:42:54 +08:00
Jintao
312b63fe06 fix checkpoint, same device bug (#427) 2023-07-29 00:06:27 +08:00
zsl01670416
847607ab66 Merge branch 'debug_chatglm6b_json_dataset' fix conflict between hf dataset and to_hf_dataset 
The type of dataset built from file is hf dataset, which can not use function to_hf_dataset.
 2023-07-28 19:12:21 +08:00
suluyan.sly
05e1357c32 Merge branch 'master-github' into master-merge-github-230728 2023-07-28 16:40:34 +08:00
Wang Qiang
34ea2b474a Upgrade stable diffusion version to a more powerful version 2.1 (#415) 2023-07-27 20:40:38 +08:00
suluyan.sly
1c6f5fe775 Merge branch 'master-github' into master-merge-github-230727 
Conflicts:
       examples/pytorch/baichuan/finetune_baichuan.py
       examples/pytorch/chatglm6b/finetune.py
 2023-07-27 17:29:27 +08:00
Wang Qiang
66cf72a75c Merge pull request #376 from XDUWQ/custom_diffusion 
Custom method for finetuning stable diffusion
 2023-07-27 10:41:38 +08:00
Jintao
4ca937d2ba support openbuddy-llama2-13b (#416) 2023-07-26 18:12:55 +08:00
XDUWQ
99aa707995 fix bugs 2023-07-26 16:35:40 +08:00
tastelikefeet
0db3d1d53b Fix bug of amp and device_map (#397) 
* fix amp

* remove useless code

* Fix bug
 2023-07-25 19:28:00 +08:00
Jintao
f03898626e ckpt output directory ignore *.safetensors (#410) 
* ckpt output file ignore *.safetensors

* update
 2023-07-25 19:27:11 +08:00
hemu.zp
fc54593a56 fix baichuan eval and support sequence_length 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13404289
* fix baichuan eval

* support sequence_length and ppl

* fix typo

* fix bug for palm

* fix bug
 2023-07-25 19:10:45 +08:00
zsl01670416
9926ad685b support getting labels from dataset in sbert text classification and building dataset from file in chatglm-6b 
1.Add getting labels from dataset in "text_classificationfinetune_text_classification.py" to simplify user's operation in flex training. Parameters "--num_labels" and "--labels" were removed in "run_train.sh".
2.In "chatglm6b / finetune.py", building dataset from file  is necessary to support flex training.
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13382745
* support getting labels from dataset in sbert text classification and building dataset from file in chatglm-6b

* support getting labels from dataset in sbert text classification and building dataset from file in chatglm-6b

* remove repetitive labels in a concise manner of using set

* reserve parameter labels in finetune_text_classification

* Merge branch 'master' of http://gitlab.alibaba-inc.com/Ali-MaaS/MaaS-lib

reserve parameter labels in finetune_text_classification

* Merge branch 'support_text_cls_labels_chatglm_json'
reserve parameter labels in finetune_text_classification
 2023-07-25 19:02:32 +08:00
hemu.zp
ed6e139759 Support llama & lora finetune without deepspeed 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13131145
* support llama + lora without deepspeed

* feat: Fix conflict, auto commit by WebIDE
 2023-07-25 17:32:46 +08:00
XDUWQ
3412a074c5 precommit 2023-07-25 15:00:28 +08:00
XDUWQ
ffbf77fcf2 update 2023-07-25 14:47:45 +08:00
XDUWQ
8e157cfa15 precommit 2023-07-24 22:10:52 +08:00
XDUWQ
426f55d57b add lora_rank for lora stable diffusion 2023-07-24 19:43:20 +08:00
XDUWQ
bc93e2dc96 add lora_rank for lora stable diffusion 2023-07-24 19:32:04 +08:00
XDUWQ
eb24e23d19 add lora_rank for lora stable diffusion 2023-07-24 19:24:52 +08:00
XDUWQ
6fb340e7f8 add lora_rank for lora stable diffusion 2023-07-24 19:17:49 +08:00
Jintao
ba4b9fc43f Added full parameter sft to llm (#402) 
* Optimized code

* update parse_args

* fix get_logger bug

* update parse_args

* Added full parameter fine-tuning

* Add support_bf16 warning

* Modify the code format and fix bugs
 2023-07-24 15:52:09 +08:00
xingjun.wang
96a5282021 check format 2023-07-21 23:09:24 +08:00
Jintao
2f7c669f33 support llama2 (#393) 
* Unify sft and infer code into a single file

* update llama2 sft infer
 2023-07-19 17:34:27 +08:00
tastelikefeet
12bc1603a9 Fix amp + device_map (#386) 
1. Fix the amp + device_map bug in chatglm2 finetune code 
2. Optional to save optimizer state 
3. Fix the logger double print problem
 2023-07-16 08:45:20 +08:00
Jintao
c6189d68a0 Fix/chatglm2 (#384) 2023-07-15 09:59:53 +08:00
XDUWQ
bcf443c672 custom diffusion 2023-07-12 15:32:48 +08:00
XDUWQ
0e99c27d54 custom diffusion 2023-07-12 15:27:08 +08:00
XDUWQ
d6368b2617 custom diffusion 2023-07-12 15:02:43 +08:00
XDUWQ
0dc57f8dec custom diffusion 2023-07-12 14:56:14 +08:00
Xingjun.Wang
0f36b081ef Merge pull request #371 from foocker/master 
ASRDataset for download_mode parameters
 2023-07-12 14:47:32 +08:00
tastelikefeet
544f6c0410 Fea/chatglm6b v2 new version (#368) 
* upgrade code

* add chatglm2 ptuning

* pre-commit passed
 2023-07-12 14:44:09 +08:00
Wang Qiang
04b940f600 Merge branch 'modelscope:master' into custom_diffusion 2023-07-12 10:08:11 +08:00
gg
49c6d8bcf6 pre-commit 2023-07-12 09:43:24 +08:00
XDUWQ
1caa45422c custom diffusion 2023-07-11 20:46:32 +08:00
gg
574b4568ff flake8 2023-07-11 18:36:54 +08:00
Jintao
d20d033e07 add example/llm (#372) 
* add example/llm

* fix lint test
 2023-07-11 17:35:11 +08:00
fq
d47684de5b Optimize comments 2023-07-11 17:12:59 +08:00