Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2026-02-24 12:10:09 +01:00
Code Issues Packages Projects Releases Wiki Activity
1,943 Commits 267 Branches 107 Tags
liuyhwangyh-patch-2
Commit Graph

1943 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
liuyhwangyh
5b92002086 ci to us host 2023-08-07 09:53:56 +08:00
wenmeng zhou
1e580e212e update qwen with cpu infer/tokenization utils/other configs 
update qwen with cpu infer/tokenization utils/other configs
 2023-08-04 16:32:52 +08:00
lukeming.lkm
3c400cbeda update qwen with cpu infer/tokenization utils/other configs 2023-08-04 14:22:53 +08:00
wenmeng zhou
7b8bb091a5 Merge pull request #437 from modelscope/merge_gitlab_0803 
Merge gitlab 0803
 2023-08-03 12:25:11 +08:00
wenmeng.zwm
2796b0eecc Merge branch 'master-gitlab' into merge_gitlab_0803 2023-08-03 12:14:27 +08:00
liuyhwangyh
0fff03d640 modify gpu/cpu binding (#436) 2023-08-03 11:27:37 +08:00
lukeming.lkm
b3a61ef6f4 update LICENSE 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13516119
 2023-08-03 11:07:34 +08:00
lukeming.lkm
914e9af3ad add LICENSE and related snippets 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13513677
 2023-08-03 09:39:20 +08:00
lukeming.lkm
561fbe9eb2 reformat stop_words_ids to sync QWen7b HF 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13510247
 2023-08-02 18:46:12 +08:00
liuyhwangyh
11832f9246 daily regression to us host (#435) 2023-08-02 18:32:53 +08:00
lukeming.lkm
1ce78d648d add kwargs for save_vocabulary 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13508568
 2023-08-02 18:12:11 +08:00
lukeming.lkm
dece963ad5 resolve stopword logits processor 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13504522

* resolve stopword logits processor
 2023-08-02 16:14:48 +08:00
hejunjie.hjj
0c679d2e09 fix bug: referring_video-object-segmentation兼容Pillow10 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13479152
* fix bug: referring_video-object-segmentation兼容Pillow10
 2023-08-02 16:08:26 +08:00
lukeming.lkm
bd2f70a6eb add quantization in qwen pipelines and relevant unittests 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13499600

* add quant features

* resolve import

* resolve format

* fix save vocab
 2023-08-02 14:05:13 +08:00
lukeming.lkm
33bd74a7be add qwen 7b base and chat 
添加QWen 7b base模型和chat模型及相关pipelines
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13482235

* add qwen 7b base and chat

* fix logger

* update examples, lint test

* add unittest for qwen base and chat

* rename qwen to qwen-7b

* resolve imports and add a registry to text-generation

* reset load model from pretrained

* fix precheck

* skip qwen test case now

* remove strange file
 2023-08-02 09:25:21 +08:00
wenmeng zhou
1a6583eee2 Feat/chatglm2 32k support (#433) 
* add support for chatglm2 32k

* fix
 2023-08-01 23:22:09 +08:00
suluyan.sly
f080603802 remove requirement realesrgan for basicsr 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13488773
* remove requirement realesrgan for basicsr
 2023-08-01 16:55:47 +08:00
mulin.lyh
c1df82acd8 init pip to aliyuncs 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13487845

* init pip to aliyuncs
 2023-08-01 16:55:33 +08:00
wenmeng zhou
2809bdbff6 Feat/chatglm2 32k support (#433) 
* add support for chatglm2 32k

* fix
 2023-08-01 10:58:29 +08:00
suluyan.sly
dc17c7d6ee remove cv requirements: basicsr 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13471366
* rm req basicsr
 2023-07-31 18:20:23 +08:00
mulin.lyh
da2badaacd [to #51274424]fix: easycv extension and tinycudann conflict 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13471150
* fix easycv extension and tinycudann conflict
 2023-07-31 18:08:53 +08:00
wenmeng zhou
f2396ddb6a Merge pull request #432 from modelscope/merge_master-gitlab_0731 
Merge master gitlab 0731
 2023-07-31 17:14:41 +08:00
yuze.zyz
f856c04785 Ignore http error 
忽略log打印，防止在check_model_is_latest时打出信息使用户不明所以。
将log改为Error栈
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13463005
 2023-07-31 17:14:12 +08:00
wenmeng.zwm
7fbd6ecc40 Merge branch 'master-gitlab' into merge_master-gitlab_0731 2023-07-31 16:16:16 +08:00
wenmeng.zwm
48a39244f1 Merge branch master-merge-github-230728 into master 
Title: Merge branch 'master-github' into master-merge-github-230728 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13456249
 2023-07-31 16:09:07 +08:00
Xingjun.Wang
ef1b429bef Merge pull request #430 from XDUWQ/delete_nerf_log 
delete unuseful logs
 2023-07-31 14:48:39 +08:00
XDUWQ
4233916867 delete unuseful logs 2023-07-31 14:36:12 +08:00
suluyana
b68b90ba15 skip plugin 2023-07-30 00:30:30 +08:00
suluyana
9ece90ee84 skip plugin test case 2023-07-29 21:35:21 +08:00
suluyan.sly
d7cd2ce28e Merge branch 'master-github' into master-merge-github-230728 2023-07-29 10:42:54 +08:00
Jintao
312b63fe06 fix checkpoint, same device bug (#427) 2023-07-29 00:06:27 +08:00
zsl01670416
847607ab66 Merge branch 'debug_chatglm6b_json_dataset' fix conflict between hf dataset and to_hf_dataset 
The type of dataset built from file is hf dataset, which can not use function to_hf_dataset.
 2023-07-28 19:12:21 +08:00
suluyan.sly
05e1357c32 Merge branch 'master-github' into master-merge-github-230728 2023-07-28 16:40:34 +08:00
tastelikefeet
972298813b 1. add check model for training 2. add model_dir for automodel (#422) 2023-07-28 16:34:02 +08:00
wenmeng.zwm
3b485d5835 fix plugin python module missing files 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13453749
* fix plugin python module missing files
 2023-07-28 16:14:56 +08:00
frozoul
2566d028cd cv/cv nerf 3d reconstruction 4k nerf damo (#389) 
* add 4k-nerf core files

* update configure file

* update dataloader and model path

* update unittest

* Delete test_4k.py

* update unittest

* update unittest

* update pre-commit

* update dataloader

* update cuda code path

* check with pre-commit

---------

Co-authored-by: zhongshu.wzs <zhongshu.wzs@alibaba-inc.com>
Co-authored-by: wenmeng zhou <wenmeng.zwm@alibaba-inc.com>
 2023-07-28 10:37:13 +08:00
tongmu.wh
475924a421 correct language recognition taks name 
modelscope平台同学最终定下语种识别任务名为speech-language-recognition，对应进行代码中的相关改动
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13444517
* correct language recognition taks name
 2023-07-27 21:03:04 +08:00
Xingjun.Wang
3bd395a0c9 Merge pull request #423 from modelscope/dev/del_redundancy_code 
del redundant code
 2023-07-27 20:53:01 +08:00
xingjun.wang
6748cd7370 del redundancy code in func: get_dataset_meta_file_list 2023-07-27 20:49:41 +08:00
Wang Qiang
34ea2b474a Upgrade stable diffusion version to a more powerful version 2.1 (#415) 2023-07-27 20:40:38 +08:00
suluyan.sly
76921561e6 Merge branch master-merge-github-230727 into master 
Title: Merge branch 'master-github' into master-merge-github-230727 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13442869
 2023-07-27 20:18:35 +08:00
mulin.lyh
a501f7a417 fix easycv cpu extension build issue, cpu not support 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13441612
* fix easycv cpu extension build issue, cpu not support
 2023-07-27 17:42:37 +08:00
suluyan.sly
1c6f5fe775 Merge branch 'master-github' into master-merge-github-230727 
Conflicts:
       examples/pytorch/baichuan/finetune_baichuan.py
       examples/pytorch/chatglm6b/finetune.py
 2023-07-27 17:29:27 +08:00
Wang Qiang
dca6143b8b support safetensors weight pipeline (#421) 2023-07-27 16:49:01 +08:00
tastelikefeet
9802dfe93b fix device error (#419) 2023-07-27 16:23:21 +08:00
mengyang.fmy
18f998a85c add text-to-360pano-image pipeline, mod cv requirements 
7月份计划上线的360全景图生成模型，自研

模型权重文件地址https://www.modelscope.cn/models/damo/cv_diffusion_text-to-360panorama-image_generation/summary


#### 依赖项说明

##### 由于要使用xformers，torch版本最好使用1.13.1
```
pip install torch==1.13.1+cu116 torchvision==0.14.1+cu116 torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/cu116
```
##### 对应的diffusers和xformers版本如下
```
pip install -U diffusers==0.18.0
pip install xformers==0.0.16
pip install triton, accelerate, transformers
```

##### ModelScope Library 需要使用cv
```
pip install modelscope
pip install "modelscope[cv]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html
```

##### 此外，还需要安装第三方的一个库，Real-ESRGAN， 安装方法如下
```
# Install basicsr - https://github.com/xinntao/BasicSR
# We use BasicSR for both training and inference
pip install basicsr
# facexlib and gfpgan are for face enhancement
pip install facexlib
pip install gfpgan
pip install Pillow
pip install tqdm
pip install realesrgan==0.3.0
```
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13346430
* add text-to-360pano-image pipeline

* add text-to-360pano-image pipeline, mod cv requirements

* rm redundant files and cv requirements; add standard input and output definations

* fix diffusers==0.18.0 and run test

* fix diffusers==0.18.0 in multi-modal and run test again

* add model_revision='v1.0.0'

* fix yapf

* add trycatch for enabling xformers

* fix key error

* add install xformers in test/setup

* skip highres.fix in ci

* feat: Fix conflict, auto commit by WebIDE
 2023-07-27 11:33:39 +08:00
Wang Qiang
66cf72a75c Merge pull request #376 from XDUWQ/custom_diffusion 
Custom method for finetuning stable diffusion
 2023-07-27 10:41:38 +08:00
suluyana
534a020e93 Merge pull request #418 from tastelikefeet/feat/fix_device_split 
fix CI
 2023-07-27 10:03:41 +08:00
yuze.zyz
00c79428d5 fix 2023-07-26 21:30:38 +08:00
Jintao
4ca937d2ba support openbuddy-llama2-13b (#416) 2023-07-26 18:12:55 +08:00