Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2026-09-02 03:59:31 +02:00
Code Issues Packages Projects Releases Wiki Activity
1,947 Commits 280 Branches 124 Tags
49158090a748ee1de460e02ca753b0561df57f0b
Commit Graph

1947 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
tongmu.wh
49158090a7 修复一些bug 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13766312
* correct language recognition taks name

* fix some bugs

* merge master
 2023-08-24 17:02:54 +08:00
bin.xue
6682fb6052 增加语音分离模型MossFormer2 2023-08-24 16:49:22 +08:00
mulin.lyh
cba4e40bc1 fix numpy pandas compatible issue 
明确受影响的模型(damo)：  
ONE-PEACE-4B	ModuleNotFoundError: MyCustomPipeline: MyCustomModel: No module named 'one_peace'，缺少依赖。
cv_resnet50_face-reconstruction	 不兼容tf2  
nlp_automatic_post_editing_for_translation_en2de	tf2.0兼容性问题，tf1.x需要  
cv_resnet18_ocr-detection-word-level_damo	tf2.x兼容性问题  
cv_resnet18_ocr-detection-line-level_damo	tf兼容性问题  
cv_resnet101_detection_fewshot-defrcn	模型限制必须detection0.3+torch1.11.0"  
speech_dfsmn_ans_psm_48k_causal	"librosa， numpy兼容性问题  
cv_mdm_motion-generation	"依赖numpy版本兼容性问题：   File ""/opt/conda/lib/python3.8/site-packages/smplx/body_models.py"",  
cv_resnet50_ocr-detection-vlpt	numpy兼容性问题  
cv_clip-it_video-summarization_language-guided_en	tf兼容性问题

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13744636
* numpy and pandas no version

* modify compatible issue

* fix numpy compatible issue

* modify ci

* fix lint issue

* replace Image.ANTIALIAS to Image.Resampling.LANCZOS pillow compatible

* skip uncompatible cases

* fix numpy compatible issue, skip cases that can not compatbile numpy or tensorflow2.x

* skip compatible cases

* fix clip model issue

* fix body 3d keypoints compatible issue
 2023-08-22 23:04:31 +08:00
yanyi.ys
be26a62c48 人像生成代码评审 
该CR提交了关于人像生成的代码，能够通过给定人像图片以及相应的target姿势数据生成相应姿势的图片。

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13715612
 2023-08-21 18:42:24 +08:00
kangxiaoyang.kxy
6c3973b9a3 fix image colorization model load issue 
之前的评审提交（https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13669766?tab=changes）可能导致 modelscope hub 的模型无法读取，对于不同 modelscope 版本使用者产生兼容性问题，因此在上次评审提交的基础上做一定修改确保兼容性。
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13716448
* fix image colorization model load issue
 2023-08-21 09:47:32 +08:00
wenmeng.zwm
7f7573f15d fix ut error 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13716099
* fix ut error
 2023-08-19 21:16:07 +08:00
kangxiaoyang.kxy
1c76007da7 update image colorization model: remove redundant layers for less memory use 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13669766
* update image colorization model: remove redundant layers for less memory use
 2023-08-19 16:27:09 +08:00
xiangpeng.wxp
344d2a2e5b [to #42322933] add truct remote code for polylm 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13628684
* add truct remote code for polylm

* [to #42322933] add truct remote code for polylm

* [to #42322933] add truct remote code for polylm
 2023-08-14 15:37:42 +08:00
xiangpeng.wxp
5e83523c9a [to #42322933] Add polylm lora trainer to modelscope 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13509769
* [to #42322933] Add polylm lora trainer to modelscope
 2023-08-10 20:57:47 +08:00
chenyafeng.cyf
33605de759 eres2net_lre_v2 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13602081

* add eres2net_base_large_lre

* eres2net_language_identification

* eres2net_lre_v2
 2023-08-10 17:44:43 +08:00
zsl01670416
43a57fe110 support load dataset for llama 
support loading dataset for llama:
1.load dataset by MsDataset when parameters train dataset name and val dataset name were set. but there is no suitable dataset in hub.
2.load dataset by MsDataset when only parameter train dataset name was set, and then split into train dataset and validation dataset . 
3.load dataset by MsDataset when user set parameter src_txt, which is a file path such as 'alpaca_data.json', and then split into training dataset and validation dataset.
4.load dataset by build dataset from file in flex training.

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13505335
 2023-08-07 19:48:36 +08:00
lingcai.wl
7757f3c64a [to #51336898] support eas deploy, fix minor problems 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13541904
* [to #51336898] support eas deploy

* [to #51336898] support eas deploy, fix minor problems
 2023-08-07 19:48:07 +08:00
zsl01670416
b0699fd8e2 support llama2 inputs to device in function generate 
fix error inputs and model were not on the same device. if they are not on the same device, inputs will be implemented function to model device.
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13546989
* support llama inputs to device in function generate

* modify test qwen text generation according to github code
 2023-08-07 15:41:28 +08:00
wenmeng.zwm
63747824fa upgrade to 1.8.1 2023-08-03 16:01:41 +08:00
lukeming.lkm
9e033104af change use_fast_att and fix bfloat loading 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13522792

* check flash att is installed even if  use_fast_att is set True and fix bfloat loading

* skip pipeline model placement for quantization

* update unittest for qwen
 2023-08-03 15:59:51 +08:00
yuze.zyz
deda508acc fix streamer arg 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13520666
* fix streamer arg
 2023-08-03 14:34:52 +08:00
lukeming.lkm
b3a61ef6f4 update LICENSE 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13516119
 2023-08-03 11:07:34 +08:00
lukeming.lkm
914e9af3ad add LICENSE and related snippets 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13513677
 2023-08-03 09:39:20 +08:00
lukeming.lkm
561fbe9eb2 reformat stop_words_ids to sync QWen7b HF 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13510247
 2023-08-02 18:46:12 +08:00
lukeming.lkm
1ce78d648d add kwargs for save_vocabulary 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13508568
 2023-08-02 18:12:11 +08:00
lukeming.lkm
dece963ad5 resolve stopword logits processor 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13504522

* resolve stopword logits processor
 2023-08-02 16:14:48 +08:00
hejunjie.hjj
0c679d2e09 fix bug: referring_video-object-segmentation兼容Pillow10 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13479152
* fix bug: referring_video-object-segmentation兼容Pillow10
 2023-08-02 16:08:26 +08:00
lukeming.lkm
bd2f70a6eb add quantization in qwen pipelines and relevant unittests 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13499600

* add quant features

* resolve import

* resolve format

* fix save vocab
 2023-08-02 14:05:13 +08:00
lukeming.lkm
33bd74a7be add qwen 7b base and chat 
添加QWen 7b base模型和chat模型及相关pipelines
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13482235

* add qwen 7b base and chat

* fix logger

* update examples, lint test

* add unittest for qwen base and chat

* rename qwen to qwen-7b

* resolve imports and add a registry to text-generation

* reset load model from pretrained

* fix precheck

* skip qwen test case now

* remove strange file
 2023-08-02 09:25:21 +08:00
wenmeng zhou
1a6583eee2 Feat/chatglm2 32k support (#433) 
* add support for chatglm2 32k

* fix
 2023-08-01 23:22:09 +08:00
suluyan.sly
f080603802 remove requirement realesrgan for basicsr 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13488773
* remove requirement realesrgan for basicsr
 2023-08-01 16:55:47 +08:00
mulin.lyh
c1df82acd8 init pip to aliyuncs 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13487845

* init pip to aliyuncs
 2023-08-01 16:55:33 +08:00
suluyan.sly
dc17c7d6ee remove cv requirements: basicsr 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13471366
* rm req basicsr
 2023-07-31 18:20:23 +08:00
mulin.lyh
da2badaacd [to #51274424]fix: easycv extension and tinycudann conflict 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13471150
* fix easycv extension and tinycudann conflict
 2023-07-31 18:08:53 +08:00
yuze.zyz
f856c04785 Ignore http error 
忽略log打印，防止在check_model_is_latest时打出信息使用户不明所以。
将log改为Error栈
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13463005
 2023-07-31 17:14:12 +08:00
wenmeng.zwm
48a39244f1 Merge branch master-merge-github-230728 into master 
Title: Merge branch 'master-github' into master-merge-github-230728 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13456249
 2023-07-31 16:09:07 +08:00
suluyana
b68b90ba15 skip plugin 2023-07-30 00:30:30 +08:00
suluyana
9ece90ee84 skip plugin test case 2023-07-29 21:35:21 +08:00
suluyan.sly
d7cd2ce28e Merge branch 'master-github' into master-merge-github-230728 2023-07-29 10:42:54 +08:00
Jintao
312b63fe06 fix checkpoint, same device bug (#427) 2023-07-29 00:06:27 +08:00
zsl01670416
847607ab66 Merge branch 'debug_chatglm6b_json_dataset' fix conflict between hf dataset and to_hf_dataset 
The type of dataset built from file is hf dataset, which can not use function to_hf_dataset.
 2023-07-28 19:12:21 +08:00
suluyan.sly
05e1357c32 Merge branch 'master-github' into master-merge-github-230728 2023-07-28 16:40:34 +08:00
tastelikefeet
972298813b 1. add check model for training 2. add model_dir for automodel (#422) 2023-07-28 16:34:02 +08:00
wenmeng.zwm
3b485d5835 fix plugin python module missing files 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13453749
* fix plugin python module missing files
 2023-07-28 16:14:56 +08:00
frozoul
2566d028cd cv/cv nerf 3d reconstruction 4k nerf damo (#389) 
* add 4k-nerf core files

* update configure file

* update dataloader and model path

* update unittest

* Delete test_4k.py

* update unittest

* update unittest

* update pre-commit

* update dataloader

* update cuda code path

* check with pre-commit

---------

Co-authored-by: zhongshu.wzs <zhongshu.wzs@alibaba-inc.com>
Co-authored-by: wenmeng zhou <wenmeng.zwm@alibaba-inc.com>
 2023-07-28 10:37:13 +08:00
tongmu.wh
475924a421 correct language recognition taks name 
modelscope平台同学最终定下语种识别任务名为speech-language-recognition，对应进行代码中的相关改动
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13444517
* correct language recognition taks name
 2023-07-27 21:03:04 +08:00
Xingjun.Wang
3bd395a0c9 Merge pull request #423 from modelscope/dev/del_redundancy_code 
del redundant code
 2023-07-27 20:53:01 +08:00
xingjun.wang
6748cd7370 del redundancy code in func: get_dataset_meta_file_list 2023-07-27 20:49:41 +08:00
Wang Qiang
34ea2b474a Upgrade stable diffusion version to a more powerful version 2.1 (#415) 2023-07-27 20:40:38 +08:00
suluyan.sly
76921561e6 Merge branch master-merge-github-230727 into master 
Title: Merge branch 'master-github' into master-merge-github-230727 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13442869
 2023-07-27 20:18:35 +08:00
mulin.lyh
a501f7a417 fix easycv cpu extension build issue, cpu not support 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13441612
* fix easycv cpu extension build issue, cpu not support
 2023-07-27 17:42:37 +08:00
suluyan.sly
1c6f5fe775 Merge branch 'master-github' into master-merge-github-230727 
Conflicts:
       examples/pytorch/baichuan/finetune_baichuan.py
       examples/pytorch/chatglm6b/finetune.py
 2023-07-27 17:29:27 +08:00
Wang Qiang
dca6143b8b support safetensors weight pipeline (#421) 2023-07-27 16:49:01 +08:00
tastelikefeet
9802dfe93b fix device error (#419) 2023-07-27 16:23:21 +08:00
mengyang.fmy
18f998a85c add text-to-360pano-image pipeline, mod cv requirements 
7月份计划上线的360全景图生成模型，自研

模型权重文件地址https://www.modelscope.cn/models/damo/cv_diffusion_text-to-360panorama-image_generation/summary


#### 依赖项说明

##### 由于要使用xformers，torch版本最好使用1.13.1
```
pip install torch==1.13.1+cu116 torchvision==0.14.1+cu116 torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/cu116
```
##### 对应的diffusers和xformers版本如下
```
pip install -U diffusers==0.18.0
pip install xformers==0.0.16
pip install triton, accelerate, transformers
```

##### ModelScope Library 需要使用cv
```
pip install modelscope
pip install "modelscope[cv]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html
```

##### 此外，还需要安装第三方的一个库，Real-ESRGAN， 安装方法如下
```
# Install basicsr - https://github.com/xinntao/BasicSR
# We use BasicSR for both training and inference
pip install basicsr
# facexlib and gfpgan are for face enhancement
pip install facexlib
pip install gfpgan
pip install Pillow
pip install tqdm
pip install realesrgan==0.3.0
```
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13346430
* add text-to-360pano-image pipeline

* add text-to-360pano-image pipeline, mod cv requirements

* rm redundant files and cv requirements; add standard input and output definations

* fix diffusers==0.18.0 and run test

* fix diffusers==0.18.0 in multi-modal and run test again

* add model_revision='v1.0.0'

* fix yapf

* add trycatch for enabling xformers

* fix key error

* add install xformers in test/setup

* skip highres.fix in ci

* feat: Fix conflict, auto commit by WebIDE
 2023-07-27 11:33:39 +08:00