Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2026-02-25 04:30:48 +01:00
Code Issues Packages Projects Releases Wiki Activity
2,543 Commits 267 Branches 107 Tags
fix/trust_remote_code
Commit Graph

2543 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Zackary Shen
ba4db97507 upload cv_nerf_3d-reconstruction_vector-quantize-compression (#407) 
* add vq_compression model

* add vq_compression model

* check pre-commit for lint test

* fix by flake8

* update

* update

* update

* the last update

* the laast update

* update test_level>=0

---------

Co-authored-by: 剑匣 <zackary.sz@alibaba-inc.com>
 2023-07-26 17:20:13 +08:00
XDUWQ
3a629bb2b4 pre 2023-07-26 16:58:58 +08:00
XDUWQ
99aa707995 fix bugs 2023-07-26 16:35:40 +08:00
XDUWQ
25d67a0b83 pre 2023-07-26 16:08:24 +08:00
XDUWQ
70da8b7809 fix bugs 2023-07-26 15:54:37 +08:00
XDUWQ
a7678bbb77 precommit 2023-07-26 15:06:42 +08:00
XDUWQ
c0b62cb0fb fix bugs 2023-07-26 15:03:46 +08:00
Xingjun.Wang
41c2c95083 dataset fix version (#413) 
* modify datasets version
 2023-07-25 22:18:25 +08:00
mulin.lyh
a780935317 torch1.11 and torch2.0.1 build script refactor 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13408751

* [to #51122016]feat: build image support torch1.x 2.x cuda11.3.0 11.7.1

* remove base  version

* env cudatookit-version

* build script merge

* torch 1.11.0 fix
 2023-07-25 21:15:00 +08:00
jiaqi.sjq
1f05acef88 fix pip install error when pysptk>=0.1.19 2023-07-25 21:13:31 +08:00
tongmu.wh
ba1a333ba6 add language recognition pipelines and models 
新增语种识别pipeline和model
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13385083
* add language recognition pipelines and models

* add a clustering method for speaker diarization

* define input and output type for language recognition
 2023-07-25 21:07:56 +08:00
tastelikefeet
0db3d1d53b Fix bug of amp and device_map (#397) 
* fix amp

* remove useless code

* Fix bug
 2023-07-25 19:28:00 +08:00
Jintao
f03898626e ckpt output directory ignore *.safetensors (#410) 
* ckpt output file ignore *.safetensors

* update
 2023-07-25 19:27:11 +08:00
hemu.zp
fc54593a56 fix baichuan eval and support sequence_length 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13404289
* fix baichuan eval

* support sequence_length and ppl

* fix typo

* fix bug for palm

* fix bug
 2023-07-25 19:10:45 +08:00
zsl01670416
9926ad685b support getting labels from dataset in sbert text classification and building dataset from file in chatglm-6b 
1.Add getting labels from dataset in "text_classificationfinetune_text_classification.py" to simplify user's operation in flex training. Parameters "--num_labels" and "--labels" were removed in "run_train.sh".
2.In "chatglm6b / finetune.py", building dataset from file  is necessary to support flex training.
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13382745
* support getting labels from dataset in sbert text classification and building dataset from file in chatglm-6b

* support getting labels from dataset in sbert text classification and building dataset from file in chatglm-6b

* remove repetitive labels in a concise manner of using set

* reserve parameter labels in finetune_text_classification

* Merge branch 'master' of http://gitlab.alibaba-inc.com/Ali-MaaS/MaaS-lib

reserve parameter labels in finetune_text_classification

* Merge branch 'support_text_cls_labels_chatglm_json'
reserve parameter labels in finetune_text_classification
 2023-07-25 19:02:32 +08:00
zeyinzi.jzyz
672c4899e9 add sd swift tuner 
SD-Tuner base on Swift (LoRA/Adapter/Prompt)
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13380798

* sd swift tuner

* fix pre-checker
 2023-07-25 19:00:49 +08:00
shuli.cly
526e1371f5 Merge the speaker-turn-detection codes, local test finished 
# Speaker Diarization Speaker-Turn Detection CR

和Dialogue-Detection一样，本模型是Speaker Diarization（`audio/speaker diarization`，语音/说话人日志）任务下的一个子模块。

本次提交的是基于文本进行判断的模型，本地模型的初始模型基于huggingface训练的，此提交中复用了部分 `nlp/token-classification` 模型的代码。为了方便后续维护以及与nlp方面代码解耦，在model、pipeline以及preprocessor中 **单独** 创建了相应模块并重新register。
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13364720
* std first commit

* local test pass for speaker-turn-detection

* update speaker-turn-detection pipeline task outputs format; update pipeline outputs; update test scripts
 2023-07-25 18:57:47 +08:00
hemu.zp
80f76ca475 Support stream output for transformers model 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13271136
* support stream for transformers model

* set test_level >= 2

* support hf model and chatglm2

* remove streaming_output for chatglm2
 2023-07-25 17:41:32 +08:00
hemu.zp
ed6e139759 Support llama & lora finetune without deepspeed 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13131145
* support llama + lora without deepspeed

* feat: Fix conflict, auto commit by WebIDE
 2023-07-25 17:32:46 +08:00
mulin.lyh
fb644002fa python build in type, return directly 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13380943
* python build in type, return directly

* fix output images bug

* compatible with demo service
 2023-07-25 16:56:01 +08:00
XDUWQ
3412a074c5 precommit 2023-07-25 15:00:28 +08:00
XDUWQ
ffbf77fcf2 update 2023-07-25 14:47:45 +08:00
wenmeng zhou
7b249400d9 refine class wrapper (#409) 2023-07-25 10:13:02 +08:00
zsl01670416
1e5e363235 fix issue of using cuda device in document segmentation pipeline inference 
Fix issue of using cuda device in document segmentation pipeline inference.Time cost will be reduced when prediction is implemented.
 2023-07-25 10:01:20 +08:00
Wang Qiang
0f551d3983 Merge pull request #408 from XDUWQ/lora_rank 
Add lora_rank parameter for lora stable diffusion
 2023-07-24 22:57:06 +08:00
XDUWQ
8e157cfa15 precommit 2023-07-24 22:10:52 +08:00
wenmeng zhou
64203e89ee Compatibility for huggingface transformers (#391) 2023-07-24 20:53:27 +08:00
XDUWQ
8e00d85317 fix bugs 2023-07-24 19:46:22 +08:00
XDUWQ
426f55d57b add lora_rank for lora stable diffusion 2023-07-24 19:43:20 +08:00
XDUWQ
bc93e2dc96 add lora_rank for lora stable diffusion 2023-07-24 19:32:04 +08:00
XDUWQ
eb24e23d19 add lora_rank for lora stable diffusion 2023-07-24 19:24:52 +08:00
XDUWQ
6fb340e7f8 add lora_rank for lora stable diffusion 2023-07-24 19:17:49 +08:00
Jintao
ba4b9fc43f Added full parameter sft to llm (#402) 
* Optimized code

* update parse_args

* fix get_logger bug

* update parse_args

* Added full parameter fine-tuning

* Add support_bf16 warning

* Modify the code format and fix bugs
 2023-07-24 15:52:09 +08:00
lylalala
f805d86aed llama2 support chat (#404) 
* support chat

* update llama2 chat testcase

* add gen kwargs and devices

* update unittest and support max_length in multi-turn dialogue
 2023-07-24 15:38:01 +08:00
Ikko Eltociear Ashimine
9dedac31fd Add Japanese README (#405) 2023-07-24 11:14:16 +08:00
tingwei.gtw
d16522723a [to #42322933] add files 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13158565
* [to #42322933] add files

* [to #42322933] add files

* [to #42322933] add files

* [to #42322933] add files

* [to #42322933] add files

* update test data

* [to #42322933] add files

* Merge remote-tracking branch 'origin' into feature/sal_try_on

* [to #42322933] add files

* Merge remote-tracking branch 'origin' into feature/sal_try_on
 2023-07-24 10:16:29 +08:00
mushenL
f77237b049 add llama2 pipeline (#399) 
* Modify the parameter passing of the text_generation_pipeline class

* add llama2 pipeline

* add llama pipeline v1.1

* add llama pipeline v1.2

* add llama pipeline v1.3

* add llama pipeline v1.0.4
 2023-07-22 21:53:04 +08:00
Xingjun.Wang
7608868290 Merge pull request #403 from modelscope/dev/check_lint 
fix yapf error
v1.7.2 		2023-07-22 00:01:01 +08:00
xingjun.wang
5a49093422 fix yapf 2023-07-21 23:37:17 +08:00
xingjun.wang
96a5282021 check format 2023-07-21 23:09:24 +08:00
LingFeng.Chen.Cn
9c4d9cdb68 fix file not found cause TypeError (#385) 
* fix file not found cause TypeError

* fix linter test

* fix linter test

* format code

---------

Co-authored-by: Krasus <chen@krasus.red>
Co-authored-by: wenmeng.zwm <wenmeng.zwm@alibaba-inc.com>
 2023-07-21 15:34:19 +08:00
shuli.cly
13e345f6d9 add sv/speaker_diarization_dialogue_detection to branch sv/semantic-dialogue-detection 
# Speaker Diarization Dialogue Detection CR

本模型是Speaker Diarization（`audio/speaker diarization`，语音/说话人日志）任务下的一个子模块。

本次提交的是基于文本进行判断的模型，其IO和中间过程和 `nlp/text-classification` 很像，且本地模型的初始模型也是基于huggingface训练的，因此此提交中复用了部分 `nlp/text-classification` 模型的代码。为了方便后续维护以及与nlp方面代码解耦，在model、pipeline以及preprocessor中 **单独** 创建了相应模块并重新register。
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13269649
* start to add speaker_diarization_dialogue_detection files; Need to change constant and test

* add sv/speaker_diarization_dialogue_detection to branch sv/semantic-dialogue-detection

* update test case

* add comments for speaker diarization dialogue detection pipelines

* add outputs type and inputs type for speaker_diarization_dialogue_detection
 2023-07-20 19:29:59 +08:00
shenweichao.swc
05c65ba225 add s2net for panorama_depth_estimation 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13310819
* add s2net codes

* fix sphdecoder

* Merge branch 'master' into dev/cv_s2net_panorama_depth_estimation

* revise comments in the pipeline

* revise the code
 2023-07-20 19:28:03 +08:00
xiangpeng.wxp
4085d821f3 [to #42322933] add polylm, a polyglot large language model 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13339595
 2023-07-20 18:21:07 +08:00
tongmu.wh
c859bea022 fix vad time stamp in sd pipeline 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13316765
* fix details of speaker models

* Merge branch 'master' into speaker-fix

* fix vad time stamp in sd pipeline
 2023-07-20 17:32:51 +08:00
LingFeng.Chen.Cn
19ea082fec Update speaker_verification_pipeline.py (#364) 
* Update speaker_verification_pipeline.py

* fix log too long when input tuple(bytes,bytes)

* Update speaker_verification_pipeline.py

* fix when bytes print too much

* Update speaker_verification_pipeline forward log

* pre-commit linter format


---------

Co-authored-by: wenmeng.zwm <wenmeng.zwm@alibaba-inc.com>
 2023-07-20 09:28:55 +08:00
tastelikefeet
78b3e74fd8 pre-commit passed (#395) 2023-07-19 20:59:58 +08:00
Jintao
2f7c669f33 support llama2 (#393) 
* Unify sft and infer code into a single file

* update llama2 sft infer
 2023-07-19 17:34:27 +08:00
tastelikefeet
3283a46c52 Add Llama2 from hf (#394) 
* update llama code

* add llama2

* revert llama code

* add llama2 registry

* fix bug
 2023-07-19 17:22:03 +08:00
XDUWQ
66795aa3ff change tests level 2023-07-19 09:41:21 +08:00