Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2026-02-24 12:10:09 +01:00
Code Issues Packages Projects Releases Wiki Activity
2,609 Commits 267 Branches 107 Tags
feat/hf_pipeline
Commit Graph

1159 Commits

Author SHA1 Message Date
suluyan
d0c4cac232 fix test user 2025-02-11 15:04:07 +08:00
suluyan
05ce8a7dc7 remove the class method rewrite of save_pretrained from patch & fix lint 2025-02-11 14:56:55 +08:00
suluyan
021e912a38 fix create_repo 2025-02-11 14:31:34 +08:00
suluyan
4945d4340c add test_case 2025-02-11 12:50:30 +08:00
Yingda Chen
1f88654aa1 support multiple include/exclude filter patterns in command line (#1214) 
Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
 2025-02-07 16:02:37 +08:00
tastelikefeet
1cf7f4ff52 fix create_commit login (#1210) 2025-02-06 18:22:29 +08:00
tastelikefeet
f74433f6b2 Add more patches for hf (#1160) 2025-02-06 11:09:37 +08:00
Yingda Chen
65075191f7 clean directory structure (#1201) 
默认
模型：
~/.cache/modelscope/hub/models/{model_owner}/{model_name}
数据集：
~/.cache/modelscope/hub/datasets/{dataset_owner}/{dataset_name}

配置MODELSCOPE_CACHE环境变量：
模型：
$MODELSCOPE_CACHE/models/{model_owner}/{model_name}
数据集：
$MODELSCOPE_CACHE/datasets/{dataset_owner}/{dataset_name}

Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
 2025-01-23 17:31:49 +08:00
suluyana
dc7e22ebeb feat: new ollama template (#1199) 
* feat: deepseek-r1, olmo2, command-r7b, deepseek-v3

---------

Co-authored-by: suluyan <suluyan.sly@alibaba-inc.com>
 2025-01-21 17:49:54 +08:00
tastelikefeet
b1404cdc2e Fix typo and unused imports (#1196) 
* fix file exists of sub folder

* fix code doc

* add ci

* lint code

* fix unused imports and typo
 2025-01-18 10:22:22 +08:00
tastelikefeet
b892164d11 Fix file exists of sub folder (#1192) 
* fix file exists of sub folder

* fix code doc

* add ci

* lint code
 2025-01-18 09:59:14 +08:00
suluyana
9aa661118c feat: all other ollama models (#1174) 
* add cases

* new models

---------

Co-authored-by: suluyan <suluyan.sly@alibaba-inc.com>
 2025-01-10 15:42:22 +08:00
suluyana
1fe211ffe5 fix pipeline builder when model is not supported (#1125) 
* fix pipeline builder when model is not supported

* fix ci & skip
---------

Co-authored-by: suluyan.sly@alibaba-inc.com <suluyan.sly@alibaba-inc.com>
 2024-12-12 19:24:38 +08:00
tastelikefeet
7579fac6f3 fix ut (#1143) 2024-12-11 10:19:29 +08:00
suluyana
40b689795a Fix ollama template (#1141) 
* add marco o1

* add qwq

* feat: llama3.3

* fix space , add exaone3.5 , fix ministral
 2024-12-10 14:22:02 +08:00
tastelikefeet
d2ef1003ea Skip obsolete sd pipeline (#1131) 2024-12-05 20:07:32 +08:00
Yingda Chen
e32a9361e3 fix accuracy case sensitiveness (#1118) 
Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
 2024-12-03 21:25:18 +08:00
Yingda Chen
2b1c839918 Format llm pipeline (#1094) 
* format llm pipeline

Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
 2024-11-22 20:04:59 +08:00
Yingda Chen
63febc58a6 add llamafile support to command line (#1087) 
* add llamafile support to command line


Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
 2024-11-20 09:24:58 +08:00
suluyana
19fc9dc3f1 feat ollama template: llama3.2-vision (#1070) 2024-11-07 16:09:12 +08:00
suluyana
ae1541846b Template.to_ollama: add new argument split (#1039) 
feat: 1. add new argument `split` 2. ollama template
 2024-11-04 13:46:07 +08:00
Yingda Chen
83f1e20e80 add repo existence check hub-api (#1060) 
* add repo existence check api

* update

---------

Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
 2024-11-02 08:04:40 +08:00
Mark
250b72fce7 fix: text error correction batch run bug (#1052) 
Co-authored-by: Mark <smartmark-pro@qq.com>
 2024-11-01 09:35:19 +08:00
Mashiro
e71057cacf feat(audio/ans): Add ZipEnhancer and related layers for acoustic nois… (#1019) 
* feat(audio): Add acoustic noise suppression pipeline and tests for zipenhancer model

* Introduce `ZipEnhancer` module and associated layers (`ZipEnhancerLayer`, `Generator`, `ZipFormer`, ...).
* Add `speech_zipenhancer_ans_multiloss_16k_base` pipeline for `ZipEnhancer` module.
* Add new test cases and update metainfo.


Co-authored-by: Haoxu Wang <wanghaoxu.whx@alibaba-inc.com>
 2024-10-24 15:38:09 +08:00
suluyana
bacdc53327 fix: set device_map=cuda if using gptq in gpu (#1027) 2024-10-17 17:13:22 +08:00
Xingjun.Wang
eab95f909f del ignore_verifications and add trust_remote_code for local py_script (#1021) 2024-10-15 12:41:12 +08:00
tastelikefeet
b0f03ffd6d Fix template of eos_token (#1013) 2024-10-10 21:40:51 +08:00
suluyana
bd1544bef0 fix template: llava-llama-3 & yi-1.5 (#1011) 2024-10-10 10:01:45 +08:00
Xingjun.Wang
2c4505e13a Adapt new datasets (#1002) 
* update ds==3.0

* update

* add http_get_ms func

* del unused code

* fix pr issue and update requirements
 2024-09-30 16:46:00 +08:00
tastelikefeet
834db59952 template and ollama in modelscope (#995) 2024-09-29 19:13:50 +08:00
tastelikefeet
74d97ea7e0 Refactor zero sized file downloading (#991) 2024-09-14 15:12:28 +08:00
tastelikefeet
96e33878b4 lint code (#970) 2024-09-02 10:16:35 +08:00
yuze.zyz
8b5a5bd1e3 fix 2024-09-01 18:37:01 +08:00
Xingjun.Wang
3f7ce97319 Fix streaming load issues (#953) 
* fix streaming context

* add ut for stream
 2024-08-22 14:43:37 +08:00
liuyhwangyh
96dfccbd75 fix compatible of origin command line (#944) 
Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>
 2024-08-07 18:31:45 +08:00
liuyhwangyh
bf7c717e17 fix file download off by one bug (#943) 
* fix file download off by one bug, command line default download model, add repo-type parameter
---------

Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>
 2024-08-07 10:40:25 +08:00
mulin.lyh
da9f3e58e3 enable download branch 2024-07-31 18:10:44 +08:00
mulin.lyh
775c29ef79 optimize code 2024-07-31 17:01:06 +08:00
Ranqing
543a2b32d7 upload high quality human normal estimation model (#903) 
* upload human normal estimation model

* upload human normal estimation unittest

* update human normal estimation

* update test data

---------

Co-authored-by: 葭润 <ranqing.rq@alibaba-inc.com>
Co-authored-by: ranqing <ranqing@Sundays-Mac.local>
Co-authored-by: suluyana <suluyan_sly@163.com>
 2024-07-31 10:19:41 +08:00
Xingjun.Wang
210ab40c54 Upgrade datasets (#921) 
* del _datasets_server import in hf_dataset_util

* fix streaming for youku-mplug and adopt latest datasets

* fix download config copy

* update ut

* add youku in test_general_datasets

* update UT for general dataset

* adapt to datasets version: 2.19.0 or later

* add assert for youku data UT

* fix disable_tqdm in some functions for 2.19.0 or later

* update get_module_with_script

* set trust_remote_code is True in load_dataset_with_ctx

* update print info

* update requirements for datasets version restriction

* fix _dataset_info

* add pillow

* update comments

* update comment

* reuse _download function in DataDownloadManager

* remove unused code

* update test_run_modelhub in Human3DAnimationTest

* set datasets>=2.18.0
 2024-07-23 22:26:12 +08:00
liuyhwangyh
f7a32e48e3 add dataset download (#906) 
* add dataset download

* fix cr issue

* fix cv matplotlib issue

* refactor code

* fix ut issue

* remove debug code

* remove unused import

* fix import issue

* sleep 65s before start docker avoid kill and run failed

---------

Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>
 2024-07-12 19:06:44 +08:00
zhyncs
5b8e1e971c fix ignore_file_pattern (#886) 
* fix ignore_file_pattern

---------

Co-authored-by: root <root@kunkun>
 2024-07-01 14:31:39 +08:00
Firmament-cyou
8cdbeec3f6 Support swift model inference for llm pipeline (#880) 
* Support swift model inference for llm pipeline

* fix bug

* Add 'swfit' llm_framework and fix stream bug

* For pass the unittest run action
 2024-06-30 19:43:13 +08:00
yfchenmodelscope
f65f45959d Add Eres2netv2ep4 (#892) 
* add eres2netv2ep4
 2024-06-27 21:09:51 +08:00
mulin.lyh
4a22b05891 restore http_get_file interface 2024-05-28 15:09:05 +08:00
liuyhwangyh
17da5e2264 fix error report (#868) 
Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>
 2024-05-28 14:38:19 +08:00
liuyhwangyh
f93a184d88 add donwload command line and local_dir parameter (#866) 
* add donwload command line and local_dir parameter

Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>
 2024-05-25 14:21:55 +08:00
liuyhwangyh
5c470f8941 Download optimize (#862) 
* fix #845

Supports resumption of downloads from breakpoints, optimized download progress bar, finer display granularity, better experience under low bandwidth, and added function of downloading specified directories.

* restore push to hub

* fix merge issue

* fix ut issue

---------

Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>
 2024-05-24 15:37:43 +08:00
yfchenmodelscope
f9451bfe38 add sdpn and tdnn (#865) 
* add sdpn and tdnn
 2024-05-24 14:39:16 +08:00
liuyhwangyh
82ee20f447 fix issue #845 (#861) 
* fix #845

Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>
 2024-05-23 20:34:52 +08:00