Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2026-02-24 04:01:10 +01:00
Code Issues Packages Projects Releases Wiki Activity
2,548 Commits 267 Branches 107 Tags
add-name
Commit Graph

2548 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yingda Chen
08c2125f8e address cr 2025-01-15 13:14:29 +08:00
Yingda Chen
2e1b2e3587 add file name 2025-01-15 12:27:50 +08:00
Xingjun.Wang
f77b51e3c9 Support upload file and folder in the hub api (#1152) 
* update features

* update api

* add upload_file and thread_executor

* update upload file

* update api.py

* add cli for uploading

* run lint

* lint in msdataset

* temp

* add tqdm_desc in thread_executor

* update

* refine upload_file and upload_folder

* add endpoint for cli

* add uploading checker

* add path_or_fileobj and path_in_repo check in upload_file func

* add size limit to lfs: 1MB by default

* update lfs limit size: 10MB

* 5MB lfs limit

* fix test issue

* add pbar for upload_blob; del size_to_chunk_mb; fix allow_patterns and ignore_patterns

* fix commit uploaded blobs

* add update action for folder

* fix issues

* add normal files check

* update

* update

* set normal file size limit to 500MB

* update tqdm
 2025-01-14 23:59:53 +08:00
Yunlin Mao
bceb7196d1 update doc with llama_index (#1180) 2025-01-13 21:03:59 +08:00
suluyana
845a0bd5fe Fix/text gen (#1177) 
* fix text-gen: read pipeline type from configuration.json first

---------

Co-authored-by: suluyan <suluyan.sly@alibaba-inc.com>
 2025-01-13 10:05:42 +08:00
Xingjun.Wang
8f3358fe50 Add repo_id and repo_type in snapshot_download (#1172) 
* add repo_id and repo_type in snapshot_download

* fix positional args

* update
 2025-01-10 23:38:46 +08:00
Yunlin Mao
26812dc028 Unify datasets cache dir (#1178) 
* fix cache

* fix lint

* fix dataset cache

* fix lint

* remove
 2025-01-10 17:56:05 +08:00
suluyana
9aa661118c feat: all other ollama models (#1174) 
* add cases

* new models

---------

Co-authored-by: suluyan <suluyan.sly@alibaba-inc.com>
 2025-01-10 15:42:22 +08:00
suluyana
cc6fbbab5f logger.warning when using remote code (#1171) 
* logger warning when using remote code

Co-authored-by: suluyan <suluyan.sly@alibaba-inc.com>
 2025-01-10 15:38:05 +08:00
Yingda Chen
768f953b9d fix path contatenation to be windows compatabile (#1176) 
* fix path contatenation to be windows compatabile

* support dataset too

---------

Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
 2025-01-10 13:48:44 +08:00
tastelikefeet
ba15de012f fix https://www.modelscope.cn/models/iic/nlp_structbert_address-parsing_chinese_base/feedback/issueDetail/20431 (#1170) 2025-01-06 17:29:44 +08:00
suluyana
299ece8b8a support ms-swift 3.0.0 (#1166) 2025-01-02 17:20:32 +08:00
Xingjun.Wang
71a54c1a51 support latest datasets version (#1163) 2025-01-02 17:06:46 +08:00
tastelikefeet
d257fdeac6 fix lint (#1168) 2025-01-02 17:06:24 +08:00
tastelikefeet
a89bb73995 fix check model (#1134) 2025-01-02 16:08:10 +08:00
Xingjun.Wang
9ee1ff8cee Update downloading progress (#1167) 
* add MODELSCOPE_SHOW_PROGRESS_THRESHOLD

* lint

* update: MODELSCOPE_SHOW_INDIVIDUAL_PROGRESS_THRESHOLD
 2024-12-31 18:28:15 +08:00
Yingda Chen
391c6cdec9 handle resuming download as retry (#1147) 
Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
 2024-12-30 12:48:55 +08:00
Xingjun.Wang
8f3cf81982 Improve&fix dataset download command line (#1158) 
* fix dataset download cli

* fix default revision

* pre-commit

* add --max-workers for cli

* pre-commit

* update default max_workers
 2024-12-27 14:30:19 +08:00
suluyana
60780769b1 Fix/daily (#1155) 
* fix(llm ppl): 1. cache position; 2. stream_gready_search; 3. swift_mapping

* fix punkt

---------

Co-authored-by: suluyan <suluyan.sly@alibaba-inc.com>
 2024-12-23 09:55:12 +08:00
Yingda Chen
9304d40539 fix user-specified cache path (#1154) 
Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
 2024-12-18 09:01:41 +08:00
suluyana
1fe211ffe5 fix pipeline builder when model is not supported (#1125) 
* fix pipeline builder when model is not supported

* fix ci & skip
---------

Co-authored-by: suluyan.sly@alibaba-inc.com <suluyan.sly@alibaba-inc.com>
 2024-12-12 19:24:38 +08:00
Yunlin Mao
c3a9bcd803 Fix python push model no disk space (#1148) 
* fix python push model
 2024-12-12 16:46:00 +08:00
Yingda Chen
8ba3a185e5 use streaming hash validation (#1127) 
* support streaming hash
Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
 2024-12-11 18:59:42 +08:00
mushenL
b9f7b60d8f fix daily fail case 1210yk (#1142) 
* init commit

* init commit

* init commit

* add AutoImageProcessor and BatchFeature

* Modify format specifications

* fix daily fail case

* fix daily fail case
 2024-12-11 10:40:26 +08:00
tastelikefeet
7579fac6f3 fix ut (#1143) 2024-12-11 10:19:29 +08:00
suluyana
40b689795a Fix ollama template (#1141) 
* add marco o1

* add qwq

* feat: llama3.3

* fix space , add exaone3.5 , fix ministral
 2024-12-10 14:22:02 +08:00
Yunlin Mao
31bf308e44 downgrade moviepy to 1.0.3 (#1133) 2024-12-10 10:10:01 +08:00
tastelikefeet
d2ef1003ea Skip obsolete sd pipeline (#1131) 2024-12-05 20:07:32 +08:00
Yingda Chen
101b9249f8 fix missing import (#1126) 
Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
 2024-12-05 09:26:34 +08:00
Yingda Chen
8d66932e3f make hash validation optional (#1124) 
Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
 2024-12-04 18:46:48 +08:00
Yingda Chen
c67ffd4cbc optional hash 2024-12-04 18:25:52 +08:00
Yingda Chen
0411beeb05 fix windows path (#1119) 
Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
 2024-12-03 21:57:50 +08:00
Yingda Chen
140ac66c62 fix windows path 2024-12-03 21:51:35 +08:00
Yingda Chen
e32a9361e3 fix accuracy case sensitiveness (#1118) 
Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
 2024-12-03 21:25:18 +08:00
Yingda Chen
1950ec1839 add hash verficiation into cache file existence check (#1116) 
* add hash check into cache file existence check

Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
 2024-12-02 19:29:11 +08:00
Jintao
0daf77d489 remove unnecessary code (#1115) 
* remove the extra rows
---------

Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
 2024-12-02 15:34:23 +08:00
Yingda Chen
3017a70262 Handle unsupported Transformers class, and add more auto classes (#1113) 
* optimize unsupported transformer class and add more automodel

Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
 2024-12-02 15:17:17 +08:00
Yunlin Mao
a4582012ff fix tqdm bar (#1108) 2024-12-01 15:38:03 +08:00
Jintao
a721220fa1 release transformers version to 4.33-4.46 (#1111) 2024-12-01 15:32:33 +08:00
tastelikefeet
46c4ce9bdd support tag ci_image (#1112) 2024-12-01 15:17:25 +08:00
Jintao
b1e1f66fad Merge release 1.20 docker (#1109) 2024-11-29 13:44:36 +08:00
Yingda Chen
95cad91c21 add transformers compatability for Vision2seq (#1107) 
* add vision2seq
---------

Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
 2024-11-29 10:35:50 +08:00
Yingda Chen
3e13cc899b add transformer support for Qwen2vl (#1106) 
* add qwen2vlconfig

* rearrange

---------

Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
 2024-11-28 20:08:14 +08:00
Yingda Chen
e2bd302175 fix potential double definition for ocr pipeline (#1102) 
* fix potential double definition issue
---------

Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
 2024-11-28 00:43:24 +08:00
Yunlin Mao
a7856a5995 add multi-thread download (#1095) 
* add thread download

* add thread download

* fix print

* change default workers to 8

* fix return cache path

* manage tqdm progress bars

---------

Co-authored-by: DaozeZhang <zdz408@126.com>
 2024-11-26 20:22:36 +08:00
Yingda Chen
6d9e6d57c0 More automodel (#1098) 
* add more hf alias

---------

Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
 2024-11-25 22:16:05 +08:00
Yingda Chen
4a3b255d53 change warning to debug (#1099) 
Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
 2024-11-25 22:15:03 +08:00
Yingda Chen
5ca12c6cc4 Llamafile support gpu flag (#1097) 
* add gpu flag when gpu is detected

* fix typo

* fix typo

* add printout prompt

---------

Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
 2024-11-25 16:31:52 +08:00
Jintao
e3f63fd1ea lazy print ast logs (#1089) 2024-11-25 10:32:16 +08:00
Yingda Chen
2b1c839918 Format llm pipeline (#1094) 
* format llm pipeline

Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
 2024-11-22 20:04:59 +08:00