2921 Commits

Author SHA1 Message Date
tastelikefeet
9379504fd3 Fix some bugs (#1746) 2026-07-07 11:26:14 +08:00
Yunlin Mao
d5d13c1f25 Fix release workflow hub dependency (#1749) 2026-07-06 11:39:03 +08:00
Xingjun.Wang
5db3b88e2a add property session in HubApi (#1748) 2026-07-03 10:38:49 +08:00
Yunlin Mao
020cad8229 [Backport] Docker fixes from release/1.38 (#1735)
* Upgrade numpy to 2.x for 1.38 Docker images

- Replace deprecated numpy aliases (np.math.ceil → math.ceil, np.Inf → np.inf)
- Upgrade Docker constraints: numpy>=2.0, cython>=3.0, remove scipy upper bound

* Add ipywidgets dependency to Docker images

* update docker

* fix

* fix

* fix

* fix cpu image

* fix(docker): force numpy>=2.0 after evalscope install

ms-opencompass pulls numpy<2.0, downgrading numpy from 2.x to 1.26.4.
Force reinstall numpy>=2.0 after pip install .[eval] to restore it.

* add metax dockerfile and its requirements for ms-swift 4.2.x (#1734)

* Update npu dockerfile (#1736)

* fix(docker): remove unrelated added docker files

---------

Co-authored-by: Jintao Huang <huangjintao.hjt@alibaba-inc.com>
Co-authored-by: dwd <dwd1044898101@gmail.com>
Co-authored-by: addsubmuldiv <zyh13227@163.com>
2026-06-26 17:10:49 +08:00
Xingjun.Wang
39583ec639 fix: patch_context() use try/finally to ensure unpatch on exception (#1743)
- Wrapped yield in try/finally block so unpatch_hub() executes even when
  the with-body raises an exception, preventing patch state leakage
2026-06-25 16:04:11 +08:00
Xingjun.Wang
cf08fb242d [Fix] Fix studio cli (#1742) 2026-06-23 16:42:49 +08:00
tastelikefeet
cecb699cd0 fix (#1740) 2026-06-18 17:16:11 +08:00
addsubmuldiv
c1d88e6ab7 Update npu dockerfile (#1736) 2026-06-15 11:07:56 +08:00
tastelikefeet
c7dba8de1e Fix some security issues (#1737) 2026-06-15 10:35:50 +08:00
dwd
37eb369e9b add metax dockerfile and its requirements for ms-swift 4.2.x (#1734) 2026-06-15 10:04:01 +08:00
Xingjun.Wang
d3effb7370 [Update] Update modelscope-hub requirement (#1733) 2026-06-09 20:08:33 +08:00
Xingjun.Wang
50f8d37bc9 [Feat & Refactor] Refactor hub and CLI modules (#1732)
* refactor(hub): shim layer delegating to modelscope-hub

- Replace hub/api.py (4674→250 lines) with shim inheriting LegacyHubApi
- Replace hub/snapshot_download.py, callback.py with thin shims
- Partial shim hub/file_download.py (retain http_get_file)
- Shim hub/constants.py and errors.py with legacy aliases
- Shim hub/git.py, repository.py, cache_manager.py, upload_*.py
- Migrate CLI entry to modelscope_hub.cli.main:run_cmd
- Adapt 6 CLI commands as modelscope_hub.cli_plugins
- Delete redundant CLI files (download/upload/login/create/etc)
- Add modelscope-hub>=0.2.0 dependency, Python>=3.10
- Add __getattr__ proxy for forward-compatible method access
- Propagate timeout/max_retries to internal LegacyClient
- Bridge MODELSCOPE_CREDENTIALS_PATH env var to HubConfig

* fix lint: isort/yapf formatting + exclude hub/api.py from hooks

* set modelscope-hub>=0.0.5

* remove unused code

* refactor(hub): standardize token naming — git_token vs token

Disambiguate git token and SDK/API token naming across the hub layer:
- ModelScopeConfig: get_token/save_token → get_git_token/save_git_token
  (old names kept as deprecated aliases with DeprecationWarning)
- GitCommandWrapper: rename token params to git_token in clone/push/config
- Repository/DatasetRepository: auth_token → git_token (deprecated compat kept)
- data_loader.py: update caller to use get_git_token()

SDK token references (HubApi(token=...), get_cookies(access_token=...),
commit_scheduler.token) remain unchanged as they correctly use `token` naming.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* remove(msdatasets): remove all Virgo-related implementation

Remove the entire Virgo dataset subsystem which is no longer needed:
- Remove VirgoDataset class and VirgoDownloader
- Remove VirgoAuthConfig and VirgoDatasetConfig
- Remove Hubs.virgo enum value
- Remove fetch_virgo_meta from DataMetaManager
- Remove download_virgo_files from DatasetContextConfig
- Remove test_virgo_dataset.py test file
- Clean up unused imports (pandas, MaxComputeUtil, valid_url, etc.)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(hub): add OSS dataset operations and meta-file download to HubApi

Add methods that msdatasets depends on but don't belong in modelscope_hub:
- _legacy_request: internal helper combining legacy HTTP transport with
  application-level envelope validation (Code/Data/Message)
- list_oss_dataset_objects: list OSS storage objects for a dataset
- delete_oss_dataset_object / delete_oss_dataset_dir: delete OSS objects
- fetch_meta_files_from_url: download and cache meta CSV/JSONL files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix imports issue

* fix: address PR review feedback

- cli/plugins.py: change --yes and --all flags to action='store_true'
- hub/git.py: replace os.linesep with .splitlines() for cross-platform safety
- hub/__init__.py: use is_file() with fallback for robust credentials path detection

* fix lint

* update ms hub version

* fix(ci): add PyPI official as fallback index for pip

Aliyun mirror may lag behind PyPI for newly published packages,
causing dependency resolution failures (e.g. modelscope-hub>=0.0.6).
Add pypi.org/simple as extra-index-url so new versions are immediately
available while keeping the Aliyun mirror as the primary source.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix UTs

* remove unused UTs

* fix ut

* update modelscope-hub installation for source code

* fix UT

* fix uts

* fix ut

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-09 20:00:20 +08:00
Xingjun.Wang
1804ac6bf8 [Feature] Add studio module (#1727) 2026-06-05 14:33:25 +08:00
Xingjun.Wang
63ff6ec6e9 Addgated_mode for create repo (#1730) 2026-06-05 14:30:18 +08:00
Seas0
491e3df224 fix: quote version of timm specs in Dockerfile.ubuntu (#1728)
Co-authored-by: Seas0 <liusichen.seas0@bytedance.com>
2026-06-03 18:50:04 +08:00
AAAkater
2719754beb 🐛 fix(clearcache): fix single model cache path (#1724)
* 🐛 fix(clearcache): fix single model cache path

- Add 'models' subdirectory under 'hub' for single model cache paths
- Align dataset and model path structures for consistency

* 🐛 fix(clearcache): use cache root helpers to handle MODELSCOPE_CACHE correctly

Unconditionally prepending `hub/` broke `$MODELSCOPE_CACHE`-set users,
since the cache root has no `hub/` segment in that case. Reuse
get_model_cache_root() / get_dataset_cache_root() from file_utils,
which already handle both default and env-set layouts. Add tests
covering both scenarios.

---------

Co-authored-by: Yunnglin <mao.looper@qq.com>
2026-06-03 18:39:39 +08:00
tastelikefeet
8bab4b9bc1 Fix bug: 1. patch_context cannot revert classmethod 2. kernels testcase miss trust_remote_code and revision (#1729) 2026-05-31 16:11:21 +08:00
Xingjun.Wang
76cf0450a1 Fix timeout for create_model_tag (#1723)
* update timeout for create_model_tag

* fix review comments
2026-05-29 14:10:45 +08:00
tastelikefeet
70197537df Fix adaseq trust_remote_code (#1721) 2026-05-20 23:12:03 +08:00
雨泓
57961c1f4b fix pipelines in plugin 2026-05-19 15:05:21 +08:00
tastelikefeet
d52d5e67b2 Add whitelist to iic and damo (#1720) 2026-05-19 12:53:02 +08:00
tastelikefeet
cddbabaed5 Fix pipeline parameter (#1717) 2026-05-15 14:06:55 +08:00
xvxuopop
6f347a203c update npu dockerfile (#1716) 2026-05-14 10:36:31 +08:00
Xingjun.Wang
d968a2cf87 [Fix] Fix split detection (#1714) 2026-05-13 17:46:20 +08:00
Xingjun.Wang
a56e2ce08f [Fix] fix invalid commit (#1715) 2026-05-13 17:46:07 +08:00
tastelikefeet
d77bc8f83b fix trainer + trust_remote_code (#1713) 2026-05-12 19:16:13 +08:00
tastelikefeet
b7d47748ed fix plugin (#1712) 2026-05-12 17:40:06 +08:00
tastelikefeet
c2b8d9f020 Fix plugin rce (#1703) 2026-05-09 14:44:09 +08:00
Xingjun.Wang
2da1d94913 fix upload cache ignore (#1709) 2026-05-08 15:14:08 +08:00
Xingjun.Wang
c15c5261e1 [Fix] Fix dataset preview args, private streaming auth, and download retries (#1700) 2026-05-08 15:13:52 +08:00
dwd
2f5f52fc3c feat(docker/Metax): add metax dockerfile and its requirements for ms-swift 4.1.x (#1689) 2026-05-06 20:57:17 +08:00
tastelikefeet
68ab75af24 fix (#1708) 2026-05-06 20:54:46 +08:00
Xingjun.Wang
13064d9486 [Fix] Fix msdatasets split issue (#1704) 2026-05-05 23:49:23 +08:00
addsubmuldiv
c18f11ccbd update ascend dockerfile (#1687) 2026-04-29 16:06:57 +08:00
Xingjun.Wang
f0ba7bf885 [Fix] Add endpoint for creating model repo (#1699) 2026-04-29 01:37:24 +08:00
tastelikefeet
b3808b7c20 Support kernels downloading (#1697) 2026-04-28 10:29:19 +08:00
Xingjun.Wang
10298197e0 [Fix] Update helper desc (#1696) 2026-04-27 21:21:04 +08:00
Xingjun.Wang
16a79bc80b [Feat & Fix] Refactor endpoint arg for CLI (#1695) 2026-04-27 17:28:37 +08:00
Xingjun.Wang
b3599deb03 [Fix] Upload report (#1694) 2026-04-27 12:31:19 +08:00
Xingjun.Wang
4c34ac4b6c [Refactor&Fix] Upload folder module refactor (#1690) 2026-04-24 18:26:21 +08:00
Xingjun.Wang
1d81c4612a [Feature] Add openapi auth support (#1686) 2026-04-22 17:39:18 +08:00
Xingjun.Wang
862afeea7b [Refactor] Refactor the modelscope download module (#1683) 2026-04-21 16:00:56 +08:00
addsubmuldiv
55af6ce8e4 update ascend dockerfile (#1679) 2026-04-17 17:04:26 +08:00
Jintao
6059df3635 [bugfix] compat swift 4.0 (#1680) 2026-04-16 15:30:02 +08:00
Xingjun.Wang
9cc43a3140 Fix GPG KEY (#1678)
* refactor hf datasets utils and fix issue

* update datasets version

* fix download cli

* add engine=python for stream loading csv

* fix dataset builder for csv engine=python

* fix datasets csv OverflowError

* update UT commit scheduler test level

* fix python gpg key
2026-04-14 15:57:33 +08:00
Xingjun.Wang
fc0e2159a3 [Fix] Fix preview csv stream loading (#1665) 2026-04-14 14:17:23 +08:00
Xingjun.Wang
25de84cfc0 [Feature] Add producer-consumer pipeline for uploading folder (#1671) 2026-04-13 14:21:27 +08:00
Xingjun.Wang
32d7c7062c [Fix] fix token auth for downloading (#1674) 2026-04-11 14:07:46 +08:00
Xingjun.Wang
87ea8623cf [Fix] Enhance the robustness of ModelScope download cli (#1662) 2026-04-01 14:28:33 +08:00
Xingjun.Wang
d4301d192c [Fix | Refactor] Fix preview loading and refactor hf datasets utils (#1654) 2026-03-27 15:28:57 +08:00