Commit Graph

186 Commits

Author SHA1 Message Date
Jintao Huang
01d07f84a4 update 2026-07-13 11:21:14 +08:00
Jintao Huang
4648455f52 Merge branch 'release/1.38' into build_swift_image 2026-07-13 11:16:57 +08:00
Jintao Huang
e2d1833fa8 update 2026-06-29 11:50:55 +08:00
Jintao Huang
206eb7f111 fix tilelang 2026-06-17 23:44:47 +08:00
Jintao Huang
b350f72373 update swift 4.3.1 2026-06-16 19:59:41 +08:00
Jintao Huang
f2b3350a83 Merge branch 'release/1.37' into build_swift_image 2026-06-16 19:52:34 +08:00
Yunnglin
52e227c48b fix(docker): force numpy>=2.0 after evalscope install
ms-opencompass pulls numpy<2.0, downgrading numpy from 2.x to 1.26.4.
Force reinstall numpy>=2.0 after pip install .[eval] to restore it.
2026-06-15 17:43:30 +08:00
addsubmuldiv
8622c814d8 Update npu dockerfile (#1736) 2026-06-15 17:43:03 +08:00
dwd
113c93b8e7 add metax dockerfile and its requirements for ms-swift 4.2.x (#1734) 2026-06-15 17:42:03 +08:00
Jintao Huang
f599019206 fix cpu image 2026-06-11 15:15:28 +08:00
Jintao Huang
c0416acc9a fix 2026-06-11 15:15:27 +08:00
Jintao Huang
29442858e9 fix 2026-06-11 15:15:26 +08:00
Jintao Huang
7053c9d6cc fix 2026-06-11 15:15:25 +08:00
Jintao Huang
faae851237 update docker 2026-06-11 15:15:23 +08:00
Yunnglin
1a53418643 Add ipywidgets dependency to Docker images 2026-06-10 14:16:53 +08:00
Xingjun.Wang
680c3638e0 [Feat & Refactor] Refactor hub and CLI modules (#1732)
* refactor(hub): shim layer delegating to modelscope-hub

- Replace hub/api.py (4674→250 lines) with shim inheriting LegacyHubApi
- Replace hub/snapshot_download.py, callback.py with thin shims
- Partial shim hub/file_download.py (retain http_get_file)
- Shim hub/constants.py and errors.py with legacy aliases
- Shim hub/git.py, repository.py, cache_manager.py, upload_*.py
- Migrate CLI entry to modelscope_hub.cli.main:run_cmd
- Adapt 6 CLI commands as modelscope_hub.cli_plugins
- Delete redundant CLI files (download/upload/login/create/etc)
- Add modelscope-hub>=0.2.0 dependency, Python>=3.10
- Add __getattr__ proxy for forward-compatible method access
- Propagate timeout/max_retries to internal LegacyClient
- Bridge MODELSCOPE_CREDENTIALS_PATH env var to HubConfig

* fix lint: isort/yapf formatting + exclude hub/api.py from hooks

* set modelscope-hub>=0.0.5

* remove unused code

* refactor(hub): standardize token naming — git_token vs token

Disambiguate git token and SDK/API token naming across the hub layer:
- ModelScopeConfig: get_token/save_token → get_git_token/save_git_token
  (old names kept as deprecated aliases with DeprecationWarning)
- GitCommandWrapper: rename token params to git_token in clone/push/config
- Repository/DatasetRepository: auth_token → git_token (deprecated compat kept)
- data_loader.py: update caller to use get_git_token()

SDK token references (HubApi(token=...), get_cookies(access_token=...),
commit_scheduler.token) remain unchanged as they correctly use `token` naming.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* remove(msdatasets): remove all Virgo-related implementation

Remove the entire Virgo dataset subsystem which is no longer needed:
- Remove VirgoDataset class and VirgoDownloader
- Remove VirgoAuthConfig and VirgoDatasetConfig
- Remove Hubs.virgo enum value
- Remove fetch_virgo_meta from DataMetaManager
- Remove download_virgo_files from DatasetContextConfig
- Remove test_virgo_dataset.py test file
- Clean up unused imports (pandas, MaxComputeUtil, valid_url, etc.)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(hub): add OSS dataset operations and meta-file download to HubApi

Add methods that msdatasets depends on but don't belong in modelscope_hub:
- _legacy_request: internal helper combining legacy HTTP transport with
  application-level envelope validation (Code/Data/Message)
- list_oss_dataset_objects: list OSS storage objects for a dataset
- delete_oss_dataset_object / delete_oss_dataset_dir: delete OSS objects
- fetch_meta_files_from_url: download and cache meta CSV/JSONL files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix imports issue

* fix: address PR review feedback

- cli/plugins.py: change --yes and --all flags to action='store_true'
- hub/git.py: replace os.linesep with .splitlines() for cross-platform safety
- hub/__init__.py: use is_file() with fallback for robust credentials path detection

* fix lint

* update ms hub version

* fix(ci): add PyPI official as fallback index for pip

Aliyun mirror may lag behind PyPI for newly published packages,
causing dependency resolution failures (e.g. modelscope-hub>=0.0.6).
Add pypi.org/simple as extra-index-url so new versions are immediately
available while keeping the Aliyun mirror as the primary source.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix UTs

* remove unused UTs

* fix ut

* update modelscope-hub installation for source code

* fix UT

* fix uts

* fix ut

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-09 22:23:14 +08:00
Seas0
204caba8b2 fix: quote version of timm specs in Dockerfile.ubuntu (#1728)
Co-authored-by: Seas0 <liusichen.seas0@bytedance.com>
2026-06-09 20:00:52 +08:00
Yunnglin
b87b344f47 Upgrade numpy to 2.x for 1.38 Docker images
- Replace deprecated numpy aliases (np.math.ceil → math.ceil, np.Inf → np.inf)
- Upgrade Docker constraints: numpy>=2.0, cython>=3.0, remove scipy upper bound
2026-06-08 14:52:00 +08:00
Jintao Huang
84243dafdf update 2026-06-04 19:27:42 +08:00
Jintao Huang
2be3e5c2ea use cuda 13.0.2 2026-06-04 19:27:30 +08:00
Jintao Huang
3a82ba027f fix 2026-06-01 10:38:47 +08:00
Jintao Huang
f9d7afa075 update swift image 4.2.3 2026-05-31 22:20:28 +08:00
Jintao Huang
121f4150fa fix cpu image 2026-05-21 16:27:12 +08:00
Jintao Huang
6bcb1f195b fix 2026-05-18 16:09:55 +08:00
Jintao Huang
99602db273 fix 2026-05-18 16:06:45 +08:00
Jintao Huang
49de5408fa Merge branch 'master' into release/1.37 2026-05-15 14:09:10 +08:00
xvxuopop
6f347a203c update npu dockerfile (#1716) 2026-05-14 10:36:31 +08:00
Jintao Huang
263a3ef0b2 fix 2026-05-10 22:58:21 +08:00
Jintao Huang
16991606f4 update docker 2026-05-10 18:00:21 +08:00
Jintao Huang
203f0bfa27 Merge branch 'release/1.36' into build_swift_image 2026-05-10 14:08:19 +08:00
Jintao Huang
5a61fcebb2 update swift 4.2 image 2026-05-10 14:04:02 +08:00
dwd
2f5f52fc3c feat(docker/Metax): add metax dockerfile and its requirements for ms-swift 4.1.x (#1689) 2026-05-06 20:57:17 +08:00
Jintao Huang
06e6311a0c fix 2026-04-30 23:47:18 +08:00
Jintao Huang
537722acea update cuda13.0 2026-04-30 23:16:26 +08:00
addsubmuldiv
c18f11ccbd update ascend dockerfile (#1687) 2026-04-29 16:06:57 +08:00
Jintao Huang
5c160e97c4 update swift 4.1.3 2026-04-25 21:26:56 +08:00
班扬
e1635d6e94 set peft<0.19 2026-04-22 17:09:56 +08:00
Xingjun.Wang
862afeea7b [Refactor] Refactor the modelscope download module (#1683) 2026-04-21 16:00:56 +08:00
班扬
0d745602af ok Merge branch 'master' of github.com:modelscope/modelscope into release/1.36 2026-04-20 11:15:05 +08:00
Jintao Huang
07b0321e78 fix 2026-04-20 07:12:09 +08:00
Jintao Huang
ae70780564 fix 2026-04-19 15:47:45 +08:00
Jintao Huang
6cc2aba435 fix 2026-04-19 14:19:09 +08:00
Jintao Huang
39adcb2ca7 fix 2026-04-19 14:16:07 +08:00
Jintao Huang
408485c1c4 update swift 4.1 2026-04-19 14:02:29 +08:00
Jintao Huang
6c9aba98c7 Merge remote-tracking branch 'origin/release/1.35' into build_swift_image 2026-04-19 13:27:35 +08:00
addsubmuldiv
55af6ce8e4 update ascend dockerfile (#1679) 2026-04-17 17:04:26 +08:00
班扬
195feeb12f remove pynini for stable image 2026-04-14 18:01:54 +08:00
班扬
454a180dea fix docker pip version 2026-04-14 16:17:50 +08:00
Xingjun.Wang
9cc43a3140 Fix GPG KEY (#1678)
* refactor hf datasets utils and fix issue

* update datasets version

* fix download cli

* add engine=python for stream loading csv

* fix dataset builder for csv engine=python

* fix datasets csv OverflowError

* update UT commit scheduler test level

* fix python gpg key
2026-04-14 15:57:33 +08:00
Jintao Huang
d92b58affa update transformers 2026-03-29 12:26:57 +08:00