Commit Graph

195 Commits

Author SHA1 Message Date
hjh0119
a9beacd5ea merge master 2026-08-03 20:49:52 +08:00
hjh0119
59dd3cda45 update swift 4.4.3 2026-08-03 20:43:15 +08:00
suluyana
0fd381159a Backport/release 1.39 to master (#1767) 2026-08-03 16:16:54 +08:00
addsubmuldiv
2905da1220 update npu dockerfile (#1755) 2026-07-23 16:30:26 +08:00
Jintao Huang
01d07f84a4 update 2026-07-13 11:21:14 +08:00
Jintao Huang
4648455f52 Merge branch 'release/1.38' into build_swift_image 2026-07-13 11:16:57 +08:00
Jintao Huang
e2d1833fa8 update 2026-06-29 11:50:55 +08:00
Yunlin Mao
020cad8229 [Backport] Docker fixes from release/1.38 (#1735)
* Upgrade numpy to 2.x for 1.38 Docker images

- Replace deprecated numpy aliases (np.math.ceil → math.ceil, np.Inf → np.inf)
- Upgrade Docker constraints: numpy>=2.0, cython>=3.0, remove scipy upper bound

* Add ipywidgets dependency to Docker images

* update docker

* fix

* fix

* fix

* fix cpu image

* fix(docker): force numpy>=2.0 after evalscope install

ms-opencompass pulls numpy<2.0, downgrading numpy from 2.x to 1.26.4.
Force reinstall numpy>=2.0 after pip install .[eval] to restore it.

* add metax dockerfile and its requirements for ms-swift 4.2.x (#1734)

* Update npu dockerfile (#1736)

* fix(docker): remove unrelated added docker files

---------

Co-authored-by: Jintao Huang <huangjintao.hjt@alibaba-inc.com>
Co-authored-by: dwd <dwd1044898101@gmail.com>
Co-authored-by: addsubmuldiv <zyh13227@163.com>
2026-06-26 17:10:49 +08:00
Jintao Huang
206eb7f111 fix tilelang 2026-06-17 23:44:47 +08:00
Jintao Huang
b350f72373 update swift 4.3.1 2026-06-16 19:59:41 +08:00
Jintao Huang
f2b3350a83 Merge branch 'release/1.37' into build_swift_image 2026-06-16 19:52:34 +08:00
Yunnglin
52e227c48b fix(docker): force numpy>=2.0 after evalscope install
ms-opencompass pulls numpy<2.0, downgrading numpy from 2.x to 1.26.4.
Force reinstall numpy>=2.0 after pip install .[eval] to restore it.
2026-06-15 17:43:30 +08:00
addsubmuldiv
8622c814d8 Update npu dockerfile (#1736) 2026-06-15 17:43:03 +08:00
dwd
113c93b8e7 add metax dockerfile and its requirements for ms-swift 4.2.x (#1734) 2026-06-15 17:42:03 +08:00
addsubmuldiv
c1d88e6ab7 Update npu dockerfile (#1736) 2026-06-15 11:07:56 +08:00
dwd
37eb369e9b add metax dockerfile and its requirements for ms-swift 4.2.x (#1734) 2026-06-15 10:04:01 +08:00
Jintao Huang
f599019206 fix cpu image 2026-06-11 15:15:28 +08:00
Jintao Huang
c0416acc9a fix 2026-06-11 15:15:27 +08:00
Jintao Huang
29442858e9 fix 2026-06-11 15:15:26 +08:00
Jintao Huang
7053c9d6cc fix 2026-06-11 15:15:25 +08:00
Jintao Huang
faae851237 update docker 2026-06-11 15:15:23 +08:00
Yunnglin
1a53418643 Add ipywidgets dependency to Docker images 2026-06-10 14:16:53 +08:00
Xingjun.Wang
680c3638e0 [Feat & Refactor] Refactor hub and CLI modules (#1732)
* refactor(hub): shim layer delegating to modelscope-hub

- Replace hub/api.py (4674→250 lines) with shim inheriting LegacyHubApi
- Replace hub/snapshot_download.py, callback.py with thin shims
- Partial shim hub/file_download.py (retain http_get_file)
- Shim hub/constants.py and errors.py with legacy aliases
- Shim hub/git.py, repository.py, cache_manager.py, upload_*.py
- Migrate CLI entry to modelscope_hub.cli.main:run_cmd
- Adapt 6 CLI commands as modelscope_hub.cli_plugins
- Delete redundant CLI files (download/upload/login/create/etc)
- Add modelscope-hub>=0.2.0 dependency, Python>=3.10
- Add __getattr__ proxy for forward-compatible method access
- Propagate timeout/max_retries to internal LegacyClient
- Bridge MODELSCOPE_CREDENTIALS_PATH env var to HubConfig

* fix lint: isort/yapf formatting + exclude hub/api.py from hooks

* set modelscope-hub>=0.0.5

* remove unused code

* refactor(hub): standardize token naming — git_token vs token

Disambiguate git token and SDK/API token naming across the hub layer:
- ModelScopeConfig: get_token/save_token → get_git_token/save_git_token
  (old names kept as deprecated aliases with DeprecationWarning)
- GitCommandWrapper: rename token params to git_token in clone/push/config
- Repository/DatasetRepository: auth_token → git_token (deprecated compat kept)
- data_loader.py: update caller to use get_git_token()

SDK token references (HubApi(token=...), get_cookies(access_token=...),
commit_scheduler.token) remain unchanged as they correctly use `token` naming.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* remove(msdatasets): remove all Virgo-related implementation

Remove the entire Virgo dataset subsystem which is no longer needed:
- Remove VirgoDataset class and VirgoDownloader
- Remove VirgoAuthConfig and VirgoDatasetConfig
- Remove Hubs.virgo enum value
- Remove fetch_virgo_meta from DataMetaManager
- Remove download_virgo_files from DatasetContextConfig
- Remove test_virgo_dataset.py test file
- Clean up unused imports (pandas, MaxComputeUtil, valid_url, etc.)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(hub): add OSS dataset operations and meta-file download to HubApi

Add methods that msdatasets depends on but don't belong in modelscope_hub:
- _legacy_request: internal helper combining legacy HTTP transport with
  application-level envelope validation (Code/Data/Message)
- list_oss_dataset_objects: list OSS storage objects for a dataset
- delete_oss_dataset_object / delete_oss_dataset_dir: delete OSS objects
- fetch_meta_files_from_url: download and cache meta CSV/JSONL files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix imports issue

* fix: address PR review feedback

- cli/plugins.py: change --yes and --all flags to action='store_true'
- hub/git.py: replace os.linesep with .splitlines() for cross-platform safety
- hub/__init__.py: use is_file() with fallback for robust credentials path detection

* fix lint

* update ms hub version

* fix(ci): add PyPI official as fallback index for pip

Aliyun mirror may lag behind PyPI for newly published packages,
causing dependency resolution failures (e.g. modelscope-hub>=0.0.6).
Add pypi.org/simple as extra-index-url so new versions are immediately
available while keeping the Aliyun mirror as the primary source.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix UTs

* remove unused UTs

* fix ut

* update modelscope-hub installation for source code

* fix UT

* fix uts

* fix ut

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-09 22:23:14 +08:00
Seas0
204caba8b2 fix: quote version of timm specs in Dockerfile.ubuntu (#1728)
Co-authored-by: Seas0 <liusichen.seas0@bytedance.com>
2026-06-09 20:00:52 +08:00
Xingjun.Wang
50f8d37bc9 [Feat & Refactor] Refactor hub and CLI modules (#1732)
* refactor(hub): shim layer delegating to modelscope-hub

- Replace hub/api.py (4674→250 lines) with shim inheriting LegacyHubApi
- Replace hub/snapshot_download.py, callback.py with thin shims
- Partial shim hub/file_download.py (retain http_get_file)
- Shim hub/constants.py and errors.py with legacy aliases
- Shim hub/git.py, repository.py, cache_manager.py, upload_*.py
- Migrate CLI entry to modelscope_hub.cli.main:run_cmd
- Adapt 6 CLI commands as modelscope_hub.cli_plugins
- Delete redundant CLI files (download/upload/login/create/etc)
- Add modelscope-hub>=0.2.0 dependency, Python>=3.10
- Add __getattr__ proxy for forward-compatible method access
- Propagate timeout/max_retries to internal LegacyClient
- Bridge MODELSCOPE_CREDENTIALS_PATH env var to HubConfig

* fix lint: isort/yapf formatting + exclude hub/api.py from hooks

* set modelscope-hub>=0.0.5

* remove unused code

* refactor(hub): standardize token naming — git_token vs token

Disambiguate git token and SDK/API token naming across the hub layer:
- ModelScopeConfig: get_token/save_token → get_git_token/save_git_token
  (old names kept as deprecated aliases with DeprecationWarning)
- GitCommandWrapper: rename token params to git_token in clone/push/config
- Repository/DatasetRepository: auth_token → git_token (deprecated compat kept)
- data_loader.py: update caller to use get_git_token()

SDK token references (HubApi(token=...), get_cookies(access_token=...),
commit_scheduler.token) remain unchanged as they correctly use `token` naming.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* remove(msdatasets): remove all Virgo-related implementation

Remove the entire Virgo dataset subsystem which is no longer needed:
- Remove VirgoDataset class and VirgoDownloader
- Remove VirgoAuthConfig and VirgoDatasetConfig
- Remove Hubs.virgo enum value
- Remove fetch_virgo_meta from DataMetaManager
- Remove download_virgo_files from DatasetContextConfig
- Remove test_virgo_dataset.py test file
- Clean up unused imports (pandas, MaxComputeUtil, valid_url, etc.)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(hub): add OSS dataset operations and meta-file download to HubApi

Add methods that msdatasets depends on but don't belong in modelscope_hub:
- _legacy_request: internal helper combining legacy HTTP transport with
  application-level envelope validation (Code/Data/Message)
- list_oss_dataset_objects: list OSS storage objects for a dataset
- delete_oss_dataset_object / delete_oss_dataset_dir: delete OSS objects
- fetch_meta_files_from_url: download and cache meta CSV/JSONL files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix imports issue

* fix: address PR review feedback

- cli/plugins.py: change --yes and --all flags to action='store_true'
- hub/git.py: replace os.linesep with .splitlines() for cross-platform safety
- hub/__init__.py: use is_file() with fallback for robust credentials path detection

* fix lint

* update ms hub version

* fix(ci): add PyPI official as fallback index for pip

Aliyun mirror may lag behind PyPI for newly published packages,
causing dependency resolution failures (e.g. modelscope-hub>=0.0.6).
Add pypi.org/simple as extra-index-url so new versions are immediately
available while keeping the Aliyun mirror as the primary source.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix UTs

* remove unused UTs

* fix ut

* update modelscope-hub installation for source code

* fix UT

* fix uts

* fix ut

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-09 20:00:20 +08:00
Yunnglin
b87b344f47 Upgrade numpy to 2.x for 1.38 Docker images
- Replace deprecated numpy aliases (np.math.ceil → math.ceil, np.Inf → np.inf)
- Upgrade Docker constraints: numpy>=2.0, cython>=3.0, remove scipy upper bound
2026-06-08 14:52:00 +08:00
Jintao Huang
84243dafdf update 2026-06-04 19:27:42 +08:00
Jintao Huang
2be3e5c2ea use cuda 13.0.2 2026-06-04 19:27:30 +08:00
Seas0
491e3df224 fix: quote version of timm specs in Dockerfile.ubuntu (#1728)
Co-authored-by: Seas0 <liusichen.seas0@bytedance.com>
2026-06-03 18:50:04 +08:00
Jintao Huang
3a82ba027f fix 2026-06-01 10:38:47 +08:00
Jintao Huang
f9d7afa075 update swift image 4.2.3 2026-05-31 22:20:28 +08:00
Jintao Huang
121f4150fa fix cpu image 2026-05-21 16:27:12 +08:00
Jintao Huang
6bcb1f195b fix 2026-05-18 16:09:55 +08:00
Jintao Huang
99602db273 fix 2026-05-18 16:06:45 +08:00
Jintao Huang
49de5408fa Merge branch 'master' into release/1.37 2026-05-15 14:09:10 +08:00
xvxuopop
6f347a203c update npu dockerfile (#1716) 2026-05-14 10:36:31 +08:00
Jintao Huang
263a3ef0b2 fix 2026-05-10 22:58:21 +08:00
Jintao Huang
16991606f4 update docker 2026-05-10 18:00:21 +08:00
Jintao Huang
203f0bfa27 Merge branch 'release/1.36' into build_swift_image 2026-05-10 14:08:19 +08:00
Jintao Huang
5a61fcebb2 update swift 4.2 image 2026-05-10 14:04:02 +08:00
dwd
2f5f52fc3c feat(docker/Metax): add metax dockerfile and its requirements for ms-swift 4.1.x (#1689) 2026-05-06 20:57:17 +08:00
Jintao Huang
06e6311a0c fix 2026-04-30 23:47:18 +08:00
Jintao Huang
537722acea update cuda13.0 2026-04-30 23:16:26 +08:00
addsubmuldiv
c18f11ccbd update ascend dockerfile (#1687) 2026-04-29 16:06:57 +08:00
Jintao Huang
5c160e97c4 update swift 4.1.3 2026-04-25 21:26:56 +08:00
班扬
e1635d6e94 set peft<0.19 2026-04-22 17:09:56 +08:00
Xingjun.Wang
862afeea7b [Refactor] Refactor the modelscope download module (#1683) 2026-04-21 16:00:56 +08:00
班扬
0d745602af ok Merge branch 'master' of github.com:modelscope/modelscope into release/1.36 2026-04-20 11:15:05 +08:00
Jintao Huang
07b0321e78 fix 2026-04-20 07:12:09 +08:00
Jintao Huang
ae70780564 fix 2026-04-19 15:47:45 +08:00