Commit Graph

115 Commits

Author SHA1 Message Date
Yunnglin
52e227c48b fix(docker): force numpy>=2.0 after evalscope install
ms-opencompass pulls numpy<2.0, downgrading numpy from 2.x to 1.26.4.
Force reinstall numpy>=2.0 after pip install .[eval] to restore it.
2026-06-15 17:43:30 +08:00
addsubmuldiv
8622c814d8 Update npu dockerfile (#1736) 2026-06-15 17:43:03 +08:00
dwd
113c93b8e7 add metax dockerfile and its requirements for ms-swift 4.2.x (#1734) 2026-06-15 17:42:03 +08:00
Jintao Huang
f599019206 fix cpu image 2026-06-11 15:15:28 +08:00
Jintao Huang
c0416acc9a fix 2026-06-11 15:15:27 +08:00
Jintao Huang
29442858e9 fix 2026-06-11 15:15:26 +08:00
Jintao Huang
7053c9d6cc fix 2026-06-11 15:15:25 +08:00
Jintao Huang
faae851237 update docker 2026-06-11 15:15:23 +08:00
Yunnglin
1a53418643 Add ipywidgets dependency to Docker images 2026-06-10 14:16:53 +08:00
Xingjun.Wang
680c3638e0 [Feat & Refactor] Refactor hub and CLI modules (#1732)
* refactor(hub): shim layer delegating to modelscope-hub

- Replace hub/api.py (4674→250 lines) with shim inheriting LegacyHubApi
- Replace hub/snapshot_download.py, callback.py with thin shims
- Partial shim hub/file_download.py (retain http_get_file)
- Shim hub/constants.py and errors.py with legacy aliases
- Shim hub/git.py, repository.py, cache_manager.py, upload_*.py
- Migrate CLI entry to modelscope_hub.cli.main:run_cmd
- Adapt 6 CLI commands as modelscope_hub.cli_plugins
- Delete redundant CLI files (download/upload/login/create/etc)
- Add modelscope-hub>=0.2.0 dependency, Python>=3.10
- Add __getattr__ proxy for forward-compatible method access
- Propagate timeout/max_retries to internal LegacyClient
- Bridge MODELSCOPE_CREDENTIALS_PATH env var to HubConfig

* fix lint: isort/yapf formatting + exclude hub/api.py from hooks

* set modelscope-hub>=0.0.5

* remove unused code

* refactor(hub): standardize token naming — git_token vs token

Disambiguate git token and SDK/API token naming across the hub layer:
- ModelScopeConfig: get_token/save_token → get_git_token/save_git_token
  (old names kept as deprecated aliases with DeprecationWarning)
- GitCommandWrapper: rename token params to git_token in clone/push/config
- Repository/DatasetRepository: auth_token → git_token (deprecated compat kept)
- data_loader.py: update caller to use get_git_token()

SDK token references (HubApi(token=...), get_cookies(access_token=...),
commit_scheduler.token) remain unchanged as they correctly use `token` naming.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* remove(msdatasets): remove all Virgo-related implementation

Remove the entire Virgo dataset subsystem which is no longer needed:
- Remove VirgoDataset class and VirgoDownloader
- Remove VirgoAuthConfig and VirgoDatasetConfig
- Remove Hubs.virgo enum value
- Remove fetch_virgo_meta from DataMetaManager
- Remove download_virgo_files from DatasetContextConfig
- Remove test_virgo_dataset.py test file
- Clean up unused imports (pandas, MaxComputeUtil, valid_url, etc.)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(hub): add OSS dataset operations and meta-file download to HubApi

Add methods that msdatasets depends on but don't belong in modelscope_hub:
- _legacy_request: internal helper combining legacy HTTP transport with
  application-level envelope validation (Code/Data/Message)
- list_oss_dataset_objects: list OSS storage objects for a dataset
- delete_oss_dataset_object / delete_oss_dataset_dir: delete OSS objects
- fetch_meta_files_from_url: download and cache meta CSV/JSONL files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix imports issue

* fix: address PR review feedback

- cli/plugins.py: change --yes and --all flags to action='store_true'
- hub/git.py: replace os.linesep with .splitlines() for cross-platform safety
- hub/__init__.py: use is_file() with fallback for robust credentials path detection

* fix lint

* update ms hub version

* fix(ci): add PyPI official as fallback index for pip

Aliyun mirror may lag behind PyPI for newly published packages,
causing dependency resolution failures (e.g. modelscope-hub>=0.0.6).
Add pypi.org/simple as extra-index-url so new versions are immediately
available while keeping the Aliyun mirror as the primary source.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix UTs

* remove unused UTs

* fix ut

* update modelscope-hub installation for source code

* fix UT

* fix uts

* fix ut

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-09 22:23:14 +08:00
Seas0
204caba8b2 fix: quote version of timm specs in Dockerfile.ubuntu (#1728)
Co-authored-by: Seas0 <liusichen.seas0@bytedance.com>
2026-06-09 20:00:52 +08:00
Yunnglin
b87b344f47 Upgrade numpy to 2.x for 1.38 Docker images
- Replace deprecated numpy aliases (np.math.ceil → math.ceil, np.Inf → np.inf)
- Upgrade Docker constraints: numpy>=2.0, cython>=3.0, remove scipy upper bound
2026-06-08 14:52:00 +08:00
xvxuopop
6f347a203c update npu dockerfile (#1716) 2026-05-14 10:36:31 +08:00
dwd
2f5f52fc3c feat(docker/Metax): add metax dockerfile and its requirements for ms-swift 4.1.x (#1689) 2026-05-06 20:57:17 +08:00
addsubmuldiv
c18f11ccbd update ascend dockerfile (#1687) 2026-04-29 16:06:57 +08:00
Xingjun.Wang
862afeea7b [Refactor] Refactor the modelscope download module (#1683) 2026-04-21 16:00:56 +08:00
addsubmuldiv
55af6ce8e4 update ascend dockerfile (#1679) 2026-04-17 17:04:26 +08:00
Xingjun.Wang
9cc43a3140 Fix GPG KEY (#1678)
* refactor hf datasets utils and fix issue

* update datasets version

* fix download cli

* add engine=python for stream loading csv

* fix dataset builder for csv engine=python

* fix datasets csv OverflowError

* update UT commit scheduler test level

* fix python gpg key
2026-04-14 15:57:33 +08:00
tastelikefeet
ce6a902d76 merge 1.35.1 to master (#1653) 2026-03-19 15:01:51 +08:00
Xingjun.Wang
832d88cb02 [Fix] lazy import oss2 (#1649)
* lazy import oss2

* fix lint
2026-03-17 16:03:42 +08:00
tastelikefeet
a420e68953 Merge branch 'release/1.34'
# Conflicts:
#	docker/build_image.py
2026-03-13 11:36:25 +08:00
vx120
467a2206e3 add metax dockerfile and its requirements for ms-swift (#1643) 2026-03-10 16:11:49 +08:00
tastelikefeet
2fedc9bb9e Feat/update docker file (#1640) 2026-03-09 14:21:38 +08:00
hjh0119
ae1e83febb remove ollama 2026-01-15 20:53:05 +08:00
hjh0119
c137ed08c2 docker zstd 2026-01-15 20:33:27 +08:00
hjh0119
de3399e8de pin optimum 2026-01-14 10:35:49 +08:00
hjh0119
e6ef4af1a1 pin optimum 2.0.0 for gpu/llm image 2026-01-13 18:18:15 +08:00
suluyana
09bc99c8c3 fix: Remove nvidia.cn from APT sources to fix apt failures in certain environments (#1575)
Co-authored-by: suluyan <suluyan.sly@aliabab-inc.com>
2026-01-09 16:09:23 +08:00
Xingjun.Wang
be6874ea8e Feat: add set_repo_visibility in hub api (#1564)
* add set_repo_visibility

* fix cr

* fix lint
2025-12-05 15:37:45 +08:00
Yunlin Mao
9cb7ce7029 Update build file and workflow (#1551)
* update build file

* update setup file

* update fairseq dep

* fix error log
2025-12-02 10:50:39 +08:00
baymax591
6f3200dfc2 [docker] Add Ascend NPU dockerfile (#1550)
Co-authored-by: ji-huazhong <hzji210@gmail.com>
Co-authored-by: vx120 <893600387@qq.com>
2025-11-26 09:56:47 +08:00
Jintao
2f3ac0fcd2 [docker] update swift docker (#1502) 2025-09-16 12:49:29 +08:00
tastelikefeet
6b70c8b577 Merge 1.29 to master (#1469) 2025-08-18 13:51:00 +08:00
tastelikefeet
169f523082 fix tensorflow cannot load gpu (#1456) 2025-08-11 11:04:17 +08:00
tastelikefeet
3bad15b779 add ms-agent into dockerfile (#1436) 2025-08-04 17:08:48 +08:00
jinghanhu
b537efebed Merge release 1.28 (#1401) 2025-08-04 17:00:14 +08:00
suluyan
7230561cbd image deps upgrade: transformers, huggingface_hub, node.js 2025-07-03 15:59:20 +08:00
suluyan
6a895360da fix boost.tar.gz not in gzip format 2025-07-03 15:59:12 +08:00
hjh0119
dee7d0576d fix pgp fail 2025-07-03 15:58:10 +08:00
Yunlin Mao
093ae87111 Update 1.26.0 docker image dependency (#1339) 2025-05-28 17:36:07 +08:00
Jintao
03dbc7bd11 Merge release1.25 to master (#1319) 2025-04-23 13:40:54 +08:00
Jintao
32aefd7428 Merge release1.24 to master (#1298) 2025-04-07 13:32:09 +08:00
Jintao
3bfff5cb53 Add swift docker (#1299) 2025-04-07 13:24:18 +08:00
tastelikefeet
e733458746 Update omegaconf (#1234) 2025-02-19 18:11:23 +08:00
tastelikefeet
7246d0b359 Merge 1.23 hotfix to master (#1227) 2025-02-17 10:30:24 +08:00
tastelikefeet
b5bb6d7bb0 Use legacy cache (#1215) 2025-02-07 17:31:32 +08:00
Xingjun.Wang
77d7262c22 fix lint (#1188) 2025-01-15 14:25:09 +08:00
Xingjun.Wang
648187626a Merge release 1.22 (#1187)
* bump version 1.22.0

* fix confict between lmdeploy & vllm

* flash-attn version

* fix version building image

* fix https://www.modelscope.cn/models/iic/nlp_structbert_address-parsing_chinese_base/feedback/issueDetail/20431 (#1170)

* fix path contatenation to be windows compatabile (#1176)

* fix path contatenation to be windows compatabile

* support dataset too

---------

Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>

* logger.warning when using remote code (#1171)

* logger warning when using remote code

Co-authored-by: suluyan <suluyan.sly@alibaba-inc.com>

* feat: all other ollama models (#1174)

* add cases

* new models

---------

Co-authored-by: suluyan <suluyan.sly@alibaba-inc.com>

* Unify datasets cache dir (#1178)

* fix cache

* fix lint

* fix dataset cache

* fix lint

* remove

* Add repo_id and repo_type in snapshot_download (#1172)

* add repo_id and repo_type in snapshot_download

* fix positional args

* update

* Fix/text gen (#1177)

* fix text-gen: read pipeline type from configuration.json first

---------

Co-authored-by: suluyan <suluyan.sly@alibaba-inc.com>

* update doc with llama_index (#1180)

* update version to 1.22.1

* merge release/1.22

---------

Co-authored-by: suluyana <suluyan_sly@163.com>
Co-authored-by: tastelikefeet <58414341+tastelikefeet@users.noreply.github.com>
Co-authored-by: Yingda Chen <yingdachen@apache.org>
Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
Co-authored-by: suluyana <110878454+suluyana@users.noreply.github.com>
Co-authored-by: suluyan <suluyan.sly@alibaba-inc.com>
Co-authored-by: Yunlin Mao <mao.looper@qq.com>
2025-01-15 14:08:52 +08:00
tastelikefeet
46c4ce9bdd support tag ci_image (#1112) 2024-12-01 15:17:25 +08:00
Jintao
b1e1f66fad Merge release 1.20 docker (#1109) 2024-11-29 13:44:36 +08:00