mirror of
https://github.com/modelscope/modelscope.git
synced 2026-07-10 12:33:28 +02:00
50f8d37bc9
* refactor(hub): shim layer delegating to modelscope-hub - Replace hub/api.py (4674→250 lines) with shim inheriting LegacyHubApi - Replace hub/snapshot_download.py, callback.py with thin shims - Partial shim hub/file_download.py (retain http_get_file) - Shim hub/constants.py and errors.py with legacy aliases - Shim hub/git.py, repository.py, cache_manager.py, upload_*.py - Migrate CLI entry to modelscope_hub.cli.main:run_cmd - Adapt 6 CLI commands as modelscope_hub.cli_plugins - Delete redundant CLI files (download/upload/login/create/etc) - Add modelscope-hub>=0.2.0 dependency, Python>=3.10 - Add __getattr__ proxy for forward-compatible method access - Propagate timeout/max_retries to internal LegacyClient - Bridge MODELSCOPE_CREDENTIALS_PATH env var to HubConfig * fix lint: isort/yapf formatting + exclude hub/api.py from hooks * set modelscope-hub>=0.0.5 * remove unused code * refactor(hub): standardize token naming — git_token vs token Disambiguate git token and SDK/API token naming across the hub layer: - ModelScopeConfig: get_token/save_token → get_git_token/save_git_token (old names kept as deprecated aliases with DeprecationWarning) - GitCommandWrapper: rename token params to git_token in clone/push/config - Repository/DatasetRepository: auth_token → git_token (deprecated compat kept) - data_loader.py: update caller to use get_git_token() SDK token references (HubApi(token=...), get_cookies(access_token=...), commit_scheduler.token) remain unchanged as they correctly use `token` naming. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * remove(msdatasets): remove all Virgo-related implementation Remove the entire Virgo dataset subsystem which is no longer needed: - Remove VirgoDataset class and VirgoDownloader - Remove VirgoAuthConfig and VirgoDatasetConfig - Remove Hubs.virgo enum value - Remove fetch_virgo_meta from DataMetaManager - Remove download_virgo_files from DatasetContextConfig - Remove test_virgo_dataset.py test file - Clean up unused imports (pandas, MaxComputeUtil, valid_url, etc.) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(hub): add OSS dataset operations and meta-file download to HubApi Add methods that msdatasets depends on but don't belong in modelscope_hub: - _legacy_request: internal helper combining legacy HTTP transport with application-level envelope validation (Code/Data/Message) - list_oss_dataset_objects: list OSS storage objects for a dataset - delete_oss_dataset_object / delete_oss_dataset_dir: delete OSS objects - fetch_meta_files_from_url: download and cache meta CSV/JSONL files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix imports issue * fix: address PR review feedback - cli/plugins.py: change --yes and --all flags to action='store_true' - hub/git.py: replace os.linesep with .splitlines() for cross-platform safety - hub/__init__.py: use is_file() with fallback for robust credentials path detection * fix lint * update ms hub version * fix(ci): add PyPI official as fallback index for pip Aliyun mirror may lag behind PyPI for newly published packages, causing dependency resolution failures (e.g. modelscope-hub>=0.0.6). Add pypi.org/simple as extra-index-url so new versions are immediately available while keeping the Aliyun mirror as the primary source. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix UTs * remove unused UTs * fix ut * update modelscope-hub installation for source code * fix UT * fix uts * fix ut --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
52 lines
2.0 KiB
Bash
Executable File
52 lines
2.0 KiB
Bash
Executable File
#!/bin/bash
|
|
set -e
|
|
set -o pipefail
|
|
# check git is install
|
|
git --version >/dev/null 2>&1 || { echo 'git not installed' ; exit 0; }
|
|
|
|
if [ -z "$MODELSCOPE_USERNAME" ] || [ -z "$MODELSCOPE_GITLAB_ACCESS_TOKEN" ]; then
|
|
:
|
|
else
|
|
git config --global credential.helper store
|
|
echo "http://${MODELSCOPE_USERNAME}:${MODELSCOPE_GITLAB_ACCESS_TOKEN}@www.modelscope.cn">~/.git-credentials
|
|
echo "https://${MODELSCOPE_USERNAME}:${MODELSCOPE_GITLAB_ACCESS_TOKEN}@www.modelscope.cn">>~/.git-credentials
|
|
chmod go-rwx ~/.git-credentials
|
|
fi
|
|
if [ -z "$MODELSCOPE_USERNAME" ] || [ -z "$MODELSCOPE_USEREMAIL" ]; then
|
|
:
|
|
else
|
|
git config --system user.name ${MODELSCOPE_USERNAME}
|
|
git config --system user.email ${MODELSCOPE_USEREMAIL}
|
|
fi
|
|
if [ -z "$MODELSCOPE_ENVIRONMENT" ]; then
|
|
:
|
|
else
|
|
git config --system --add http.http://www.modelscope.cn.extraHeader "Modelscope_Environment: $MODELSCOPE_ENVIRONMENT"
|
|
git config --system --add http.https://www.modelscope.cn.extraHeader "Modelscope_Environment: $MODELSCOPE_ENVIRONMENT"
|
|
fi
|
|
|
|
if [ -z "$MODELSCOPE_USERNAME" ]; then
|
|
:
|
|
else
|
|
git config --system --add http.http://www.modelscope.cn.extraHeader "Modelscope_User: $MODELSCOPE_USERNAME"
|
|
git config --system --add http.https://www.modelscope.cn.extraHeader "Modelscope_User: $MODELSCOPE_USERNAME"
|
|
fi
|
|
|
|
if [ -z "$MODELSCOPE_USERID" ]; then
|
|
:
|
|
else
|
|
git config --system --add http.http://www.modelscope.cn.extraHeader "Modelscope_Userid: $MODELSCOPE_USERID"
|
|
git config --system --add http.https://www.modelscope.cn.extraHeader "Modelscope_Userid: $MODELSCOPE_USERID"
|
|
fi
|
|
|
|
if [ -z "$MODELSCOPE_HAVANAID" ]; then
|
|
:
|
|
else
|
|
git config --system --add http.http://www.modelscope.cn.extraHeader "Modelscope_Havanaid: $MODELSCOPE_HAVANAID"
|
|
git config --system --add http.https://www.modelscope.cn.extraHeader "Modelscope_Havanaid: $MODELSCOPE_HAVANAID"
|
|
fi
|
|
|
|
pip config set global.index-url https://mirrors.cloud.aliyuncs.com/pypi/simple
|
|
pip config set global.extra-index-url https://pypi.org/simple
|
|
pip config set install.trusted-host mirrors.cloud.aliyuncs.com
|