* refactor(hub): shim layer delegating to modelscope-hub
- Replace hub/api.py (4674→250 lines) with shim inheriting LegacyHubApi
- Replace hub/snapshot_download.py, callback.py with thin shims
- Partial shim hub/file_download.py (retain http_get_file)
- Shim hub/constants.py and errors.py with legacy aliases
- Shim hub/git.py, repository.py, cache_manager.py, upload_*.py
- Migrate CLI entry to modelscope_hub.cli.main:run_cmd
- Adapt 6 CLI commands as modelscope_hub.cli_plugins
- Delete redundant CLI files (download/upload/login/create/etc)
- Add modelscope-hub>=0.2.0 dependency, Python>=3.10
- Add __getattr__ proxy for forward-compatible method access
- Propagate timeout/max_retries to internal LegacyClient
- Bridge MODELSCOPE_CREDENTIALS_PATH env var to HubConfig
* fix lint: isort/yapf formatting + exclude hub/api.py from hooks
* set modelscope-hub>=0.0.5
* remove unused code
* refactor(hub): standardize token naming — git_token vs token
Disambiguate git token and SDK/API token naming across the hub layer:
- ModelScopeConfig: get_token/save_token → get_git_token/save_git_token
(old names kept as deprecated aliases with DeprecationWarning)
- GitCommandWrapper: rename token params to git_token in clone/push/config
- Repository/DatasetRepository: auth_token → git_token (deprecated compat kept)
- data_loader.py: update caller to use get_git_token()
SDK token references (HubApi(token=...), get_cookies(access_token=...),
commit_scheduler.token) remain unchanged as they correctly use `token` naming.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* remove(msdatasets): remove all Virgo-related implementation
Remove the entire Virgo dataset subsystem which is no longer needed:
- Remove VirgoDataset class and VirgoDownloader
- Remove VirgoAuthConfig and VirgoDatasetConfig
- Remove Hubs.virgo enum value
- Remove fetch_virgo_meta from DataMetaManager
- Remove download_virgo_files from DatasetContextConfig
- Remove test_virgo_dataset.py test file
- Clean up unused imports (pandas, MaxComputeUtil, valid_url, etc.)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(hub): add OSS dataset operations and meta-file download to HubApi
Add methods that msdatasets depends on but don't belong in modelscope_hub:
- _legacy_request: internal helper combining legacy HTTP transport with
application-level envelope validation (Code/Data/Message)
- list_oss_dataset_objects: list OSS storage objects for a dataset
- delete_oss_dataset_object / delete_oss_dataset_dir: delete OSS objects
- fetch_meta_files_from_url: download and cache meta CSV/JSONL files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix imports issue
* fix: address PR review feedback
- cli/plugins.py: change --yes and --all flags to action='store_true'
- hub/git.py: replace os.linesep with .splitlines() for cross-platform safety
- hub/__init__.py: use is_file() with fallback for robust credentials path detection
* fix lint
* update ms hub version
* fix(ci): add PyPI official as fallback index for pip
Aliyun mirror may lag behind PyPI for newly published packages,
causing dependency resolution failures (e.g. modelscope-hub>=0.0.6).
Add pypi.org/simple as extra-index-url so new versions are immediately
available while keeping the Aliyun mirror as the primary source.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix UTs
* remove unused UTs
* fix ut
* update modelscope-hub installation for source code
* fix UT
* fix uts
* fix ut
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1. Merge(add) daily regression from github PR (daily_regression.yaml)
2. Add lora stable diffusion from github PR
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13010802
* fix: device arg not work, rename device to ngpu (#272)
* Correcting the lora stable diffusion example script (#300)
* add vad model and punc model in README.md
add vad model and punc model
* Merge pull request #302 from modelscope/langgz-patch-1
add vad model and punc model in README.md
* add 1.6
* modify ignore
* Merge pull request #307 from modelscope/dev_rs_16
Merge release 1.6
* undo datetime to 2099
* Merge pull request #311 from modelscope/fix_master_version
undo datetime to 2099
* add daily regression workflow
* modify workflow name
* fix cron format issue
* lora trainer
* Merge pull request #315 from liuyhwangyh/add_regression_workflow
add daily regression workflow
1. Refactor training_args
2. Refactor hooks
3. Add train_id for push_to_hub
4. Support both output_dir/output_sub_dir for checkpoint_hooks
5. Support copy when hardlink fails when checkpointing
6. Support mixed dataset config file as a CLI argument
7. Add eval txt in output folder
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12384253
* support the ignorance of file pattern
1. backward compatible with to_task_dataset function for DefaultTrainer in adaseq repo
2. fix registry issue for RedsImageDeblurringDataset and GoproImageDeblurringDataset
3. add ut TestCustomDatasetsCompatibility
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11981956
Refactor the task_datasets module:
1. Add new module modelscope.msdatasets.dataset_cls.custom_datasets.
2. Add new function: modelscope.msdatasets.ms_dataset.MsDataset.to_custom_dataset().
2. Add calling to_custom_dataset() func in MsDataset.load() to adapt new custom_datasets module.
3. Refactor the pipeline for loading custom dataset:
1) Only use MsDataset.load() function to load the custom datasets.
2) Combine MsDataset.load() with class EpochBasedTrainer.
4. Add new entry func for building datasets in EpochBasedTrainer: see modelscope.trainers.trainer.EpochBasedTrainer.build_dataset()
5. Add new func to build the custom dataset from model configuration, see: modelscope.trainers.trainer.EpochBasedTrainer.build_dataset_from_cfg()
6. Add new registry function for building custom datasets, see: modelscope.msdatasets.dataset_cls.custom_datasets.builder.build_custom_dataset()
7. Refine the class SiameseUIETrainer to adapt the new custom_datasets module.
8. Add class TorchCustomDataset as a superclass for custom datasets classes.
9. To move modules/classes/functions:
1) Move module msdatasets.audio to custom_datasets
2) Move module msdatasets.cv to custom_datasets
3) Move module bad_image_detecting to custom_datasets
4) Move module damoyolo to custom_datasets
5) Move module face_2d_keypoints to custom_datasets
6) Move module hand_2d_keypoints to custom_datasets
7) Move module human_wholebody_keypoint to custom_datasets
8) Move module image_classification to custom_datasets
9) Move module image_inpainting to custom_datasets
10) Move module image_portrait_enhancement to custom_datasets
11) Move module image_quality_assessment_degradation to custom_datasets
12) Move module image_quality_assmessment_mos to custom_datasets
13) Move class LanguageGuidedVideoSummarizationDataset to custom_datasets
14) Move class MGeoRankingDataset to custom_datasets
15) Move module movie_scene_segmentation custom_datasets
16) Move module object_detection to custom_datasets
17) Move module referring_video_object_segmentation to custom_datasets
18) Move module sidd_image_denoising to custom_datasets
19) Move module video_frame_interpolation to custom_datasets
20) Move module video_stabilization to custom_datasets
21) Move module video_super_resolution to custom_datasets
22) Move class GoproImageDeblurringDataset to custom_datasets
23) Move class EasyCVBaseDataset to custom_datasets
24) Move class ImageInstanceSegmentationCocoDataset to custom_datasets
25) Move class RedsImageDeblurringDataset to custom_datasets
26) Move class TextRankingDataset to custom_datasets
27) Move class VecoDataset to custom_datasets
28) Move class VideoSummarizationDataset to custom_datasets
10. To delete modules/functions/classes:
1) Del module task_datasets
2) Del to_task_dataset() in EpochBasedTrainer
3) Del build_dataset() in EpochBasedTrainer and renew a same name function.
11. Rename class Datasets to CustomDatasets in metainfo.py
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11872747
