Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2025-12-19 09:39:23 +01:00
Code Issues Packages Projects Releases Wiki Activity
Files
release/1.5
modelscope/docs/source/api/modelscope.msdatasets.dataset_cls.custom_datasets.rst

42 lines
1.1 KiB
ReStructuredText
Raw Permalink Normal View History

Refactor the task_datasets module Refactor the task_datasets module: 1. Add new module modelscope.msdatasets.dataset_cls.custom_datasets. 2. Add new function: modelscope.msdatasets.ms_dataset.MsDataset.to_custom_dataset(). 2. Add calling to_custom_dataset() func in MsDataset.load() to adapt new custom_datasets module. 3. Refactor the pipeline for loading custom dataset: 1) Only use MsDataset.load() function to load the custom datasets. 2) Combine MsDataset.load() with class EpochBasedTrainer. 4. Add new entry func for building datasets in EpochBasedTrainer: see modelscope.trainers.trainer.EpochBasedTrainer.build_dataset() 5. Add new func to build the custom dataset from model configuration, see: modelscope.trainers.trainer.EpochBasedTrainer.build_dataset_from_cfg() 6. Add new registry function for building custom datasets, see: modelscope.msdatasets.dataset_cls.custom_datasets.builder.build_custom_dataset() 7. Refine the class SiameseUIETrainer to adapt the new custom_datasets module. 8. Add class TorchCustomDataset as a superclass for custom datasets classes. 9. To move modules/classes/functions: 1) Move module msdatasets.audio to custom_datasets 2) Move module msdatasets.cv to custom_datasets 3) Move module bad_image_detecting to custom_datasets 4) Move module damoyolo to custom_datasets 5) Move module face_2d_keypoints to custom_datasets 6) Move module hand_2d_keypoints to custom_datasets 7) Move module human_wholebody_keypoint to custom_datasets 8) Move module image_classification to custom_datasets 9) Move module image_inpainting to custom_datasets 10) Move module image_portrait_enhancement to custom_datasets 11) Move module image_quality_assessment_degradation to custom_datasets 12) Move module image_quality_assmessment_mos to custom_datasets 13) Move class LanguageGuidedVideoSummarizationDataset to custom_datasets 14) Move class MGeoRankingDataset to custom_datasets 15) Move module movie_scene_segmentation custom_datasets 16) Move module object_detection to custom_datasets 17) Move module referring_video_object_segmentation to custom_datasets 18) Move module sidd_image_denoising to custom_datasets 19) Move module video_frame_interpolation to custom_datasets 20) Move module video_stabilization to custom_datasets 21) Move module video_super_resolution to custom_datasets 22) Move class GoproImageDeblurringDataset to custom_datasets 23) Move class EasyCVBaseDataset to custom_datasets 24) Move class ImageInstanceSegmentationCocoDataset to custom_datasets 25) Move class RedsImageDeblurringDataset to custom_datasets 26) Move class TextRankingDataset to custom_datasets 27) Move class VecoDataset to custom_datasets 28) Move class VideoSummarizationDataset to custom_datasets 10. To delete modules/functions/classes: 1) Del module task_datasets 2) Del to_task_dataset() in EpochBasedTrainer 3) Del build_dataset() in EpochBasedTrainer and renew a same name function. 11. Rename class Datasets to CustomDatasets in metainfo.py Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11872747
2023-03-10 09:03:32 +08:00
modelscope.msdatasets.dataset_cls.custom_datasets
====================
.. automodule:: modelscope.msdatasets.dataset_cls.custom_datasets
.. currentmodule:: modelscope.msdatasets.dataset_cls.custom_datasets
.. autosummary::
:toctree: generated
:nosignatures:
:template: classtemplate.rst
EasyCVBaseDataset
TorchCustomDataset
MovieSceneSegmentationDataset
ImageInstanceSegmentationCocoDataset
GoproImageDeblurringDataset
LanguageGuidedVideoSummarizationDataset
MGeoRankingDataset
RedsImageDeblurringDataset
TextRankingDataset
VecoDataset
VideoSummarizationDataset
BadImageDetectingDataset
ImageInpaintingDataset
ImagePortraitEnhancementDataset
ImageQualityAssessmentDegradationDataset
ImageQualityAssessmentMosDataset
ReferringVideoObjectSegmentationDataset
SiddImageDenoisingDataset
VideoFrameInterpolationDataset
VideoStabilizationDataset
VideoSuperResolutionDataset
SegDataset
FaceKeypointDataset
HandCocoWholeBodyDataset
WholeBodyCocoTopDownDataset
ClsDataset
DetImagesMixDataset
DetDataset
Reference in New Issue Copy Permalink