Files
modelscope/docs
zhangzhicheng.zzc df42427ae0 [to #42322933] nlp preprocessor refactor
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9269314

    * init

* token to ids

* add model

* model forward ready

* add intent

* intent preprocessor ready

* intent success

* merge master

* test with model hub

* add flake8

* update

* update

* update

* Merge branch 'master' into nlp/space/gen

* delete file about gen

* init

* fix flake8 bug

* [to #42322933] init

* bug fix

* [to #42322933] init

* update pipeline registry info

* Merge remote-tracking branch 'origin/master' into feat/nli

* [to #42322933] init

* [to #42322933] init

* modify forward

* [to #42322933] init

* generation ready

* init

* Merge branch 'master' into feat/zero_shot_classification

# Conflicts:
#	modelscope/preprocessors/__init__.py

* [to #42322933] bugfix

* [to #42322933] pre commit fix

* fill mask

* registry multi models on model and pipeline

* add tests

* test level >= 0

* local gen ready

* merge with master

* dialog modeling ready

* fix comments: rename and refactor AliceMindMLM; adjust pipeline

* space intent and modeling(generation) are ready

* bug fix

* add dep

* add dep

* support dst data processor

* merge with nlp/space/dst

* merge with master

* Merge remote-tracking branch 'origin' into feat/fill_mask

Conflicts:
	modelscope/models/nlp/__init__.py
	modelscope/pipelines/builder.py
	modelscope/pipelines/outputs.py
	modelscope/preprocessors/nlp.py
	requirements/nlp.txt

* merge with master

* merge with master 2/2

* fix comments

* fix isort for pre-commit check

* allow params pass to pipeline's __call__ method

* Merge remote-tracking branch 'origin/master' into feat/zero_shot_classification

* merge with nli task

* merge with sentiment_classification

* merge with zero_shot_classfication

* merge with fill_mask

* merge with space

* merge with master head

* Merge remote-tracking branch 'origin' into feat/fill_mask

Conflicts:
	modelscope/utils/constant.py

* fix: pipeline module_name from model_type to 'fill_mask' & fix merge bug

* unfiinished change

* fix bug

* unfinished

* unfinished

* revise modelhub dependency

* Merge branch 'feat/nlp_refactor' of http://gitlab.alibaba-inc.com/Ali-MaaS/MaaS-lib into feat/nlp_refactor

* add eval() to pipeline call

* add test level

* ut run passed

* add default args

* tmp

* merge master

* all ut passed

* remove an useless enum

* revert a mis modification

* revert a mis modification

* Merge commit 'ace8af92465f7d772f035aebe98967726655f12c' into feat/nlp

* commit 'ace8af92465f7d772f035aebe98967726655f12c':
  [to #42322933] Add cv-action-recongnition-pipeline to maas lib
  [to #42463204]  support Pil.Image for image_captioning_pipeline
  [to #42670107] restore pydataset test
  [to #42322933] add create if not exist and add(back) create model example         Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9130661
  [to #41474818]fix: fix errors in task name definition

# Conflicts:
#	modelscope/pipelines/builder.py
#	modelscope/utils/constant.py

* Merge branch 'feat/nlp' into feat/nlp_refactor

* feat/nlp:
  [to #42322933] Add cv-action-recongnition-pipeline to maas lib
  [to #42463204]  support Pil.Image for image_captioning_pipeline
  [to #42670107] restore pydataset test
  [to #42322933] add create if not exist and add(back) create model example         Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9130661
  [to #41474818]fix: fix errors in task name definition

# Conflicts:
#	modelscope/pipelines/builder.py

* fix compile bug

* refactor space

* Merge branch 'feat/nlp_refactor' of http://gitlab.alibaba-inc.com/Ali-MaaS/MaaS-lib into feat/nlp_refactor

* Merge remote-tracking branch 'origin' into feat/fill_mask

* fix

* pre-commit lint

* lint file

* lint file

* lint file

* update modelhub dependency

* lint file

* ignore dst_processor temporary

* solve comment: 1. change MaskedLMModelBase to MaskedLanguageModelBase 2. remove a useless import

* recommit

* remove MaskedLanguageModel from __all__

* Merge commit '1a0d4af55a2eee69d89633874890f50eda8f8700' into feat/nlp_refactor

* commit '1a0d4af55a2eee69d89633874890f50eda8f8700':
  [to #42322933] test level check         Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9143809
  [to #42322933] update nlp models name in metainfo         Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9134657

# Conflicts:
#	modelscope/metainfo.py

* update

* revert pipeline params update

* remove zeroshot

* update sequence classfication outpus

* merge with fill mask

* Merge remote-tracking branch 'origin' into feat/fill_mask

* fix

* fix flake8 warning of dst

* Merge remote-tracking branch 'origin/feat/fill_mask' into feat/nlp

* merge with master

* remove useless test.py

* Merge remote-tracking branch 'origin/master' into feat/nlp

* remove unformatted space trainer

* revise based on comment except chinease comment

* skip ci blocking

* translation pipeline

* csanmt model for translation pipeline

* update

* update

* update builder.py

* change Chinese notes of space3.0 into English

* translate chinese comment to english

* add space to metainfo

* update casnmt_translation

* update csanmt transformer

* merge with master

* update csanmt translation

* update lint

* update metainfo.py

* Update translation_pipeline.py

* Update builder.py

* fix: 1. make csanmt derived from Model 2. add kwargs to prevent from call error

* pre-commit check

* temp exclue flake8

* temp ignore translation files

* fix bug

* pre-commit passed

* fixbug

* fixbug

* revert pre commit ignorance

* pre-commit passed

* fix bug

* merge with master

* add missing setting

* merge with master

* add outputs

* modify test level

* modify chinese comment

* remove useless doc

* space outputs normalization

* Merge remote-tracking branch 'origin/master' into nlp/translation

* update translation_pipeline.py

* Merge remote-tracking branch 'origin/master' into feat/nlp

* Merge remote-tracking branch 'origin/master' into nlp/translation

* add new __init__ method

* add new __init__ method

* update output format

* Merge remote-tracking branch 'origin/master' into feat/nlp

* update output format

* merge with master

* merge with nlp/translate

* update the translation comment

* update the translation comment

* Merge branch 'nlp/translation' into feat/nlp

* Merge remote-tracking branch 'origin/master' into feat/nlp

* Merge remote-tracking branch 'origin/master' into feat/nlp

* nlp preprocessor refactor

* add get_model_type in util.hub

* update the default preprocessor args

* update the fill mask preprocessor

* bug typo fixed
2022-07-06 14:11:34 +08:00
..

maintain docs

  1. build docs

    # in root directory:
    make docs
    
  2. doc string format

    We adopt the google style docstring format as the standard, please refer to the following documents.

    1. Google Python style guide docstring link
    2. Google docstring example link
    3. sampletorch.nn.modules.conv link
    4. load function as an example
    def load(file, file_format=None, **kwargs):
        """Load data from json/yaml/pickle files.
    
        This method provides a unified api for loading data from serialized files.
    
        Args:
            file (str or :obj:`Path` or file-like object): Filename or a file-like
                object.
            file_format (str, optional): If not specified, the file format will be
                inferred from the file extension, otherwise use the specified one.
                Currently supported formats include "json", "yaml/yml".
    
        Examples:
            >>> load('/path/of/your/file')  # file is storaged in disk
            >>> load('https://path/of/your/file')  # file is storaged in Internet
            >>> load('oss://path/of/your/file')  # file is storaged in petrel
    
        Returns:
            The content from the file.
        """