Commit Graph

262 Commits

Author SHA1 Message Date
ly119399
e683a2d4cd merge master 2022-06-12 14:55:32 +08:00
wenmeng.zwm
1f6b376599 [to #42373878] refactor maaslib to modelscope
1.  refactor maaslib to modelscope
2.  fix UT error
3.  support pipeline which does not register default model

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8988388
2022-06-09 20:16:26 +08:00
myf272609
0f5b214ce0 [to #42322933] Add cv-person-image-cartoon-pipeline to maas lib
mass lib 接入人像卡通化算法
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8925552

    * update

* fix style issues

* fix style issues

* fix style issues

* fix style issues

* add requirements

* fix bug

* adapt class init

* fix

* fix tf2 issue

* feat: Fix confilct, auto commit by WebIDE

* fix commit issues

* fix
2022-06-09 15:18:38 +08:00
menrui.mr
68e64a90d4 [to #42322933] add image_caption_pipeline with OFA
1. add OFA whl for image caption pipeline
2. fix a bug in pipelines/builder.py
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8930942

    * [to #41669377] docs and tools refinement and release 

1. add build_doc linter script
2. add sphinx-docs support
3. add development doc and api doc
4. change version to 0.1.0 for the first internal release version

Link: https://code.aone.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8775307

* [to #41669377] add pipeline tutorial and fix bugs 

1. add pipleine tutorial
2. fix bugs when using pipeline with certain model and preprocessor

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8814301

* refine doc

* refine doc

* upload ofa for caption(with source code but not whl)

* remove data in gitignore

* append uncommitted data dir in ofa

* remove ofa_dir , use ofa.whl instead.

* update BPE

* rollback changes used in debugging.

* Merge branch 'master' into ofa/image_caption

# Conflicts:
#	docs/README.md
#	docs/source/conf.py
#	docs/source/index.rst
#	docs/source/tutorials/pipeline.md
#	maas_lib/models/nlp/sequence_classification_model.py
#	maas_lib/pipelines/builder.py
#	maas_lib/version.py
#	setup.py
#	tests/pipelines/test_text_classification.py

* 1. fix a bug in pipelines/builder.py.
2. modify model_path to model in image_captioning.py.

* 1. rename test_image_captioning.py.

* format all files using pre-commit.

* add fairseq in requirements.txt

* add fairseq in requirements.txt

* change fairseq path to git repo to a whl on oss in ofa.txt.

* change module_name to 'ofa'

* Merge remote-tracking branch 'origin/master' into ofa/image_caption

# Conflicts:
#	maas_lib/pipelines/builder.py

* optim requirements for ofa / refine image_captioning.py

* uncommited change.

* feat: Fix confilct, auto commit by WebIDE
2022-06-08 18:38:19 +08:00
feiwu.yfw
235880f300 [to #42339763] merge pydataset into maas-lib
* merge pydataset to the repo
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8955999
2022-06-08 18:29:39 +08:00
hemu.zp
8a76f40754 [to #42322933]Add text-generation-pipeline with Palm model.
将 Palm 中文模型接入 MaaS,添加了文本生成 pipeline
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8934393

    * add text_generation model and pipeline

* fix bug

* fix bug

* add TextGenerator in pipeline

* fix bug

* update checkpoint and test inputs

* remove magic number..

* fix bug

* adjust code with AutoModel

* clear comments and tidy up the code

* move model.eval() into generator

* update master interface and lint code

* replace 'palm-text-generation' with 'palm'

* add text_generation model and pipeline

* fix bug

* fix bug

* add TextGenerator in pipeline

* fix bug

* fix conflict of pipeline.txt

* remove magic number..

* fix bug

* adjust code with AutoModel

* clear comments and tidy up the code

* move model.eval() into generator

* fix conflict

* replace 'palm-text-generation' with 'palm'

* fix conflict

* add test_run_modelhub

* update sofa version

* modify sofa version

* add test_run_with_model_name

* fix bug
2022-06-08 17:11:04 +08:00
ly119399
4a244abbf1 token to ids 2022-06-08 00:38:50 +08:00
yingda.chen
5995cc4607 add PyDataset support
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8868644
2022-05-31 18:27:19 +08:00
wenmeng.zwm
c4bfd6cced [to #41999503] refine doc and requirements for linux and mac
1. refine quick start and pipeline doc
2. remove tf pytorch easynlp from requirements
3. lazy import for torch and tensorflow
4. test successfully on linux and mac intel cpu
5. update api doc

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8882373
2022-05-31 11:49:46 +08:00
wenmeng.zwm
25a2028b54 [to #41401401] modelhub and Trainer support
* add trainer interface
 * add trainer script
 * add model init support for pipelineadd pipeline tutorial and fix bugs 
 * add text classification evaluation to maas lib 
 * add quickstart and prepare env doc
 * relax requirements for torch and sentencepiece
 * merge release/0.1 and fix conflict
 * modelhub support for model and pipeline

 Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8868339
2022-05-30 11:53:53 +08:00
wenmeng.zwm
5e469008fd [to #41401401] add preprocessor, model and pipeline
* add preprocessor module
 * add model base and builder
 * update task constant
 * add load image preprocessor and its dependency
 * add pipeline interface and UT covered
 * support default pipeline for task
 * add image matting pipeline
 * refine nlp tokenize interface
 * add nlp pipeline 
 * fix UT failed
 * add test for Compose

Link: https://code.aone.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8769235

* add preprocessor module

* add test for Compose

* fix citest error

* fix abs class error

* add model base and builder

* update task constant

* add load image preprocessor and its dependency

* add pipeline interface and UT covered

* support default pipeline for task

* refine models and pipeline interface

* add pipeline folder structure

* add image matting pipeline

* refine nlp tokenize interface

* add nlp pipeline 

1.add preprossor model pipeline for nlp text classification
2. add corresponding test

Link: https://code.aone.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8757371

* new nlp pipeline

* format pre-commit code

* update easynlp pipeline

* update model_name for easynlp pipeline; add test for maas_lib/utils/typeassert.py

* update test_typeassert.py

* refactor code

1. rename typeassert to type_assert
2. use lazy import to make easynlp dependency optional
3. refine image matting UT

* fix linter test failed

* update requirements.txt

* fix UT failed

* fix citest script to update requirements
2022-05-19 22:18:35 +08:00
wenmeng.zwm
0a756f6a0d [to #41402703] add basic modules
* add constant
 * add logger module
 * add registry and builder module
 * add fileio module
 * add requirements and setup.cfg
 * add config module and tests
 * add citest script

Link: https://code.aone.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8718998
2022-05-17 10:15:00 +08:00