Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2025-12-25 20:49:37 +01:00
Code Issues Packages Projects Releases Wiki Activity
1,326 Commits 263 Branches 106 Tags
9faf588bc68e7d79786efbfbb89fe29b964e8bd2
Commit Graph

27 Commits

Author SHA1 Message Date
mulin.lyh
e54694690f [to #46993990]feat: run ci cases base on code diff to reduct ci test time 2023-02-06 08:00:19 +00:00
mulin.lyh
7039e93c99 skip temp failed case 2022-12-01 16:50:09 +08:00
mulin.lyh
70deb0190b [to #46289830]feat: hub sdk support retry and continue-download after error 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10814720
 2022-11-24 15:01:24 +08:00
mulin.lyh
3791ee7ad2 [to #45821936]fix: fix block user specify revision after release_datetime 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10572162
 2022-10-29 13:44:47 +08:00
mulin.lyh
3b75623be4 [to #45773874]fix: get_model revision=None bug, and hub case occasionally delete test model failed 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10549680
 2022-10-27 17:06:18 +08:00
jiaqi.sjq
7b84adc914 [to #42322933]Fix remove files in local model not take effect to remote repo after push_model 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10533214
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10533214
 2022-10-26 19:15:43 +08:00
mulin.lyh
384377b8f5 * [to #45486649]feat: modelscope model version use model repo tag, unsupport branch or commit it, client user-agent header unified 2022-10-26 13:55:51 +08:00
yuze.zyz
605cd7f44a [to #42322933] NLP 1030 Refactor 
Features:
1. Refactor the directory structure of nlp models. All model files are placed into either the model folder or the task_model folder
2. Refactor all the comments to google style
3. Add detail comments to important tasks and nlp models, to list the description of the model, and its preprocessor&trainer
4. Model Exporting now supports a direct all to TorchModelExporter(no need to derive from it)
5. Refactor model save_pretrained method to support direct running(independent from trainer)
6. Remove the judgement of Model in the pipeline base class, to support outer register models running in our pipelines
7. Nlp trainer now has a NLPTrainingArguments class , user can pass arguments into the dataclass, and use it as a normal cfg_modify_fn, to simplify the operation of modify cfg.
8. Merge the BACKBONES and the MODELS, so user can get a backbone with the Model.from_pretrained call
9. Model.from_pretrained now support a task argument, so user can use a backbone and load it with a specific task class.
10. Support Preprocessor.from_pretrained method
11. Add standard return classes to important nlp tasks, so some of the pipelines and the models are independent now, the return values of the models will always be tensors, and the pipelines will take care of the conversion to numpy and the following stuffs.
12. Split the file of the nlp preprocessors, to make the dir structure more clear.

Bugs Fixing:
1. Fix a bug that lr_scheduler can be called earlier than the optimizer's step
2. Fix a bug that the direct call of Pipelines (not from pipeline(xxx)) throws error
3. Fix a bug that the trainer will not call the correct TaskDataset class
4. Fix a bug that the internal loading of dataset will throws error in the trainer class
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10490585
 2022-10-25 12:26:25 +08:00
yingda.chen
3cd5f73da0 [to #42322933] refactor push_model 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10463992
 2022-10-20 10:28:15 +08:00
jiaqi.sjq
87b4a52b3c Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10462681 2022-10-19 23:29:58 +08:00
jiaqi.sjq
2eb835aca4 [to #42322933]Add uuid to model which created by ut test 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10434107
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10434107

    * [Update] update finetune_result_upload

* [Update] rename finetune_result_upload to model_dir_upload

* Merge branch 'master' into feat/upload_ckpt

* Merge branch 'master' into feat/upload_ckpt

* [Fix] fix import error

* [Fix] fix import error

* Merge branch 'master' into feat/upload_ckpt

* [Update] changes name to upload_folder and using tempfile to save repo

* Merge branch 'master' into feat/upload_ckpt

* [Fix] fix commit

* Merge branch 'master' into feat/upload_ckpt

* [Fix] fix format

* Merge branch 'master' into feat/upload_ckpt

* [Fix] add uuid after model created from upload ut
 2022-10-18 11:12:12 +08:00
jiaqi.sjq
8fa385e27c [to #42322933] Add upload in hub api 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10386689
 2022-10-17 15:42:24 +08:00
wenmeng.zwm
6808e9a301 [to #44902099] add license for framework files 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10189613
 2022-09-20 17:49:31 +08:00
mulin.lyh
ff58300d09 [to #44857956]fix: disable git command username/password prompt 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10106602

    * [to #44857956]fix: disable git command username/password prompt
 2022-09-14 06:44:04 +08:00
mulin.lyh
c57c91c359 [to #44031139]fix: fix hub ci case delete test project failure 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9736416

    * [to #44031139]fix: fix hub ci case delete test project failure
 2022-08-12 13:56:47 +08:00
mulin.lyh
3272b2755c [to #43909247]fix: access token in code, move to environment 
move access token to environment variable.
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9681287

    * [to #43909247]fix: access token in code, move to environment
 2022-08-09 17:46:56 +08:00
mulin.lyh
578f82e501 [to #43887377]fix: sdk api concurrent call snapshort download file will conflict 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9672696

    * [to #43887377]fix: sdk api concurrent call snapshort download file  will conflict
 2022-08-08 17:19:30 +08:00
mulin.lyh
47dda0a5f9 [to #43698011]feat: login with access token 2022-08-02 18:16:28 +08:00
mulin.lyh
c4fb2445d6 [to #43259593]feat: add list model api 
添加list model查询api
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9446917
 2022-07-22 22:46:01 +08:00
mulin.lyh
69047b99ae [to #43387011]feat: ci test to new host and running in docker 
ci 测试迁移新的机器，并且在容器中运行，减小互相干扰的可能
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9427096

    * add docker ci script
 2022-07-20 11:49:05 +08:00
mulin.lyh
3eed264bad [to #43040150] fix: login warn refactor, error message when exception, ut case to new user 
未登录warn提示信息重构，只有遇到请求失败，exception时，如果cookie为空，提示用户login，单元测试用户修改成单独
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9283579

    * [to #42322933] add space dialog-state tracking pipeline 
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9227018

    * init

* token to ids

* add model

* model forward ready

* add intent

* intent preprocessor ready

* intent success

* merge master

* test with model hub

* add flake8

* update

* update

* update

* Merge branch 'master' into nlp/space/gen

* delete file about gen

* init

* fix flake8 bug

* [to #42322933] init

* bug fix

* [to #42322933] init

* update pipeline registry info

* Merge remote-tracking branch 'origin/master' into feat/nli

* [to #42322933] init

* [to #42322933] init

* modify forward

* [to #42322933] init

* generation ready

* init

* Merge branch 'master' into feat/zero_shot_classification

# Conflicts:
#	modelscope/preprocessors/__init__.py

* [to #42322933] bugfix

* [to #42322933] pre commit fix

* fill mask

* registry multi models on model and pipeline

* add tests

* test level >= 0

* local gen ready

* merge with master

* dialog modeling ready

* fix comments: rename and refactor AliceMindMLM; adjust pipeline

* space intent and modeling(generation) are ready

* bug fix

* add dep

* add dep

* support dst data processor

* merge with nlp/space/dst

* merge with master

* Merge remote-tracking branch 'origin' into feat/fill_mask

Conflicts:
	modelscope/models/nlp/__init__.py
	modelscope/pipelines/builder.py
	modelscope/pipelines/outputs.py
	modelscope/preprocessors/nlp.py
	requirements/nlp.txt

* merge with master

* merge with master 2/2

* fix comments

* fix isort for pre-commit check

* allow params pass to pipeline's __call__ method

* Merge remote-tracking branch 'origin/master' into feat/zero_shot_classification

* merge with nli task

* merge with sentiment_classification

* merge with zero_shot_classfication

* merge with fill_mask

* merge with space

* merge with master head

* Merge remote-tracking branch 'origin' into feat/fill_mask

Conflicts:
	modelscope/utils/constant.py

* fix: pipeline module_name from model_type to 'fill_mask' & fix merge bug

* unfiinished change

* fix bug

* unfinished

* unfinished

* revise modelhub dependency

* Merge branch 'feat/nlp_refactor' of http://gitlab.alibaba-inc.com/Ali-MaaS/MaaS-lib into feat/nlp_refactor

* add eval() to pipeline call

* add test level

* ut run passed

* add default args

* tmp

* merge master

* all ut passed

* remove an useless enum

* revert a mis modification

* revert a mis modification

* Merge commit 'ace8af92465f7d772f035aebe98967726655f12c' into feat/nlp

* commit 'ace8af92465f7d772f035aebe98967726655f12c':
  [to #42322933] Add cv-action-recongnition-pipeline to maas lib
  [to #42463204]  support Pil.Image for image_captioning_pipeline
  [to #42670107] restore pydataset test
  [to #42322933] add create if not exist and add(back) create model example         Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9130661
  [to #41474818]fix: fix errors in task name definition

# Conflicts:
#	modelscope/pipelines/builder.py
#	modelscope/utils/constant.py

* Merge branch 'feat/nlp' into feat/nlp_refactor

* feat/nlp:
  [to #42322933] Add cv-action-recongnition-pipeline to maas lib
  [to #42463204]  support Pil.Image for image_captioning_pipeline
  [to #42670107] restore pydataset test
  [to #42322933] add create if not exist and add(back) create model example         Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9130661
  [to #41474818]fix: fix errors in task name definition

# Conflicts:
#	modelscope/pipelines/builder.py

* fix compile bug

* refactor space

* Merge branch 'feat/nlp_refactor' of http://gitlab.alibaba-inc.com/Ali-MaaS/MaaS-lib into feat/nlp_refactor

* Merge remote-tracking branch 'origin' into feat/fill_mask

* fix

* pre-commit lint

* lint file

* lint file

* lint file

* update modelhub dependency

* lint file

* ignore dst_processor temporary

* solve comment: 1. change MaskedLMModelBase to MaskedLanguageModelBase 2. remove a useless import

* recommit

* remove MaskedLanguageModel from __all__

* Merge commit '1a0d4af55a2eee69d89633874890f50eda8f8700' into feat/nlp_refactor

* commit '1a0d4af55a2eee69d89633874890f50eda8f8700':
  [to #42322933] test level check         Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9143809
  [to #42322933] update nlp models name in metainfo         Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9134657

# Conflicts:
#	modelscope/metainfo.py

* update

* revert pipeline params update

* remove zeroshot

* update sequence classfication outpus

* merge with fill mask

* Merge remote-tracking branch 'origin' into feat/fill_mask

* fix

* init dialog state tracking

* fix flake8 warning of dst

* Merge remote-tracking branch 'origin/feat/fill_mask' into feat/nlp

* merge with master

* remove useless test.py

* add init

* merge nlp

* Merge remote-tracking branch 'origin/master' into feat/nlp

* remove unformatted space trainer

* Merge branch 'feat/nlp' into nlp/space/dst

* revise based on comment except chinease comment

* skip ci blocking

* change Chinese notes of space3.0 into English

* translate chinese comment to english

* add space to metainfo

* space dst pipeline is ready, but model's result is wrong

* merge feat/nlp

* merge with master

* change processor

* change example

* test case ready

* dst loacl ready

* update dst conf

* merge feat/nlp

* inform revise

* inherit bug fix

* init

* add 2 complete examples

* fix bug

* add test case

* merge with master

* modify model name

* add missing setting

* add outputs

* modify test level

* modify chinese comment

* remove useless doc

* merge feat/nlp

* Merge remote-tracking branch 'origin' into nlp/space/dst

* Merge branch 'feat/nlp' into nlp/space/dst

* dst test ready

* merge feat nlp

* space outputs normalization

* update dst

* merge feat nlp

* Merge branch 'master' into nlp/space/dst

* update requirement

* merge with master

* Merge remote-tracking branch 'origin/master' into nlp/space/dst

* formating output

* update requirements/nlp

* merge with master

* add test cases

* Merge remote-tracking branch 'origin/master' into nlp/space/dst

* merge with master

* login warn refactor, error message when exception, ut case to new user

* login warn refactor, error message when exception, ut case to new user
 2022-07-08 20:03:23 +08:00
mulin.lyh
c2ae3de640 [to #42879757 #42887254]fix: Repository initialize lfs after clone, add download times counter 
Repository在clone后执行 git lfs install初始化lfs支持，下载技术添加到header里
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9204291

    * [to #42879757 #42887254]fix: Repository initialize lfs after clone, add download times counter
 2022-06-30 14:27:53 +08:00
mulin.lyh
0d17eb5b39 [to #42849800 #42822853 #42822836 #42822791 #42822717 #42820011]fix: bug test bugs 
修复测试bug
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9186775

    * [to #42849800 #42822853 #42822836 #42822791 #42822717 #42820011]fix: test bugs
 2022-06-28 21:12:15 +08:00
yingda.chen
0acbfe1663 [to #42322933] interface refine with doc 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9159678
 2022-06-24 23:54:10 +08:00
mulin.lyh
6991620f59 [to #42698276]fix: git repo operations supports, gitlab token certification support. 2022-06-24 16:43:32 +08:00
yingda.chen
8400ca1fc7 [to #42322933] add create if not exist and add(back) create model example 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9130661
 2022-06-22 18:39:08 +08:00
mulin.lyh
76c6ff6329 [to #42675838]merge model hub code 
合并model hub 代码
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9096493
 2022-06-21 20:04:25 +08:00