requirements/multi-modal.txt

accelerate
cloudpickle
decord>=0.6.0
diffusers>=0.25.0
# 0.12.1 has issue of No such file or directory: 'fairseq/version.txt'
# 0.12.2 not support py311
#fairseq==0.12.2
fairseq @ https://github.com/liyaodev/fairseq/releases/download/v0.12.3.1/fairseq-0.12.3.1-cp311-cp311-linux_x86_64.whl ; python_version == "3.11" and platform_system == "Linux" and platform_machine == "x86_64"
ftfy>=6.0.3
librosa==0.10.1
opencv-python
pycocoevalcap>=1.2
pycocotools>=2.0.4
pydot
# compatible with taming-transformers-rom1504
pytorch_lightning<=1.7.7
rapidfuzz
# rough-score was just recently updated from 0.0.4 to 0.0.7
# which introduced compatability issues that are being investigated
rouge_score<=0.0.4
sacrebleu
safetensors
# scikit-video
soundfile
taming-transformers-rom1504
timm
tokenizers
torchvision
transformers>=4.27.1
# triton==2.0.0.dev20221120
unicodedata2
zhconv
add Damo Chinese Stable Diffusion and fix bugs in DiffusersPipeline Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11386214 * chinese sd & fix sdwrapper * impl preprocess and postprocess in DiffusersPipeline * add accelerate requirements 2023-01-11 21:49:58 +08:00			`accelerate`
Merge code from github 1. Merge(add) daily regression from github PR (daily_regression.yaml) 2. Add lora stable diffusion from github PR Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13010802 * fix: device arg not work, rename device to ngpu (#272) * Correcting the lora stable diffusion example script (#300) * add vad model and punc model in README.md add vad model and punc model * Merge pull request #302 from modelscope/langgz-patch-1 add vad model and punc model in README.md * add 1.6 * modify ignore * Merge pull request #307 from modelscope/dev_rs_16 Merge release 1.6 * undo datetime to 2099 * Merge pull request #311 from modelscope/fix_master_version undo datetime to 2099 * add daily regression workflow * modify workflow name * fix cron format issue * lora trainer * Merge pull request #315 from liuyhwangyh/add_regression_workflow add daily regression workflow 2023-06-21 10:22:06 +08:00			`cloudpickle`
			`decord>=0.6.0`
upgrade diffusers version 2024-01-03 17:22:35 +08:00			`diffusers>=0.25.0`
Merge release1.24 to master (#1298) 2025-04-07 13:32:09 +08:00			`# 0.12.1 has issue of No such file or directory: 'fairseq/version.txt'`
			`# 0.12.2 not support py311`
			`#fairseq==0.12.2`
[Feature] Add pyproject.toml (#1542) * add pyproject.toml * fix 2025-11-06 12:40:04 +08:00			`fairseq @ https://github.com/liyaodev/fairseq/releases/download/v0.12.3.1/fairseq-0.12.3.1-cp311-cp311-linux_x86_64.whl ; python_version == "3.11" and platform_system == "Linux" and platform_machine == "x86_64"`
			`ftfy>=6.0.3`
fix python3.10 compatible issue Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/14678226 * modify librosa version * fix python3.10 compatible issue * remove healpy in requirements for windowns compatible 2023-11-29 17:41:44 +08:00			`librosa==0.10.1`
add structure tasks: sudoku & text2sql Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11314581 2023-01-09 15:53:49 +08:00			`opencv-python`
[to #42322933] add image_caption_pipeline with OFA 1. add OFA whl for image caption pipeline 2. fix a bug in pipelines/builder.py Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8930942 * [to #41669377] docs and tools refinement and release 1. add build_doc linter script 2. add sphinx-docs support 3. add development doc and api doc 4. change version to 0.1.0 for the first internal release version Link: https://code.aone.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8775307 * [to #41669377] add pipeline tutorial and fix bugs 1. add pipleine tutorial 2. fix bugs when using pipeline with certain model and preprocessor Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8814301 * refine doc * refine doc * upload ofa for caption(with source code but not whl) * remove data in gitignore * append uncommitted data dir in ofa * remove ofa_dir , use ofa.whl instead. * update BPE * rollback changes used in debugging. * Merge branch 'master' into ofa/image_caption # Conflicts: # docs/README.md # docs/source/conf.py # docs/source/index.rst # docs/source/tutorials/pipeline.md # maas_lib/models/nlp/sequence_classification_model.py # maas_lib/pipelines/builder.py # maas_lib/version.py # setup.py # tests/pipelines/test_text_classification.py * 1. fix a bug in pipelines/builder.py. 2. modify model_path to model in image_captioning.py. * 1. rename test_image_captioning.py. * format all files using pre-commit. * add fairseq in requirements.txt * add fairseq in requirements.txt * change fairseq path to git repo to a whl on oss in ofa.txt. * change module_name to 'ofa' * Merge remote-tracking branch 'origin/master' into ofa/image_caption # Conflicts: # maas_lib/pipelines/builder.py * optim requirements for ofa / refine image_captioning.py * uncommited change. * feat: Fix confilct, auto commit by WebIDE 2022-06-08 18:38:19 +08:00			`pycocoevalcap>=1.2`
			`pycocotools>=2.0.4`
Merge code from github 1. Merge(add) daily regression from github PR (daily_regression.yaml) 2. Add lora stable diffusion from github PR Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13010802 * fix: device arg not work, rename device to ngpu (#272) * Correcting the lora stable diffusion example script (#300) * add vad model and punc model in README.md add vad model and punc model * Merge pull request #302 from modelscope/langgz-patch-1 add vad model and punc model in README.md * add 1.6 * modify ignore * Merge pull request #307 from modelscope/dev_rs_16 Merge release 1.6 * undo datetime to 2099 * Merge pull request #311 from modelscope/fix_master_version undo datetime to 2099 * add daily regression workflow * modify workflow name * fix cron format issue * lora trainer * Merge pull request #315 from liuyhwangyh/add_regression_workflow add daily regression workflow 2023-06-21 10:22:06 +08:00			`pydot`
[to #45892407]fix: fix pytorch_lighting incompatible with taming-transformers-rom1504 Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10604329 * [to #45892407]fix: fix pytorch_lighting incompatible with taming-transformers-rom1504 2022-11-02 13:51:59 +08:00			`# compatible with taming-transformers-rom1504`
			`pytorch_lightning<=1.7.7`
add structure tasks: sudoku & text2sql Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11314581 2023-01-09 15:53:49 +08:00			`rapidfuzz`
[to #42322933] set rouge_score version 2022/7/22 更新的 rouge_socre 0.0.7 版本存在 import 报错的问题，暂时将 rouge_score 库版本设置为 <= 0.0.4 2022-07-22 18:22:43 +08:00			`# rough-score was just recently updated from 0.0.4 to 0.0.7`
			`# which introduced compatability issues that are being investigated`
			`rouge_score<=0.0.4`
[to #42322933]caption finetune done, add belu metric Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10318299 2022-10-25 12:55:41 +08:00			`sacrebleu`
add clip_interrogator Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12388571 * init commit of clip_interrogator * fix format * fix environment dependencies * fix backend.mps * fix test image path * add model introduction and paper link * add comment example for pipeline, fix device control 2023-05-12 16:06:52 +08:00			`safetensors`
add text-to-video-synthesis 文本生成视频（text-to-video-synthesis）代码 Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11767775 2023-03-09 00:57:43 +08:00			`# scikit-video`
ofa增加asr任务 ofa增加asr任务infer Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10761019 2022-11-25 12:16:33 +08:00			`soundfile`
[to #42322933] Add ofa-text-to-image-synthesis to maas lib Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9590940 2022-08-04 15:33:31 +08:00			`taming-transformers-rom1504`
[to #42322933] add image_caption_pipeline with OFA 1. add OFA whl for image caption pipeline 2. fix a bug in pipelines/builder.py Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8930942 * [to #41669377] docs and tools refinement and release 1. add build_doc linter script 2. add sphinx-docs support 3. add development doc and api doc 4. change version to 0.1.0 for the first internal release version Link: https://code.aone.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8775307 * [to #41669377] add pipeline tutorial and fix bugs 1. add pipleine tutorial 2. fix bugs when using pipeline with certain model and preprocessor Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8814301 * refine doc * refine doc * upload ofa for caption(with source code but not whl) * remove data in gitignore * append uncommitted data dir in ofa * remove ofa_dir , use ofa.whl instead. * update BPE * rollback changes used in debugging. * Merge branch 'master' into ofa/image_caption # Conflicts: # docs/README.md # docs/source/conf.py # docs/source/index.rst # docs/source/tutorials/pipeline.md # maas_lib/models/nlp/sequence_classification_model.py # maas_lib/pipelines/builder.py # maas_lib/version.py # setup.py # tests/pipelines/test_text_classification.py * 1. fix a bug in pipelines/builder.py. 2. modify model_path to model in image_captioning.py. * 1. rename test_image_captioning.py. * format all files using pre-commit. * add fairseq in requirements.txt * add fairseq in requirements.txt * change fairseq path to git repo to a whl on oss in ofa.txt. * change module_name to 'ofa' * Merge remote-tracking branch 'origin/master' into ofa/image_caption # Conflicts: # maas_lib/pipelines/builder.py * optim requirements for ofa / refine image_captioning.py * uncommited change. * feat: Fix confilct, auto commit by WebIDE 2022-06-08 18:38:19 +08:00			`timm`
[to #42322933] lazy load on trainer 2022-08-04 14:07:14 +08:00			`tokenizers`
[to #42362425] fix audio_requirement and refine quickstart, changelog doc * make audio requirements optional * add changelog for version v0.2 * add numpy constraint for compatibility with tensorflow1.15 * update faq * fix nlp requiring tensorflow * add torchvision to multimodal dependency * bump version from 0.2.1 to 0.2.2 * add warning msg when tensorflow is not installed Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9268278 2022-07-05 21:44:33 +08:00			`torchvision`
add clip_interrogator Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12388571 * init commit of clip_interrogator * fix format * fix environment dependencies * fix backend.mps * fix test image path * add model introduction and paper link * add comment example for pipeline, fix device control 2023-05-12 16:06:52 +08:00			`transformers>=4.27.1`
add text-to-video-synthesis 文本生成视频（text-to-video-synthesis）代码 Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11767775 2023-03-09 00:57:43 +08:00			`# triton==2.0.0.dev20221120`
[to #42322933]fix ocr prepreocess & conflict 修复ocr预处理逻辑不一致问题 Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10581697 2022-11-01 10:22:11 +08:00			`unicodedata2`
			`zhconv`