requirements/framework.txt

addict
attrs
datasets>=2.16.0,<2.19.0
einops
filelock>=3.3.0
huggingface_hub
numpy
oss2
pandas
Pillow>=6.2.0
# pyarrow 9.0.0 introduced event_loop core dump
pyarrow>=6.0.0,!=9.0.0
python-dateutil>=2.1
pyyaml
requests>=2.25
scipy
setuptools
simplejson>=3.3.0
sortedcontainers>=1.5.9
tqdm>=4.64.0
urllib3>=1.26
yapf
[to #41402703] add basic modules * add constant * add logger module * add registry and builder module * add fileio module * add requirements and setup.cfg * add config module and tests * add citest script Link: https://code.aone.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8718998 2022-05-17 10:15:00 +08:00			`addict`
[to #45631658]feat support eas deploy 服务端文档链接(可能需要登录)： https://test.modelscope.cn/api/v1/deployer/docs Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10478609 2022-10-24 15:12:48 +08:00			`attrs`
set datasets <2.19.0 2024-04-19 19:40:00 +08:00			`datasets>=2.16.0,<2.19.0`
[to #42966122] requirements enchanment and self-host repo support * add self-hosted repo: * add extra requirements for different field and reduce necessary requirements * update docker file with so required by audio * add requirements checker which will be used later when implement lazy import * remove repeated requirements and replace opencv-python-headless with opencv-python example usage: ```shell pip install model_scope[all] -f https://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/release/maas/repo.html pip install model_scope[cv] -f https://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/release/maas/repo.html pip install model_scope[nlp] -f https://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/release/maas/repo.html pip install model_scope[audio] -f https://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/release/maas/repo.html pip install model_scope[multi-modal] -f https://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/release/maas/repo.html ``` Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9211383 2022-07-01 16:38:06 +08:00			`einops`
[to #42675838]merge model hub code 合并model hub 代码 Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9096493 2022-06-21 20:04:25 +08:00			`filelock>=3.3.0`
Dataset refactor (#807) * add main entry in ms_dataset * update func get_data_patterns import * modify return_config_only * modify return_config_only to dataset_info_only * udpate version for test * del get_logger(__name__) * fix py script loading * fix loading py and without py * add subset support * add hf_datasets_util; refine list_repo_tree_ms; fix private datasets loading issue * update version to rc5 * fix and support preview for dataset_info_only mode * fix urlencode * update to rc7 * loading of dataset_infos.json is deprecated; 2. add some ut * update version * add escapechar for read_csv and to_csv * add params: Source=SDK * add create_dataset func * overwrite _get_paths_info * update & version * update list_repo_tree name * add get_module_with_script, fix download imports * fix py script loading issue in dataset_module_factory * fix create dataset * update log info in api 2024-03-22 17:30:34 +08:00			`huggingface_hub`
Update framework.txt (#453) remove version restrict for numpy and pandas 2023-08-09 18:00:44 +08:00			`numpy`
[to #43660556] msdataset数据集加载 Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9552632 * load csv dataset from modelscoop 2022-07-29 12:22:48 +08:00			`oss2`
Update framework.txt (#453) remove version restrict for numpy and pandas 2023-08-09 18:00:44 +08:00			`pandas`
[to #42461396] add Pillow version constraint and update hub version 1. use Pillow >= 6.2.0 2. change skip test msg for image caption Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9041194 2022-06-15 11:43:52 +08:00			`Pillow>=6.2.0`
limit pyarrow version 2023-01-31 11:35:09 +08:00			`# pyarrow 9.0.0 introduced event_loop core dump`
[to #46993990]feat: run ci cases base on code diff to reduct ci test time 2023-02-06 08:00:19 +00:00			`pyarrow>=6.0.0,!=9.0.0`
Add requirements for jsonplus Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12331254 2023-04-14 14:05:36 +08:00			`python-dateutil>=2.1`
[to #41402703] add basic modules * add constant * add logger module * add registry and builder module * add fileio module * add requirements and setup.cfg * add config module and tests * add citest script Link: https://code.aone.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8718998 2022-05-17 10:15:00 +08:00			`pyyaml`
[to #50607174]feat: support parallel download large model file Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13113235 * [to #50607174]feat: support parallel download large model file 2023-06-30 15:30:00 +08:00			`requests>=2.25`
[to #41669377] add speech AEC pipeline Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8973072 * [to #41669377] docs and tools refinement and release 1. add build_doc linter script 2. add sphinx-docs support 3. add development doc and api doc 4. change version to 0.1.0 for the first internal release version Link: https://code.aone.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8775307 * [to #41669377] add pipeline tutorial and fix bugs 1. add pipleine tutorial 2. fix bugs when using pipeline with certain model and preprocessor Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8814301 * refine doc * feat: add audio aec pipeline and preprocessor * feat: add audio aec model classes * feat: add audio aec loss functions * refactor:delete no longer used loss function * [to #42281043] support kwargs in pipeline Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8949062 * support kwargs in pipeline * update develop doc with CR instruction * Merge branch 'release/0.1' into dev/aec * style: reformat code by pre-commit tools * feat:support maas_lib pipeline auto downloading model * test:add aec test case as sample code * feat:aec pipeline use config from maashub * feat:aec pipeline use feature parameters from maashub * update setup.cfg to disable PEP8 rule W503 in flake8 and yapf * format:fix double quoted strings, indent issues and optimize import * refactor:extract some constant in aec pipeline * refactor: delete no longer used __main__ statement * chore:change all Chinese comments to English * fix: change file name style to lower case * refactor: rename model name * feat:load C++ .so from LD_LIBRARY_PATH * feat:register PROPROCESSOR for LinearAECAndFbank * refactory:move aec process from postprocess() to forward() and update comments * refactory:add more readable error message when audio sample rate is not 16000 * fix: package maas_lib renamed to modelscope in import statement * feat: optimize the error message of audio layer classes * format: delete empty lines * refactor: rename audio preprocessor and optimize error message * refactor: change aec model id to damo/speech_dfsmn_aec_psm_16k * refactor: change sample audio file url to public oss * Merge branch 'master' into dev/aec * feat: add output info for aec pipeline * fix: normalize output audio data to [-1.0, 1.0] * refactor:use constant from ModelFile * feat: AEC pipeline can use c++ lib in current working directory and the test will download it * fix: c++ downloading should work wherever test is triggerd 2022-06-17 19:56:11 +08:00			`scipy`
remove pai-easycv requirement Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12564234 2023-05-11 14:31:18 +08:00			`setuptools`
Add requirements for jsonplus Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12331254 2023-04-14 14:05:36 +08:00			`simplejson>=3.3.0`
Add sortedcontainers to requirements Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12310786 2023-04-12 22:26:56 +08:00			`sortedcontainers>=1.5.9`
[to #42675838]merge model hub code 合并model hub 代码 Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9096493 2022-06-21 20:04:25 +08:00			`tqdm>=4.64.0`
[to #50607174]feat: support parallel download large model file Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13113235 * [to #50607174]feat: support parallel download large model file 2023-06-30 15:30:00 +08:00			`urllib3>=1.26`
[to #41402703] add basic modules * add constant * add logger module * add registry and builder module * add fileio module * add requirements and setup.cfg * add config module and tests * add citest script Link: https://code.aone.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/8718998 2022-05-17 10:15:00 +08:00			`yapf`