Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2026-09-01 19:49:03 +02:00
Code Issues Packages Projects Releases Wiki Activity

merge with master

Browse Source
This commit is contained in:
智丞
2022-07-02 21:58:42 +08:00
parent 90c5e18183 5b98cc1513
commit 54122b2dab
65 changed files with 2550 additions and 506 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
.dev_scripts/run_docker.sh Normal file
View File

@@ -0,0 +1,7 @@
#sudo docker run --name zwm_maas -v /home/wenmeng.zwm/workspace:/home/wenmeng.zwm/workspace --net host -ti reg.docker.alibaba-inc.com/pai-dlc/tensorflow-training:2.3-gpu-py36-cu101-ubuntu18.04 bash
#sudo docker run --name zwm_maas_pytorch -v /home/wenmeng.zwm/workspace:/home/wenmeng.zwm/workspace --net host -ti reg.docker.alibaba-inc.com/pai-dlc/pytorch-training:1.10PAI-gpu-py36-cu113-ubuntu18.04 bash
CONTAINER_NAME=modelscope-dev
IMAGE_NAME=registry.cn-shanghai.aliyuncs.com/modelscope/modelscope
IMAGE_VERSION=v0.1.1-16-g62856fa-devel
MOUNT_DIR=/home/wenmeng.zwm/workspace
sudo docker run --name $CONTAINER_NAME -v $MOUNT_DIR:$MOUNT_DIR --net host -ti ${IMAGE_NAME}:${IMAGE_VERSION} bash

1
.gitattributes vendored
View File

@@ -1,3 +1,4 @@
*.png filter=lfs diff=lfs merge=lfs -text
*.jpg filter=lfs diff=lfs merge=lfs -text
*.mp4 filter=lfs diff=lfs merge=lfs -text
*.wav filter=lfs diff=lfs merge=lfs -text

4
.gitignore vendored
View File

@@ -124,7 +124,3 @@ replace.sh
# Pytorch
*.pth
# audio
*.wav

3
data/test/audios/kws_bofangyinyue.wav Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a72a7b8d1e8be6ebaa09aeee0d71472569bc62cc4872ecfdbd1651bb3d03eaba
size 69110

3
data/test/audios/kws_xiaoyunxiaoyun.wav Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c6b1671bcfa872278c99490cd1acb08297b8df4dc78f268e4b6a582b4364e4a1
size 297684

3
docker/pytorch.dockerfile
View File

@@ -30,7 +30,8 @@ RUN apt-get update &&\
zip \
zlib1g-dev \
unzip \
pkg-config
pkg-config \
libsndfile1
# install modelscope and its python env
WORKDIR /opt/modelscope

3
docs/source/index.rst
View File

@@ -13,6 +13,7 @@ ModelScope doc
quick_start.md
develop.md
faq.md
.. toctree::
:maxdepth: 2
@@ -20,6 +21,8 @@ ModelScope doc
tutorials/index
.. toctree::
:maxdepth: 2
:caption: Changelog

100
docs/source/quick_start.md
View File

@@ -1,72 +1,55 @@
# 快速开始
ModelScope Library目前支持tensorflow，pytorch深度学习框架进行模型训练、推理， 在Python 3.7+, Pytorch 1.8+, Tensorflow1.15+，Tensorflow 2.6上测试可运行。
注： 当前（630）版本仅支持python3.7 以及linux环境，其他环境(mac,windows等)支持预计730完成。
## python环境配置
首先，参考[文档](https://docs.anaconda.com/anaconda/install/) 安装配置Anaconda环境
安装完成后，执行如下命令为modelscope library创建对应的python环境。
```shell
conda create -n modelscope python=3.6
conda create -n modelscope python=3.7
conda activate modelscope
```
检查python和pip命令是否切换到conda环境下。
## 安装深度学习框架
* 安装pytorch[参考链接](https://pytorch.org/get-started/locally/)
```shell
which python
# ~/workspace/anaconda3/envs/modelscope/bin/python
which pip
# ~/workspace/anaconda3/envs/modelscope/bin/pip
pip install torch torchvision
```
注： 本项目只支持`python3`环境，请勿使用python2环境。
## 第三方依赖安装
ModelScope Library目前支持tensorflow，pytorch两大深度学习框架进行模型训练、推理， 在Python 3.6+, Pytorch 1.8+, Tensorflow 2.6上测试可运行，用户可以根据所选模型对应的计算框架进行安装，可以参考如下链接进行安装所需框架:
* [Pytorch安装指导](https://pytorch.org/get-started/locally/)
* [Tensorflow安装指导](https://www.tensorflow.org/install/pip)
部分第三方依赖库需要提前安装numpy
* 安装Tensorflow[参考链接](https://www.tensorflow.org/install/pip)
```shell
pip install --upgrade tensorflow
```
pip install numpy
```
## ModelScope library 安装
注： 如果在安装过程中遇到错误，请前往[常见问题](faq.md)查找解决方案。
### pip安装
执行如下命令：
```shell
pip install -r http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/release/maas/modelscope.txt
pip install model_scope[all] -f https://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/release/maas/repo.html
```
安装成功后，可以执行如下命令进行验证安装是否正确
```shell
python -c "from modelscope.pipelines import pipeline;print(pipeline('image-matting',model='damo/image-matting-person')('http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/data/test/maas/image_matting/test.png'))"
```
### 使用源码安装
适合本地开发调试使用，修改源码后可以直接执行
下载源码可以直接clone代码到本地
```shell
git clone git@gitlab.alibaba-inc.com:Ali-MaaS/MaaS-lib.git modelscope
git fetch origin master
git checkout master
cd modelscope
#安装依赖
```
安装依赖并设置PYTHONPATH
```shell
pip install -r requirements.txt
# 设置PYTHONPATH
export PYTHONPATH=`pwd`
```
### 安装验证
安装成功后，可以执行如下命令进行验证安装是否正确
```shell
python -c "from modelscope.pipelines import pipeline;print(pipeline('image-matting',model='damo/image-matting-person')('http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/data/test/maas/image_matting/test.png'))"
python -c "from modelscope.pipelines import pipeline;print(pipeline('word-segmentation')('今天天气不错，适合 出去游玩'))"
{'output': '今天 天气 不错 ， 适合 出去 游玩'}
```
## 推理
pipeline函数提供了简洁的推理接口，相关介绍和示例请参考[pipeline使用教程](tutorials/pipeline.md)
## 训练
@@ -75,46 +58,3 @@ to be done
## 评估
to be done
## 推理
pipeline函数提供了简洁的推理接口，示例如下， 更多pipeline介绍和示例请参考[pipeline使用教程](tutorials/pipeline.md)
```python
import cv2
import os.path as osp
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks
# 根据任务名创建pipeline
img_matting = pipeline(Tasks.image_matting, model='damo/image-matting-person')
# 直接提供图像文件的url作为pipeline推理的输入
result = img_matting(
'http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/data/test/maas/image_matting/test.png'
)
cv2.imwrite('result.png', result['output_png'])
print(f'Output written to {osp.abspath("result.png")}')
```
此外，pipeline接口也能接收Dataset作为输入，上面的代码同样可以实现为
```python
import cv2
import os.path as osp
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks
from modelscope.msdatasets import MsDataset
# 使用图像url构建MsDataset，此处也可通过 input_location = '/dir/to/images' 来使用本地文件夹
input_location = [
'http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/data/test/maas/image_matting/test.png'
]
dataset = MsDataset.load(input_location, target='image')
img_matting = pipeline(Tasks.image_matting, model='damo/image-matting-person')
# 输入为MsDataset时，输出的结果为迭代器
result = img_matting(dataset)
cv2.imwrite('result.png', next(result)['output_png'])
print(f'Output written to {osp.abspath("result.png")}')
```

94
docs/source/tutorials/pipeline.md
View File

@@ -1,84 +1,62 @@
# Pipeline使用教程
本文将简单介绍如何使用`pipeline`函数加载模型进行推理。`pipeline`函数支持按照任务类型、模型名称从模型仓库
拉取模型进行进行推理，当前支持的任务有
* 人像抠图 (image-matting)
* 基于bert的语义情感分析 (bert-sentiment-analysis)
本文将从如下方面进行讲解如何使用Pipeline模块：
本文简单介绍如何使用`pipeline`函数加载模型进行推理。`pipeline`函数支持按照任务类型、模型名称从模型仓库拉取模型进行进行推理，包含以下几个方面：
* 使用pipeline()函数进行推理
* 指定特定预处理、特定模型进行推理
* 不同场景推理任务示例
## 环境准备
详细步骤可以参考 [快速开始](../quick_start.md)
## Pipeline基本用法
下面以中文分词任务为例，说明pipeline函数的基本用法
1. pipeline函数支持指定特定任务名称，加载任务默认模型，创建对应Pipeline对象
1. pipeline函数支持指定特定任务名称，加载任务默认模型，创建对应pipeline对象
执行如下python代码
```python
>>> from modelscope.pipelines import pipeline
>>> img_matting = pipeline(task='image-matting', model='damo/image-matting-person')
from modelscope.pipelines import pipeline
word_segmentation = pipeline('word-segmentation')
```
2. 传入单张图像url进行处理
2. 输入文本
``` python
>>> import cv2
>>> result = img_matting('http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/data/test/maas/image_matting/test.png')
>>> cv2.imwrite('result.png', result['output_png'])
>>> import os.path as osp
>>> print(f'result file path is {osp.abspath("result.png")}')
input = '今天天气不错，适合出去游玩'
print(word_segmentation(input))
{'output': '今天 天气 不错 ， 适合 出去 游玩'}
```
pipeline对象也支持传入一个列表输入，返回对应输出列表，每个元素对应输入样本的返回结果
3. 输入多条样本
pipeline对象也支持传入多个样本列表输入，返回对应输出列表，每个元素对应输入样本的返回结果
```python
>>> results = img_matting(
[
'http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/data/test/maas/image_matting/test.png',
'http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/data/test/maas/image_matting/test.png',
'http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/data/test/maas/image_matting/test.png',
])
inputs = ['今天天气不错，适合出去游玩','这本书很好，建议你看看']
print(word_segmentation(inputs))
[{'output': '今天 天气 不错 ， 适合 出去 游玩'}, {'output': '这 本 书 很 好 ， 建议 你 看看'}]
```
如果pipeline对应有一些后处理参数，也支持通过调用时候传入.
```python
>>> pipe = pipeline(task_name)
>>> result = pipe(input, post_process_args)
```
## 指定预处理、模型进行推理
pipeline函数支持传入实例化的预处理对象、模型对象，从而支持用户在推理过程中定制化预处理、模型。
下面以文本情感分类为例进行介绍。
由于demo模型为EasyNLP提供的模型，首先，安装EasyNLP
```shell
pip install https://atp-modelzoo-sh.oss-cn-shanghai.aliyuncs.com/release/package/whl/easynlp-0.0.4-py2.py3-none-any.whl
```
下载模型文件
```shell
wget https://atp-modelzoo-sh.oss-cn-shanghai.aliyuncs.com/release/easynlp_modelzoo/alibaba-pai/bert-base-sst2.zip && unzip bert-base-sst2.zip
```
创建tokenizer和模型
1. 首先，创建预处理方法和模型
```python
>>> from modelscope.models import Model
>>> from modelscope.preprocessors import SequenceClassificationPreprocessor
>>> model = Model.from_pretrained('damo/bert-base-sst2')
>>> tokenizer = SequenceClassificationPreprocessor(
model.model_dir, first_sequence='sentence', second_sequence=None)
from modelscope.models import Model
from modelscope.preprocessors import TokenClassifcationPreprocessor
model = Model.from_pretrained('damo/nlp_structbert_word-segmentation_chinese-base')
tokenizer = TokenClassifcationPreprocessor(model.model_dir)
```
使用tokenizer和模型对象创建pipeline
2. 使用tokenizer和模型对象创建pipeline
```python
>>> from modelscope.pipelines import pipeline
>>> semantic_cls = pipeline('text-classification', model=model, preprocessor=tokenizer)
>>> semantic_cls("Hello world!")
from modelscope.pipelines import pipeline
word_seg = pipeline('word-segmentation', model=model, preprocessor=tokenizer)
input = '今天天气不错，适合出去游玩'
print(word_seg(input))
{'output': '今天 天气 不错 ， 适合 出去 游玩'}
```
## 不同场景任务推理示例
人像抠图、语义分类建上述两个例子。 其他例子未来添加。
下面以一个图像任务：人像抠图（'image-matting'）为例，进一步说明pipeline的用法
```python
import cv2
import os.path as osp
from modelscope.pipelines import pipeline
img_matting = pipeline('image-matting')
result = img_matting('http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/data/test/maas/image_matting/test.png')
cv2.imwrite('result.png', result['output_png'])
```

74
modelscope/hub/api.py
View File

@@ -1,6 +1,8 @@
import os
import pickle
import shutil
import subprocess
from collections import defaultdict
from http.cookiejar import CookieJar
from os.path import expanduser
from typing import List, Optional, Tuple, Union
@@ -8,8 +10,11 @@ from typing import List, Optional, Tuple, Union
import requests
from modelscope.utils.logger import get_logger
from ..msdatasets.config import DOWNLOADED_DATASETS_PATH, HUB_DATASET_ENDPOINT
from ..utils.constant import DownloadMode
from .constants import MODELSCOPE_URL_SCHEME
from .errors import InvalidParameter, NotExistError, is_ok, raise_on_error
from .errors import (InvalidParameter, NotExistError, datahub_raise_on_error,
is_ok, raise_on_error)
from .utils.utils import (get_endpoint, get_gitlab_domain,
model_id_to_group_owner_name)
@@ -18,8 +23,9 @@ logger = get_logger()
class HubApi:
def __init__(self, endpoint=None):
def __init__(self, endpoint=None, dataset_endpoint=None):
self.endpoint = endpoint if endpoint is not None else get_endpoint()
self.dataset_endpoint = dataset_endpoint if dataset_endpoint is not None else HUB_DATASET_ENDPOINT
def login(
self,
@@ -241,6 +247,70 @@ class HubApi:
files.append(file)
return files
def list_datasets(self):
path = f'{self.dataset_endpoint}/api/v1/datasets'
headers = None
params = {}
r = requests.get(path, params=params, headers=headers)
r.raise_for_status()
dataset_list = r.json()['Data']
return [x['Name'] for x in dataset_list]
def fetch_dataset_scripts(self,
dataset_name: str,
namespace: str,
download_mode: Optional[DownloadMode],
version: Optional[str] = 'master'):
if namespace is None:
raise ValueError(
f'Dataset from Hubs.modelscope should have a valid "namespace", but get {namespace}'
)
version = version or 'master'
cache_dir = os.path.join(DOWNLOADED_DATASETS_PATH, dataset_name,
namespace, version)
download_mode = DownloadMode(download_mode
or DownloadMode.REUSE_DATASET_IF_EXISTS)
if download_mode == DownloadMode.FORCE_REDOWNLOAD and os.path.exists(
cache_dir):
shutil.rmtree(cache_dir)
os.makedirs(cache_dir, exist_ok=True)
datahub_url = f'{self.dataset_endpoint}/api/v1/datasets/{namespace}/{dataset_name}'
r = requests.get(datahub_url)
resp = r.json()
datahub_raise_on_error(datahub_url, resp)
dataset_id = resp['Data']['Id']
datahub_url = f'{self.dataset_endpoint}/api/v1/datasets/{dataset_id}/repo/tree?Revision={version}'
r = requests.get(datahub_url)
resp = r.json()
datahub_raise_on_error(datahub_url, resp)
file_list = resp['Data']
if file_list is None:
raise NotExistError(
f'The modelscope dataset [dataset_name = {dataset_name}, namespace = {namespace}, '
f'version = {version}] dose not exist')
file_list = file_list['Files']
local_paths = defaultdict(list)
for file_info in file_list:
file_path = file_info['Path']
if file_path.endswith('.py'):
datahub_url = f'{self.dataset_endpoint}/api/v1/datasets/{dataset_id}/repo/files?' \
f'Revision={version}&Path={file_path}'
r = requests.get(datahub_url)
r.raise_for_status()
content = r.json()['Data']['Content']
local_path = os.path.join(cache_dir, file_path)
if os.path.exists(local_path):
logger.warning(
f"Reusing dataset {dataset_name}'s python file ({local_path})"
)
local_paths['py'].append(local_path)
continue
with open(local_path, 'w') as f:
f.writelines(content)
local_paths['py'].append(local_path)
return local_paths
class ModelScopeConfig:
path_credential = expanduser('~/.modelscope/credentials')

4
modelscope/hub/constants.py
View File

@@ -1,6 +1,8 @@
MODELSCOPE_URL_SCHEME = 'http://'
DEFAULT_MODELSCOPE_DOMAIN = '47.94.223.21:31090'
DEFAULT_MODELSCOPE_IP = '47.94.223.21'
DEFAULT_MODELSCOPE_DOMAIN = DEFAULT_MODELSCOPE_IP + ':31090'
DEFAULT_MODELSCOPE_GITLAB_DOMAIN = '101.201.119.157:31102'
DEFAULT_MODELSCOPE_DATA_ENDPOINT = MODELSCOPE_URL_SCHEME + DEFAULT_MODELSCOPE_IP + ':31752'
DEFAULT_MODELSCOPE_GROUP = 'damo'
MODEL_ID_SEPARATOR = '/'

2
modelscope/hub/utils/caching.py
View File

@@ -1,9 +1,7 @@
import hashlib
import logging
import os
import pickle
import tempfile
import time
from shutil import move, rmtree
from modelscope.utils.logger import get_logger

39
modelscope/models/__init__.py
View File

@@ -1,15 +1,30 @@
# Copyright (c) Alibaba, Inc. and its affiliates.
from .audio.ans.frcrn import FRCRNModel
from .audio.kws import GenericKeyWordSpotting
from .audio.tts.am import SambertNetHifi16k
from .audio.tts.vocoder import Hifigan16k
from .base import Model
from .builder import MODELS, build_model
from .multi_modal import OfaForImageCaptioning
from .nlp import (BertForMaskedLM, BertForSequenceClassification, SbertForNLI,
SbertForSentenceSimilarity, SbertForSentimentClassification,
SbertForTokenClassification, SbertForZeroShotClassification,
SpaceForDialogIntent, SpaceForDialogModeling,
SpaceForDialogStateTracking, StructBertForMaskedLM,
VecoForMaskedLM)
try:
from .audio.tts.am import SambertNetHifi16k
from .audio.tts.vocoder import Hifigan16k
except ModuleNotFoundError as e:
if str(e) == "No module named 'tensorflow'":
pass
else:
raise ModuleNotFoundError(e)
try:
from .audio.kws import GenericKeyWordSpotting
from .multi_modal import OfaForImageCaptioning
from .nlp import (BertForMaskedLM, BertForSequenceClassification,
SbertForNLI, SbertForSentenceSimilarity,
SbertForSentimentClassification,
SbertForTokenClassification,
SbertForZeroShotClassification, SpaceForDialogIntent,
SpaceForDialogModeling, SpaceForDialogStateTracking,
StructBertForMaskedLM, VecoForMaskedLM)
from .audio.ans.frcrn import FRCRNModel
except ModuleNotFoundError as e:
if str(e) == "No module named 'pytorch'":
pass
else:
raise ModuleNotFoundError(e)

9
modelscope/models/multi_modal/clip/clip_model.py
View File

@@ -108,7 +108,11 @@ class CLIPForMultiModalEmbedding(Model):
return text_ids_tensor, text_mask_tensor
def forward(self, input: Dict[str, Any]) -> Dict[str, Any]:
output = {'img_embedding': None, 'text_embedding': None}
from modelscope.pipelines.outputs import OutputKeys
output = {
OutputKeys.IMG_EMBEDDING: None,
OutputKeys.TEXT_EMBEDDING: None
}
if 'img' in input and input['img'] is not None:
input_img = input['img']
if isinstance(input_img, Image.Image):
@@ -130,7 +134,8 @@ class CLIPForMultiModalEmbedding(Model):
img_embedding = self.clip_model(
input_data=img_tensor, input_type='img')
output['img_embedding'] = img_embedding.data.cpu().numpy()
from modelscope.pipelines.outputs import OutputKeys
output[OutputKeys.IMG_EMBEDDING] = img_embedding.data.cpu().numpy()
if 'text' in input and input['text'] is not None:
text_str = input['text']

3
modelscope/models/multi_modal/image_captioning_model.py
View File

@@ -76,9 +76,10 @@ class OfaForImageCaptioning(Model):
input = fairseq.utils.move_to_cuda(input, device=self._device)
results, _ = self.eval_caption(self.task, self.generator, self.models,
input)
from ...pipelines.outputs import OutputKeys
return {
'image_id': results[0]['image_id'],
'caption': results[0]['caption']
OutputKeys.CAPTION: results[0][OutputKeys.CAPTION]
}
def postprocess(self, inputs: Dict[str, Any]) -> Dict[str, Any]:

6
modelscope/msdatasets/config.py
View File

@@ -2,6 +2,8 @@ import os
from pathlib import Path
# Cache location
from modelscope.hub.constants import DEFAULT_MODELSCOPE_DATA_ENDPOINT
DEFAULT_CACHE_HOME = '~/.cache'
CACHE_HOME = os.getenv('CACHE_HOME', DEFAULT_CACHE_HOME)
DEFAULT_MS_CACHE_HOME = os.path.join(CACHE_HOME, 'modelscope/hub')
@@ -18,5 +20,5 @@ DEFAULT_DOWNLOADED_DATASETS_PATH = os.path.join(MS_DATASETS_CACHE,
DOWNLOADED_DATASETS_PATH = Path(
os.getenv('DOWNLOADED_DATASETS_PATH', DEFAULT_DOWNLOADED_DATASETS_PATH))
MS_HUB_ENDPOINT = os.environ.get('MS_HUB_ENDPOINT',
'http://47.94.223.21:31752')
HUB_DATASET_ENDPOINT = os.environ.get('HUB_DATASET_ENDPOINT',
DEFAULT_MODELSCOPE_DATA_ENDPOINT)

6
modelscope/msdatasets/ms_dataset.py
View File

@@ -11,7 +11,6 @@ from datasets.utils.file_utils import (is_relative_path,
relative_to_absolute_path)
from modelscope.msdatasets.config import MS_DATASETS_CACHE
from modelscope.msdatasets.utils.ms_api import MsApi
from modelscope.utils.constant import DownloadMode, Hubs
from modelscope.utils.logger import get_logger
@@ -146,8 +145,9 @@ class MsDataset:
use_hf = True
elif is_relative_path(dataset_name) and dataset_name.count(
'/') == 0:
ms_api = MsApi()
dataset_scripts = ms_api.fetch_dataset_scripts(
from modelscope.hub.api import HubApi
api = HubApi()
dataset_scripts = api.fetch_dataset_scripts(
dataset_name, namespace, download_mode, version)
if 'py' in dataset_scripts: # dataset copied from hf datasets
dataset_name = dataset_scripts['py'][0]

0
modelscope/msdatasets/utils/__init__.py
View File

84
modelscope/msdatasets/utils/ms_api.py
View File

@@ -1,84 +0,0 @@
import os
import shutil
from collections import defaultdict
from typing import Optional
import requests
from modelscope.hub.errors import NotExistError, datahub_raise_on_error
from modelscope.msdatasets.config import (DOWNLOADED_DATASETS_PATH,
MS_HUB_ENDPOINT)
from modelscope.utils.constant import DownloadMode
from modelscope.utils.logger import get_logger
logger = get_logger()
class MsApi:
def __init__(self, endpoint=MS_HUB_ENDPOINT):
self.endpoint = endpoint
def list_datasets(self):
path = f'{self.endpoint}/api/v1/datasets'
headers = None
params = {}
r = requests.get(path, params=params, headers=headers)
r.raise_for_status()
dataset_list = r.json()['Data']
return [x['Name'] for x in dataset_list]
def fetch_dataset_scripts(self,
dataset_name: str,
namespace: str,
download_mode: Optional[DownloadMode],
version: Optional[str] = 'master'):
if namespace is None:
raise ValueError(
f'Dataset from Hubs.modelscope should have a valid "namespace", but get {namespace}'
)
version = version or 'master'
cache_dir = os.path.join(DOWNLOADED_DATASETS_PATH, dataset_name,
namespace, version)
download_mode = DownloadMode(download_mode
or DownloadMode.REUSE_DATASET_IF_EXISTS)
if download_mode == DownloadMode.FORCE_REDOWNLOAD and os.path.exists(
cache_dir):
shutil.rmtree(cache_dir)
os.makedirs(cache_dir, exist_ok=True)
datahub_url = f'{self.endpoint}/api/v1/datasets/{namespace}/{dataset_name}'
r = requests.get(datahub_url)
resp = r.json()
datahub_raise_on_error(datahub_url, resp)
dataset_id = resp['Data']['Id']
datahub_url = f'{self.endpoint}/api/v1/datasets/{dataset_id}/repo/tree?Revision={version}'
r = requests.get(datahub_url)
resp = r.json()
datahub_raise_on_error(datahub_url, resp)
file_list = resp['Data']
if file_list is None:
raise NotExistError(
f'The modelscope dataset [dataset_name = {dataset_name}, namespace = {namespace}, '
f'version = {version}] dose not exist')
file_list = file_list['Files']
local_paths = defaultdict(list)
for file_info in file_list:
file_path = file_info['Path']
if file_path.endswith('.py'):
datahub_url = f'{self.endpoint}/api/v1/datasets/{dataset_id}/repo/files?' \
f'Revision={version}&Path={file_path}'
r = requests.get(datahub_url)
r.raise_for_status()
content = r.json()['Data']['Content']
local_path = os.path.join(cache_dir, file_path)
if os.path.exists(local_path):
logger.warning(
f"Reusing dataset {dataset_name}'s python file ({local_path})"
)
local_paths['py'].append(local_path)
continue
with open(local_path, 'w') as f:
f.writelines(content)
local_paths['py'].append(local_path)
return local_paths

19
modelscope/pipelines/audio/__init__.py
View File

@@ -1,3 +1,16 @@
from .kws_kwsbp_pipeline import * # noqa F403
from .linear_aec_pipeline import LinearAECPipeline
from .text_to_speech_pipeline import * # noqa F403
try:
from .kws_kwsbp_pipeline import * # noqa F403
from .linear_aec_pipeline import LinearAECPipeline
except ModuleNotFoundError as e:
if str(e) == "No module named 'torch'":
pass
else:
raise ModuleNotFoundError(e)
try:
from .text_to_speech_pipeline import * # noqa F403
except ModuleNotFoundError as e:
if str(e) == "No module named 'tensorflow'":
pass
else:
raise ModuleNotFoundError(e)

5
modelscope/pipelines/audio/ans_pipeline.py
View File

@@ -10,6 +10,7 @@ from modelscope.metainfo import Pipelines
from modelscope.utils.constant import Tasks
from ..base import Input, Pipeline
from ..builder import PIPELINES
from ..outputs import OutputKeys
def audio_norm(x):
@@ -108,10 +109,10 @@ class ANSPipeline(Pipeline):
current_idx += stride
else:
outputs = self.model(ndarray)['wav_l2'][0].cpu().numpy()
return {'output_pcm': outputs[:nsamples]}
return {OutputKeys.OUTPUT_PCM: outputs[:nsamples]}
def postprocess(self, inputs: Dict[str, Any], **kwargs) -> Dict[str, Any]:
if 'output_path' in kwargs.keys():
sf.write(kwargs['output_path'], inputs['output_pcm'],
sf.write(kwargs['output_path'], inputs[OutputKeys.OUTPUT_PCM],
self.SAMPLE_RATE)
return inputs

50
modelscope/pipelines/audio/kws_kwsbp_pipeline.py
View File

@@ -5,6 +5,8 @@ import stat
import subprocess
from typing import Any, Dict, List
import json
from modelscope.metainfo import Pipelines
from modelscope.models import Model
from modelscope.pipelines.base import Pipeline
@@ -39,6 +41,12 @@ class KeyWordSpottingKwsbpPipeline(Pipeline):
self._preprocessor = preprocessor
self._model = model
self._keywords = None
if 'keywords' in kwargs.keys():
self._keywords = kwargs['keywords']
print('self._keywords len: ', len(self._keywords))
print('self._keywords: ', self._keywords)
def __call__(self, kws_type: str, wav_path: List[str]) -> Dict[str, Any]:
assert kws_type in ['wav', 'pos_testsets', 'neg_testsets',
@@ -197,6 +205,16 @@ class KeyWordSpottingKwsbpPipeline(Pipeline):
return rst_dict
def _run_with_kwsbp(self, inputs: Dict[str, Any]) -> Dict[str, Any]:
opts: str = ''
# setting customized keywords
keywords_json = self._set_customized_keywords()
if len(keywords_json) > 0:
keywords_json_file = os.path.join(inputs['workspace'],
'keyword_custom.json')
with open(keywords_json_file, 'w') as f:
json.dump(keywords_json, f)
opts = '--keyword-custom ' + keywords_json_file
if inputs['kws_set'] == 'roc':
inputs['keyword_grammar_path'] = os.path.join(
@@ -211,7 +229,7 @@ class KeyWordSpottingKwsbpPipeline(Pipeline):
' --sample-rate=' + inputs['sample_rate'] + \
' --keyword-grammar=' + inputs['keyword_grammar_path'] + \
' --wave-scp=' + os.path.join(inputs['pos_data_path'], 'wave.list') + \
' --num-thread=1 > ' + dump_log_path + ' 2>&1'
' --num-thread=1 ' + opts + ' > ' + dump_log_path + ' 2>&1'
os.system(kws_cmd)
if inputs['kws_set'] in ['pos_testsets', 'roc']:
@@ -236,7 +254,7 @@ class KeyWordSpottingKwsbpPipeline(Pipeline):
' --sample-rate=' + inputs['sample_rate'] + \
' --keyword-grammar=' + inputs['keyword_grammar_path'] + \
' --wave-scp=' + wav_list_path + \
' --num-thread=1 > ' + dump_log_path + ' 2>&1'
' --num-thread=1 ' + opts + ' > ' + dump_log_path + ' 2>&1'
p = subprocess.Popen(kws_cmd, shell=True)
process.append(p)
j += 1
@@ -268,7 +286,7 @@ class KeyWordSpottingKwsbpPipeline(Pipeline):
' --sample-rate=' + inputs['sample_rate'] + \
' --keyword-grammar=' + inputs['keyword_grammar_path'] + \
' --wave-scp=' + wav_list_path + \
' --num-thread=1 > ' + dump_log_path + ' 2>&1'
' --num-thread=1 ' + opts + ' > ' + dump_log_path + ' 2>&1'
p = subprocess.Popen(kws_cmd, shell=True)
process.append(p)
j += 1
@@ -447,3 +465,29 @@ class KeyWordSpottingKwsbpPipeline(Pipeline):
threshold_cur += step
return output
def _set_customized_keywords(self) -> Dict[str, Any]:
if self._keywords is not None:
word_list_inputs = self._keywords
word_list = []
for i in range(len(word_list_inputs)):
key = word_list_inputs[i]
new_item = {}
if key.__contains__('keyword'):
name = key['keyword']
new_name: str = ''
for n in range(0, len(name), 1):
new_name += name[n]
new_name += ' '
new_name = new_name.strip()
new_item['name'] = new_name
if key.__contains__('threshold'):
threshold1: float = key['threshold']
new_item['threshold1'] = threshold1
word_list.append(new_item)
out = {'word_list': word_list}
return out
else:
return ''

7
modelscope/pipelines/audio/linear_aec_pipeline.py
View File

@@ -12,6 +12,7 @@ from modelscope.preprocessors.audio import LinearAECAndFbank
from modelscope.utils.constant import ModelFile, Tasks
from ..base import Pipeline
from ..builder import PIPELINES
from ..outputs import OutputKeys
FEATURE_MVN = 'feature.DEY.mvn.txt'
@@ -120,7 +121,7 @@ class LinearAECPipeline(Pipeline):
}
"""
output_data = self._process(inputs['feature'], inputs['base'])
return {'output_pcm': output_data}
return {OutputKeys.OUTPUT_PCM: output_data}
def postprocess(self, inputs: Dict[str, Any], **kwargs) -> Dict[str, Any]:
r"""The post process. Will save audio to file, if the output_path is given.
@@ -140,8 +141,8 @@ class LinearAECPipeline(Pipeline):
"""
if 'output_path' in kwargs.keys():
wav.write(kwargs['output_path'], self.preprocessor.SAMPLE_RATE,
inputs['output_pcm'].astype(np.int16))
inputs['output_pcm'] = inputs['output_pcm'] / 32768.0
inputs[OutputKeys.OUTPUT_PCM].astype(np.int16))
inputs[OutputKeys.OUTPUT_PCM] = inputs[OutputKeys.OUTPUT_PCM] / 32768.0
return inputs
def _process(self, fbanks, mixture):

23
modelscope/pipelines/cv/__init__.py
View File

@@ -1,5 +1,18 @@
from .action_recognition_pipeline import ActionRecognitionPipeline
from .animal_recog_pipeline import AnimalRecogPipeline
from .image_cartoon_pipeline import ImageCartoonPipeline
from .image_matting_pipeline import ImageMattingPipeline
from .ocr_detection_pipeline import OCRDetectionPipeline
try:
from .action_recognition_pipeline import ActionRecognitionPipeline
from .animal_recog_pipeline import AnimalRecogPipeline
except ModuleNotFoundError as e:
if str(e) == "No module named 'torch'":
pass
else:
raise ModuleNotFoundError(e)
try:
from .image_cartoon_pipeline import ImageCartoonPipeline
from .image_matting_pipeline import ImageMattingPipeline
from .ocr_detection_pipeline import OCRDetectionPipeline
except ModuleNotFoundError as e:
if str(e) == "No module named 'tensorflow'":
pass
else:
raise ModuleNotFoundError(e)

3
modelscope/pipelines/cv/action_recognition_pipeline.py
View File

@@ -16,6 +16,7 @@ from modelscope.utils.constant import ModelFile, Tasks
from modelscope.utils.logger import get_logger
from ..base import Pipeline
from ..builder import PIPELINES
from ..outputs import OutputKeys
logger = get_logger()
@@ -49,7 +50,7 @@ class ActionRecognitionPipeline(Pipeline):
def forward(self, input: Dict[str, Any]) -> Dict[str, Any]:
pred = self.perform_inference(input['video_data'])
output_label = self.label_mapping[str(pred)]
return {'output_label': output_label}
return {OutputKeys.LABELS: output_label}
@torch.no_grad()
def perform_inference(self, data, max_bsz=4):

7
modelscope/pipelines/cv/animal_recog_pipeline.py
View File

@@ -18,6 +18,7 @@ from modelscope.utils.constant import ModelFile, Tasks
from modelscope.utils.logger import get_logger
from ..base import Pipeline
from ..builder import PIPELINES
from ..outputs import OutputKeys
logger = get_logger()
@@ -121,7 +122,9 @@ class AnimalRecogPipeline(Pipeline):
label_mapping = f.readlines()
score = torch.max(inputs['outputs'])
inputs = {
'scores': score.item(),
'labels': label_mapping[inputs['outputs'].argmax()].split('\t')[1]
OutputKeys.SCORES:
score.item(),
OutputKeys.LABELS:
label_mapping[inputs['outputs'].argmax()].split('\t')[1]
}
return inputs

5
modelscope/pipelines/cv/image_cartoon_pipeline.py
View File

@@ -17,6 +17,7 @@ from modelscope.utils.constant import Tasks
from modelscope.utils.logger import get_logger
from ..base import Pipeline
from ..builder import PIPELINES
from ..outputs import OutputKeys
if tf.__version__ >= '2.0':
tf = tf.compat.v1
@@ -94,7 +95,7 @@ class ImageCartoonPipeline(Pipeline):
landmarks = self.detect_face(img)
if landmarks is None:
print('No face detected!')
return {'output_png': None}
return {OutputKeys.OUTPUT_IMG: None}
# background process
pad_bg, pad_h, pad_w = padTo16x(img_brg)
@@ -143,7 +144,7 @@ class ImageCartoonPipeline(Pipeline):
res = cv2.resize(res, (ori_w, ori_h), interpolation=cv2.INTER_AREA)
return {'output_png': res}
return {OutputKeys.OUTPUT_IMG: res}
def postprocess(self, inputs: Dict[str, Any]) -> Dict[str, Any]:
return inputs

7
modelscope/pipelines/cv/image_matting_pipeline.py
View File

@@ -12,6 +12,7 @@ from modelscope.utils.constant import ModelFile, Tasks
from modelscope.utils.logger import get_logger
from ..base import Pipeline
from ..builder import PIPELINES
from ..outputs import OutputKeys
logger = get_logger()
@@ -60,9 +61,9 @@ class ImageMattingPipeline(Pipeline):
def forward(self, input: Dict[str, Any]) -> Dict[str, Any]:
with self._session.as_default():
feed_dict = {self.input_name: input['img']}
output_png = self._session.run(self.output, feed_dict=feed_dict)
output_png = cv2.cvtColor(output_png, cv2.COLOR_RGBA2BGRA)
return {'output_png': output_png}
output_img = self._session.run(self.output, feed_dict=feed_dict)
output_img = cv2.cvtColor(output_img, cv2.COLOR_RGBA2BGRA)
return {OutputKeys.OUTPUT_IMG: output_img}
def postprocess(self, inputs: Dict[str, Any]) -> Dict[str, Any]:
return inputs

3
modelscope/pipelines/cv/ocr_detection_pipeline.py
View File

@@ -16,6 +16,7 @@ from modelscope.utils.constant import ModelFile, Tasks
from modelscope.utils.logger import get_logger
from ..base import Pipeline
from ..builder import PIPELINES
from ..outputs import OutputKeys
from .ocr_utils import model_resnet_mutex_v4_linewithchar, ops, utils
if tf.__version__ >= '2.0':
@@ -174,5 +175,5 @@ class OCRDetectionPipeline(Pipeline):
dt_nms = utils.nms_python(dt_n9)
dt_polygons = np.array([o[:8] for o in dt_nms])
result = {'det_polygons': dt_polygons}
result = {OutputKeys.POLYGONS: dt_polygons}
return result

12
modelscope/pipelines/multi_modal/__init__.py
View File

@@ -1,3 +1,9 @@
from .image_captioning_pipeline import ImageCaptionPipeline
from .multi_modal_embedding_pipeline import MultiModalEmbeddingPipeline
from .visual_question_answering_pipeline import VisualQuestionAnsweringPipeline
try:
from .image_captioning_pipeline import ImageCaptionPipeline
from .multi_modal_embedding_pipeline import MultiModalEmbeddingPipeline
from .visual_question_answering_pipeline import VisualQuestionAnsweringPipeline
except ModuleNotFoundError as e:
if str(e) == "No module named 'torch'":
pass
else:
raise ModuleNotFoundError(e)

28
modelscope/pipelines/nlp/__init__.py
View File

@@ -1,11 +1,17 @@
from .dialog_intent_prediction_pipeline import * # noqa F403
from .dialog_modeling_pipeline import * # noqa F403
from .dialog_state_tracking_pipeline import * # noqa F403
from .fill_mask_pipeline import * # noqa F403
from .nli_pipeline import * # noqa F403
from .sentence_similarity_pipeline import * # noqa F403
from .sentiment_classification_pipeline import * # noqa F403
from .sequence_classification_pipeline import * # noqa F403
from .text_generation_pipeline import * # noqa F403
from .word_segmentation_pipeline import * # noqa F403
from .zero_shot_classification_pipeline import * # noqa F403
try:
from .dialog_intent_prediction_pipeline import * # noqa F403
from .dialog_modeling_pipeline import * # noqa F403
from .dialog_state_tracking_pipeline import * # noqa F403
from .fill_mask_pipeline import * # noqa F403
from .nli_pipeline import * # noqa F403
from .sentence_similarity_pipeline import * # noqa F403
from .sentiment_classification_pipeline import * # noqa F403
from .sequence_classification_pipeline import * # noqa F403
from .text_generation_pipeline import * # noqa F403
from .word_segmentation_pipeline import * # noqa F403
from .zero_shot_classification_pipeline import * # noqa F403
except ModuleNotFoundError as e:
if str(e) == "No module named 'torch'":
pass
else:
raise ModuleNotFoundError(e)

7
modelscope/pipelines/nlp/dialog_intent_prediction_pipeline.py
View File

@@ -8,6 +8,7 @@ from ...preprocessors import DialogIntentPredictionPreprocessor
from ...utils.constant import Tasks
from ..base import Pipeline
from ..builder import PIPELINES
from ..outputs import OutputKeys
__all__ = ['DialogIntentPredictionPipeline']
@@ -44,9 +45,9 @@ class DialogIntentPredictionPipeline(Pipeline):
pos = np.where(pred == np.max(pred))
result = {
'prediction': pred,
'label_pos': pos[0],
'label': self.categories[pos[0][0]]
OutputKeys.PREDICTION: pred,
OutputKeys.LABEL_POS: pos[0],
OutputKeys.LABEL: self.categories[pos[0][0]]
}
return result

4
modelscope/pipelines/nlp/dialog_modeling_pipeline.py
View File

@@ -8,6 +8,7 @@ from ...preprocessors import DialogModelingPreprocessor
from ...utils.constant import Tasks
from ..base import Pipeline, Tensor
from ..builder import PIPELINES
from ..outputs import OutputKeys
__all__ = ['DialogModelingPipeline']
@@ -42,7 +43,6 @@ class DialogModelingPipeline(Pipeline):
inputs['resp'])
assert len(sys_rsp) > 2
sys_rsp = sys_rsp[1:len(sys_rsp) - 1]
inputs['response'] = sys_rsp
inputs[OutputKeys.RESPONSE] = sys_rsp
return inputs

3
modelscope/pipelines/nlp/fill_mask_pipeline.py
View File

@@ -11,6 +11,7 @@ from ...utils.config import Config
from ...utils.constant import ModelFile, Tasks
from ..base import Pipeline, Tensor
from ..builder import PIPELINES
from ..outputs import OutputKeys
__all__ = ['FillMaskPipeline']
_type_map = {'veco': 'roberta', 'sbert': 'bert'}
@@ -108,4 +109,4 @@ class FillMaskPipeline(Pipeline):
pred_string = rep_tokens(pred_string, self.rep_map[process_type])
pred_strings.append(pred_string)
return {'text': pred_strings}
return {OutputKeys.TEXT: pred_strings}

3
modelscope/pipelines/nlp/nli_pipeline.py
View File

@@ -11,6 +11,7 @@ from ...preprocessors import NLIPreprocessor
from ...utils.constant import Tasks
from ..base import Pipeline
from ..builder import PIPELINES
from ..outputs import OutputKeys
__all__ = ['NLIPipeline']
@@ -69,4 +70,4 @@ class NLIPipeline(Pipeline):
cls_names = [self.model.id2label[cid] for cid in cls_ids]
return {'scores': probs, 'labels': cls_names}
return {OutputKeys.SCORES: probs, OutputKeys.LABELS: cls_names}

3
modelscope/pipelines/nlp/sentence_similarity_pipeline.py
View File

@@ -10,6 +10,7 @@ from ...preprocessors import SequenceClassificationPreprocessor
from ...utils.constant import Tasks
from ..base import Input, Pipeline
from ..builder import PIPELINES
from ..outputs import OutputKeys
__all__ = ['SentenceSimilarityPipeline']
@@ -69,4 +70,4 @@ class SentenceSimilarityPipeline(Pipeline):
probs = probs[cls_ids].tolist()
cls_names = [self.model.id2label[cid] for cid in cls_ids]
b = 0
return {'scores': probs[b], 'labels': cls_names[b]}
return {OutputKeys.SCORES: probs[b], OutputKeys.LABELS: cls_names[b]}

4
modelscope/pipelines/nlp/sentiment_classification_pipeline.py
View File

@@ -13,6 +13,7 @@ from ...preprocessors import SentimentClassificationPreprocessor
from ...utils.constant import Tasks
from ..base import Input, Pipeline
from ..builder import PIPELINES
from ..outputs import OutputKeys
__all__ = ['SentimentClassificationPipeline']
@@ -73,5 +74,4 @@ class SentimentClassificationPipeline(Pipeline):
probs = probs[cls_ids].tolist()
cls_names = [self.model.id2label[cid] for cid in cls_ids]
return {'scores': probs, 'labels': cls_names}
return {OutputKeys.SCORES: probs, OutputKeys.LABELS: cls_names}

3
modelscope/pipelines/nlp/sequence_classification_pipeline.py
View File

@@ -9,6 +9,7 @@ from modelscope.utils.constant import Tasks
from ...models import Model
from ..base import Input, Pipeline
from ..builder import PIPELINES
from ..outputs import OutputKeys
__all__ = ['SequenceClassificationPipeline']
@@ -64,4 +65,4 @@ class SequenceClassificationPipeline(Pipeline):
cls_names = [self.model.id2label[cid] for cid in cls_ids]
return {'scores': probs, 'labels': cls_names}
return {OutputKeys.SCORES: probs, OutputKeys.LABELS: cls_names}

3
modelscope/pipelines/nlp/text_generation_pipeline.py
View File

@@ -9,6 +9,7 @@ from ...preprocessors import TextGenerationPreprocessor
from ...utils.constant import Tasks
from ..base import Pipeline, Tensor
from ..builder import PIPELINES
from ..outputs import OutputKeys
__all__ = ['TextGenerationPipeline']
@@ -70,4 +71,4 @@ class TextGenerationPipeline(Pipeline):
for _old, _new in replace_tokens_roberta:
pred_string = pred_string.replace(_old, _new)
pred_string.strip()
return {'text': pred_string}
return {OutputKeys.TEXT: pred_string}

6
modelscope/pipelines/nlp/word_segmentation_pipeline.py
View File

@@ -9,6 +9,7 @@ from ...preprocessors import TokenClassifcationPreprocessor
from ...utils.constant import Tasks
from ..base import Pipeline, Tensor
from ..builder import PIPELINES
from ..outputs import OutputKeys
__all__ = ['WordSegmentationPipeline']
@@ -73,7 +74,4 @@ class WordSegmentationPipeline(Pipeline):
if chunk:
chunks.append(chunk)
seg_result = ' '.join(chunks)
rst = {
'output': seg_result,
}
return rst
return {OutputKeys.OUTPUT: seg_result}

5
modelscope/pipelines/nlp/zero_shot_classification_pipeline.py
View File

@@ -14,6 +14,7 @@ from ...preprocessors import ZeroShotClassificationPreprocessor
from ...utils.constant import Tasks
from ..base import Input, Pipeline
from ..builder import PIPELINES
from ..outputs import OutputKeys
__all__ = ['ZeroShotClassificationPipeline']
@@ -82,7 +83,7 @@ class ZeroShotClassificationPipeline(Pipeline):
scores = softmax(logits, axis=-1)
reversed_index = list(reversed(scores.argsort()))
result = {
'labels': [candidate_labels[i] for i in reversed_index],
'scores': [scores[i].item() for i in reversed_index]
OutputKeys.LABELS: [candidate_labels[i] for i in reversed_index],
OutputKeys.SCORES: [scores[i].item() for i in reversed_index],
}
return result

131
modelscope/pipelines/outputs.py
View File

@@ -2,54 +2,76 @@
from modelscope.utils.constant import Tasks
class OutputKeys(object):
SCORES = 'scores'
LABEL = 'label'
LABELS = 'labels'
LABEL_POS = 'label_pos'
POSES = 'poses'
CAPTION = 'caption'
BOXES = 'boxes'
TEXT = 'text'
POLYGONS = 'polygons'
OUTPUT = 'output'
OUTPUT_IMG = 'output_img'
OUTPUT_PCM = 'output_pcm'
IMG_EMBEDDING = 'img_embedding'
TEXT_EMBEDDING = 'text_embedding'
RESPONSE = 'response'
PREDICTION = 'prediction'
TASK_OUTPUTS = {
# ============ vision tasks ===================
# image classification result for single sample
# {
# "labels": ["dog", "horse", "cow", "cat"],
# "scores": [0.9, 0.1, 0.05, 0.05]
# "labels": ["dog", "horse", "cow", "cat"],
# }
Tasks.image_classification: ['scores', 'labels'],
Tasks.image_tagging: ['scores', 'labels'],
Tasks.image_classification: [OutputKeys.SCORES, OutputKeys.LABELS],
Tasks.image_tagging: [OutputKeys.SCORES, OutputKeys.LABELS],
# object detection result for single sample
# {
# "scores": [0.9, 0.1, 0.05, 0.05]
# "labels": ["dog", "horse", "cow", "cat"],
# "boxes": [
# [x1, y1, x2, y2],
# [x1, y1, x2, y2],
# [x1, y1, x2, y2],
# ],
# "labels": ["dog", "horse", "cow", "cat"],
# "scores": [0.9, 0.1, 0.05, 0.05]
# }
Tasks.object_detection: ['scores', 'labels', 'boxes'],
Tasks.object_detection:
[OutputKeys.SCORES, OutputKeys.LABELS, OutputKeys.BOXES],
# instance segmentation result for single sample
# {
# "masks": [
# np.array in bgr channel order
# ],
# "scores": [0.9, 0.1, 0.05, 0.05],
# "labels": ["dog", "horse", "cow", "cat"],
# "scores": [0.9, 0.1, 0.05, 0.05]
# "boxes": [
# np.array in bgr channel order
# ]
# }
Tasks.image_segmentation: ['scores', 'labels', 'boxes'],
Tasks.image_segmentation:
[OutputKeys.SCORES, OutputKeys.LABELS, OutputKeys.BOXES],
# image generation/editing/matting result for single sample
# {
# "output_png": np.array with shape(h, w, 4)
# "output_img": np.array with shape(h, w, 4)
# for matting or (h, w, 3) for general purpose
# }
Tasks.image_editing: ['output_png'],
Tasks.image_matting: ['output_png'],
Tasks.image_generation: ['output_png'],
Tasks.image_editing: [OutputKeys.OUTPUT_IMG],
Tasks.image_matting: [OutputKeys.OUTPUT_IMG],
Tasks.image_generation: [OutputKeys.OUTPUT_IMG],
# action recognition result for single video
# {
# "output_label": "abseiling"
# }
Tasks.action_recognition: ['output_label'],
Tasks.action_recognition: [OutputKeys.LABELS],
# pose estimation result for single sample
# {
@@ -58,48 +80,55 @@ TASK_OUTPUTS = {
# "boxes": np.array with shape [num_pose, 4], each box is
# [x1, y1, x2, y2]
# }
Tasks.pose_estimation: ['poses', 'boxes'],
Tasks.pose_estimation: [OutputKeys.POSES, OutputKeys.BOXES],
# ocr detection result for single sample
# {
# "det_polygons": np.array with shape [num_text, 8], each box is
# "polygons": np.array with shape [num_text, 8], each polygon is
# [x1, y1, x2, y2, x3, y3, x4, y4]
# }
Tasks.ocr_detection: ['det_polygons'],
Tasks.ocr_detection: [OutputKeys.POLYGONS],
# ============ nlp tasks ===================
# text classification result for single sample
# {
# "labels": ["happy", "sad", "calm", "angry"],
# "scores": [0.9, 0.1, 0.05, 0.05]
# "labels": ["happy", "sad", "calm", "angry"],
# }
Tasks.text_classification: ['scores', 'labels'],
Tasks.text_classification: [OutputKeys.SCORES, OutputKeys.LABELS],
# text generation result for single sample
# {
# "text": "this is text generated by a model."
# "text": "this is the text generated by a model."
# }
Tasks.text_generation: ['text'],
Tasks.text_generation: [OutputKeys.TEXT],
# fill mask result for single sample
# {
# "text": "this is the text which masks filled by model."
# }
Tasks.fill_mask: ['text'],
Tasks.fill_mask: [OutputKeys.TEXT],
# word segmentation result for single sample
# {
# "output": "今天 天气 不错 ， 适合 出去 游玩"
# }
Tasks.word_segmentation: ['output'],
Tasks.word_segmentation: [OutputKeys.OUTPUT],
# sentence similarity result for single sample
# {
# "labels": "1",
# "scores": 0.9
# "labels": "1",
# }
Tasks.sentence_similarity: ['scores', 'labels'],
Tasks.sentence_similarity: [OutputKeys.SCORES, OutputKeys.LABELS],
# sentiment classification result for single sample
# {
# "labels": ["happy", "sad", "calm", "angry"],
# "scores": [0.9, 0.1, 0.05, 0.05]
# }
Tasks.sentiment_classification: [OutputKeys.SCORES, OutputKeys.LABELS],
# sentiment classification result for single sample
# {
@@ -110,10 +139,43 @@ TASK_OUTPUTS = {
# zero-shot classification result for single sample
# {
# "scores": [0.9, 0.1, 0.05, 0.05]
# "labels": ["happy", "sad", "calm", "angry"],
# }
Tasks.zero_shot_classification: [OutputKeys.SCORES, OutputKeys.LABELS],
# nli result for single sample
# {
# "labels": ["happy", "sad", "calm", "angry"],
# "scores": [0.9, 0.1, 0.05, 0.05]
# }
Tasks.zero_shot_classification: ['scores', 'labels'],
Tasks.nli: [OutputKeys.SCORES, OutputKeys.LABELS],
# {'pred': array([2.62349960e-03, 4.12110658e-03, 4.12748595e-05, 3.77560973e-05,
# 1.08599677e-04, 1.72710388e-05, 2.95618793e-05, 1.93638436e-04,
# 6.45841064e-05, 1.15997791e-04, 5.11605394e-05, 9.87020373e-01,
# 2.66957268e-05, 4.72324500e-05, 9.74208378e-05, 4.18022355e-05,
# 2.97343540e-05, 5.81317654e-05, 5.44203431e-05, 6.28319322e-05,
# 7.34537680e-05, 6.61411541e-05, 3.62534920e-05, 8.58885178e-05,
# 8.24327726e-05, 4.66077945e-05, 5.32869453e-05, 4.16190960e-05,
# 5.97518992e-05, 3.92273068e-05, 3.44069012e-05, 9.92335918e-05,
# 9.25978165e-05, 6.26462061e-05, 3.32317031e-05, 1.32061413e-03,
# 2.01607945e-05, 3.36636294e-05, 3.99156743e-05, 5.84108493e-05,
# 2.53432900e-05, 4.95731190e-04, 2.64443643e-05, 4.46992999e-05,
# 2.42672231e-05, 4.75615161e-05, 2.66230145e-05, 4.00083954e-05,
# 2.90536875e-04, 4.23891543e-05, 8.63691166e-05, 4.98188965e-05,
# 3.47019341e-05, 4.52718523e-05, 4.20905781e-05, 5.50173208e-05,
# 4.92360487e-05, 3.56021264e-05, 2.13957210e-05, 6.17428886e-05,
# 1.43893281e-04, 7.32152112e-05, 2.91354867e-04, 2.46623786e-05,
# 3.61441926e-05, 3.38475402e-05, 3.44323053e-05, 5.70138109e-05,
# 4.31488479e-05, 4.94503947e-05, 4.30105974e-05, 1.00963116e-04,
# 2.82062047e-05, 1.15582036e-04, 4.48261271e-05, 3.99339879e-05,
# 7.27692823e-05], dtype=float32), 'label_pos': array([11]), 'label': 'lost_or_stolen_card'}
Tasks.dialog_intent_prediction:
[OutputKeys.PREDICTION, OutputKeys.LABEL_POS, OutputKeys.LABEL],
# sys : ['you', 'are', 'welcome', '.', 'have', 'a', 'great', 'day', '!']
Tasks.dialog_modeling: [OutputKeys.RESPONSE],
# nli result for single sample
# {
@@ -189,7 +251,7 @@ TASK_OUTPUTS = {
# {
# "output_pcm": np.array with shape(samples,) and dtype float32
# }
Tasks.speech_signal_process: ['output_pcm'],
Tasks.speech_signal_process: [OutputKeys.OUTPUT_PCM],
# ============ multi-modal tasks ===================
@@ -197,14 +259,15 @@ TASK_OUTPUTS = {
# {
# "caption": "this is an image caption text."
# }
Tasks.image_captioning: ['caption'],
Tasks.image_captioning: [OutputKeys.CAPTION],
# multi-modal embedding result for single sample
# {
# "img_embedding": np.array with shape [1, D],
# "text_embedding": np.array with shape [1, D]
# }
Tasks.multi_modal_embedding: ['img_embedding', 'text_embedding'],
Tasks.multi_modal_embedding:
[OutputKeys.IMG_EMBEDDING, OutputKeys.TEXT_EMBEDDING],
# visual grounding result for single sample
# {
@@ -215,11 +278,11 @@ TASK_OUTPUTS = {
# ],
# "scores": [0.9, 0.1, 0.05, 0.05]
# }
Tasks.visual_grounding: ['boxes', 'scores'],
Tasks.visual_grounding: [OutputKeys.BOXES, OutputKeys.SCORES],
# text_to_image result for a single sample
# {
# "image": np.ndarray with shape [height, width, 3]
# "output_img": np.ndarray with shape [height, width, 3]
# }
Tasks.text_to_image_synthesis: ['image']
Tasks.text_to_image_synthesis: [OutputKeys.OUTPUT_IMG]
}

19
modelscope/preprocessors/__init__.py
View File

@@ -1,14 +1,21 @@
# Copyright (c) Alibaba, Inc. and its affiliates.
from .audio import LinearAECAndFbank
from .base import Preprocessor
from .builder import PREPROCESSORS, build_preprocessor
from .common import Compose
from .image import LoadImage, load_image
from .kws import WavToLists
from .multi_modal import * # noqa F403
from .nlp import * # noqa F403
from .space.dialog_intent_prediction_preprocessor import * # noqa F403
from .space.dialog_modeling_preprocessor import * # noqa F403
from .space.dialog_state_tracking_preprocessor import * # noqa F403
from .text_to_speech import * # noqa F403
try:
from .audio import LinearAECAndFbank
from .multi_modal import * # noqa F403
from .nlp import * # noqa F403
from .space.dialog_intent_prediction_preprocessor import * # noqa F403
from .space.dialog_modeling_preprocessor import * # noqa F403
from .space.dialog_state_tracking_preprocessor import * # noqa F403
except ModuleNotFoundError as e:
if str(e) == "No module named 'tensorflow'":
pass
else:
raise ModuleNotFoundError(e)

1523
modelscope/preprocessors/space/fields/dst_processors.py Normal file
View File

File diff suppressed because it is too large Load Diff

3
modelscope/trainers/nlp/sequence_classification_trainer.py
View File

@@ -14,8 +14,7 @@ PATH = None
logger = get_logger(PATH)
@TRAINERS.register_module(
Tasks.text_classification, module_name=r'bert-sentiment-analysis')
@TRAINERS.register_module(module_name=r'bert-sentiment-analysis')
class SequenceClassificationTrainer(BaseTrainer):
def __init__(self, cfg_file: str, *args, **kwargs):

79
modelscope/utils/check_requirements.py Normal file
View File

@@ -0,0 +1,79 @@
# Copyright (c) Alibaba, Inc. and its affiliates.
from modelscope.utils.constant import Fields, Requirements
from modelscope.utils.import_utils import requires
def get_msg(field):
msg = f'\n{field} requirements not installed, please execute ' \
f'`pip install requirements/{field}.txt` or ' \
f'`pip install modelscope[{field}]`'
return msg
class NLPModuleNotFoundError(ModuleNotFoundError):
def __init__(self, e: ModuleNotFoundError) -> None:
e.msg += get_msg(Fields.nlp)
super().__init__(e)
class CVModuleNotFoundError(ModuleNotFoundError):
def __init__(self, e: ModuleNotFoundError) -> None:
e.msg += get_msg(Fields.cv)
super().__init__(e)
class AudioModuleNotFoundError(ModuleNotFoundError):
def __init__(self, e: ModuleNotFoundError) -> None:
e.msg += get_msg(Fields.audio)
super().__init__(e)
class MultiModalModuleNotFoundError(ModuleNotFoundError):
def __init__(self, e: ModuleNotFoundError) -> None:
e.msg += get_msg(Fields.multi_modal)
super().__init__(e)
def check_nlp():
try:
requires('nlp models', (
Requirements.torch,
Requirements.tokenizers,
))
except ImportError as e:
raise NLPModuleNotFoundError(e)
def check_cv():
try:
requires('cv models', (
Requirements.torch,
Requirements.tokenizers,
))
except ImportError as e:
raise CVModuleNotFoundError(e)
def check_audio():
try:
requires('audio models', (
Requirements.torch,
Requirements.tf,
))
except ImportError as e:
raise AudioModuleNotFoundError(e)
def check_multi_modal():
try:
requires('multi-modal models', (
Requirements.torch,
Requirements.tokenizers,
))
except ImportError as e:
raise MultiModalModuleNotFoundError(e)

5
modelscope/utils/config.py
View File

@@ -17,9 +17,10 @@ from typing import Dict
import addict
from yapf.yapflib.yapf_api import FormatCode
from modelscope.utils.import_utils import (import_modules,
import_modules_from_file,
validate_py_syntax)
from modelscope.utils.logger import get_logger
from modelscope.utils.pymod import (import_modules, import_modules_from_file,
validate_py_syntax)
if platform.system() == 'Windows':
import regex as re # type: ignore

13
modelscope/utils/constant.py
View File

@@ -102,5 +102,18 @@ class ModelFile(object):
TORCH_MODEL_BIN_FILE = 'pytorch_model.bin'
class Requirements(object):
"""Requirement names for each module
"""
protobuf = 'protobuf'
sentencepiece = 'sentencepiece'
sklearn = 'sklearn'
scipy = 'scipy'
timm = 'timm'
tokenizers = 'tokenizers'
tf = 'tf'
torch = 'torch'
TENSORFLOW = 'tensorflow'
PYTORCH = 'pytorch'

324
modelscope/utils/import_utils.py Normal file
View File

@@ -0,0 +1,324 @@
# Copyright (c) Alibaba, Inc. and its affiliates.
# Part of the implementation is borrowed from huggingface/transformers.
import ast
import functools
import importlib.util
import os
import os.path as osp
import sys
import types
from collections import OrderedDict
from functools import wraps
from importlib import import_module
from itertools import chain
from types import ModuleType
from typing import Any
import json
from packaging import version
from modelscope.utils.constant import Fields
from modelscope.utils.logger import get_logger
if sys.version_info < (3, 8):
import importlib_metadata
else:
import importlib.metadata as importlib_metadata
logger = get_logger()
def import_modules_from_file(py_file: str):
""" Import module from a certrain file
Args:
py_file: path to a python file to be imported
Return:
"""
dirname, basefile = os.path.split(py_file)
if dirname == '':
dirname == './'
module_name = osp.splitext(basefile)[0]
sys.path.insert(0, dirname)
validate_py_syntax(py_file)
mod = import_module(module_name)
sys.path.pop(0)
return module_name, mod
def import_modules(imports, allow_failed_imports=False):
"""Import modules from the given list of strings.
Args:
imports (list | str | None): The given module names to be imported.
allow_failed_imports (bool): If True, the failed imports will return
None. Otherwise, an ImportError is raise. Default: False.
Returns:
list[module] | module | None: The imported modules.
Examples:
>>> osp, sys = import_modules(
... ['os.path', 'sys'])
>>> import os.path as osp_
>>> import sys as sys_
>>> assert osp == osp_
>>> assert sys == sys_
"""
if not imports:
return
single_import = False
if isinstance(imports, str):
single_import = True
imports = [imports]
if not isinstance(imports, list):
raise TypeError(
f'custom_imports must be a list but got type {type(imports)}')
imported = []
for imp in imports:
if not isinstance(imp, str):
raise TypeError(
f'{imp} is of type {type(imp)} and cannot be imported.')
try:
imported_tmp = import_module(imp)
except ImportError:
if allow_failed_imports:
logger.warning(f'{imp} failed to import and is ignored.')
imported_tmp = None
else:
raise ImportError
imported.append(imported_tmp)
if single_import:
imported = imported[0]
return imported
def validate_py_syntax(filename):
with open(filename, 'r', encoding='utf-8') as f:
# Setting encoding explicitly to resolve coding issue on windows
content = f.read()
try:
ast.parse(content)
except SyntaxError as e:
raise SyntaxError('There are syntax errors in config '
f'file {filename}: {e}')
# following code borrows implementation from huggingface/transformers
ENV_VARS_TRUE_VALUES = {'1', 'ON', 'YES', 'TRUE'}
ENV_VARS_TRUE_AND_AUTO_VALUES = ENV_VARS_TRUE_VALUES.union({'AUTO'})
USE_TF = os.environ.get('USE_TF', 'AUTO').upper()
USE_TORCH = os.environ.get('USE_TORCH', 'AUTO').upper()
_torch_version = 'N/A'
if USE_TORCH in ENV_VARS_TRUE_AND_AUTO_VALUES and USE_TF not in ENV_VARS_TRUE_VALUES:
_torch_available = importlib.util.find_spec('torch') is not None
if _torch_available:
try:
_torch_version = importlib_metadata.version('torch')
logger.info(f'PyTorch version {_torch_version} available.')
except importlib_metadata.PackageNotFoundError:
_torch_available = False
else:
logger.info('Disabling PyTorch because USE_TF is set')
_torch_available = False
_tf_version = 'N/A'
if USE_TF in ENV_VARS_TRUE_AND_AUTO_VALUES and USE_TORCH not in ENV_VARS_TRUE_VALUES:
_tf_available = importlib.util.find_spec('tensorflow') is not None
if _tf_available:
candidates = (
'tensorflow',
'tensorflow-cpu',
'tensorflow-gpu',
'tf-nightly',
'tf-nightly-cpu',
'tf-nightly-gpu',
'intel-tensorflow',
'intel-tensorflow-avx512',
'tensorflow-rocm',
'tensorflow-macos',
)
_tf_version = None
# For the metadata, we have to look for both tensorflow and tensorflow-cpu
for pkg in candidates:
try:
_tf_version = importlib_metadata.version(pkg)
break
except importlib_metadata.PackageNotFoundError:
pass
_tf_available = _tf_version is not None
if _tf_available:
if version.parse(_tf_version) < version.parse('2'):
pass
else:
logger.info(f'TensorFlow version {_tf_version} available.')
else:
logger.info('Disabling Tensorflow because USE_TORCH is set')
_tf_available = False
_timm_available = importlib.util.find_spec('timm') is not None
try:
_timm_version = importlib_metadata.version('timm')
logger.debug(f'Successfully imported timm version {_timm_version}')
except importlib_metadata.PackageNotFoundError:
_timm_available = False
def is_scipy_available():
return importlib.util.find_spec('scipy') is not None
def is_sklearn_available():
if importlib.util.find_spec('sklearn') is None:
return False
return is_scipy_available() and importlib.util.find_spec('sklearn.metrics')
def is_sentencepiece_available():
return importlib.util.find_spec('sentencepiece') is not None
def is_protobuf_available():
if importlib.util.find_spec('google') is None:
return False
return importlib.util.find_spec('google.protobuf') is not None
def is_tokenizers_available():
return importlib.util.find_spec('tokenizers') is not None
def is_timm_available():
return _timm_available
def is_torch_available():
return _torch_available
def is_torch_cuda_available():
if is_torch_available():
import torch
return torch.cuda.is_available()
else:
return False
def is_tf_available():
return _tf_available
# docstyle-ignore
PROTOBUF_IMPORT_ERROR = """
{0} requires the protobuf library but it was not found in your environment. Checkout the instructions on the
installation page of its repo: https://github.com/protocolbuffers/protobuf/tree/master/python#installation and
follow the ones that match your environment.
"""
# docstyle-ignore
SENTENCEPIECE_IMPORT_ERROR = """
{0} requires the SentencePiece library but it was not found in your environment. Checkout the instructions on the
installation page of its repo: https://github.com/google/sentencepiece#installation and follow the ones
that match your environment.
"""
# docstyle-ignore
SKLEARN_IMPORT_ERROR = """
{0} requires the scikit-learn library but it was not found in your environment. You can install it with:
```
pip install -U scikit-learn
```
In a notebook or a colab, you can install it by executing a cell with
```
!pip install -U scikit-learn
```
"""
# docstyle-ignore
TENSORFLOW_IMPORT_ERROR = """
{0} requires the TensorFlow library but it was not found in your environment. Checkout the instructions on the
installation page: https://www.tensorflow.org/install and follow the ones that match your environment.
"""
# docstyle-ignore
TIMM_IMPORT_ERROR = """
{0} requires the timm library but it was not found in your environment. You can install it with pip:
`pip install timm`
"""
# docstyle-ignore
TOKENIZERS_IMPORT_ERROR = """
{0} requires the 🤗 Tokenizers library but it was not found in your environment. You can install it with:
```
pip install tokenizers
```
In a notebook or a colab, you can install it by executing a cell with
```
!pip install tokenizers
```
"""
# docstyle-ignore
PYTORCH_IMPORT_ERROR = """
{0} requires the PyTorch library but it was not found in your environment. Checkout the instructions on the
installation page: https://pytorch.org/get-started/locally/ and follow the ones that match your environment.
"""
# docstyle-ignore
SCIPY_IMPORT_ERROR = """
{0} requires the scipy library but it was not found in your environment. You can install it with pip:
`pip install scipy`
"""
REQUIREMENTS_MAAPING = OrderedDict([
('protobuf', (is_protobuf_available, PROTOBUF_IMPORT_ERROR)),
('sentencepiece', (is_sentencepiece_available,
SENTENCEPIECE_IMPORT_ERROR)),
('sklearn', (is_sklearn_available, SKLEARN_IMPORT_ERROR)),
('tf', (is_tf_available, TENSORFLOW_IMPORT_ERROR)),
('timm', (is_timm_available, TIMM_IMPORT_ERROR)),
('tokenizers', (is_tokenizers_available, TOKENIZERS_IMPORT_ERROR)),
('torch', (is_torch_available, PYTORCH_IMPORT_ERROR)),
('scipy', (is_scipy_available, SCIPY_IMPORT_ERROR)),
])
def requires(obj, requirements):
if not isinstance(requirements, (list, tuple)):
requirements = [requirements]
if isinstance(obj, str):
name = obj
else:
name = obj.__name__ if hasattr(obj,
'__name__') else obj.__class__.__name__
checks = (REQUIREMENTS_MAAPING[req] for req in requirements)
failed = [msg.format(name) for available, msg in checks if not available()]
if failed:
raise ImportError(''.join(failed))
def torch_required(func):
# Chose a different decorator name than in tests so it's clear they are not the same.
@functools.wraps(func)
def wrapper(*args, **kwargs):
if is_torch_available():
return func(*args, **kwargs)
else:
raise ImportError(f'Method `{func.__name__}` requires PyTorch.')
return wrapper
def tf_required(func):
# Chose a different decorator name than in tests so it's clear they are not the same.
@functools.wraps(func)
def wrapper(*args, **kwargs):
if is_tf_available():
return func(*args, **kwargs)
else:
raise ImportError(f'Method `{func.__name__}` requires TF.')
return wrapper

90
modelscope/utils/pymod.py
View File

@@ -1,90 +0,0 @@
# Copyright (c) Alibaba, Inc. and its affiliates.
import ast
import os
import os.path as osp
import sys
import types
from importlib import import_module
from modelscope.utils.logger import get_logger
logger = get_logger()
def import_modules_from_file(py_file: str):
""" Import module from a certrain file
Args:
py_file: path to a python file to be imported
Return:
"""
dirname, basefile = os.path.split(py_file)
if dirname == '':
dirname == './'
module_name = osp.splitext(basefile)[0]
sys.path.insert(0, dirname)
validate_py_syntax(py_file)
mod = import_module(module_name)
sys.path.pop(0)
return module_name, mod
def import_modules(imports, allow_failed_imports=False):
"""Import modules from the given list of strings.
Args:
imports (list | str | None): The given module names to be imported.
allow_failed_imports (bool): If True, the failed imports will return
None. Otherwise, an ImportError is raise. Default: False.
Returns:
list[module] | module | None: The imported modules.
Examples:
>>> osp, sys = import_modules(
... ['os.path', 'sys'])
>>> import os.path as osp_
>>> import sys as sys_
>>> assert osp == osp_
>>> assert sys == sys_
"""
if not imports:
return
single_import = False
if isinstance(imports, str):
single_import = True
imports = [imports]
if not isinstance(imports, list):
raise TypeError(
f'custom_imports must be a list but got type {type(imports)}')
imported = []
for imp in imports:
if not isinstance(imp, str):
raise TypeError(
f'{imp} is of type {type(imp)} and cannot be imported.')
try:
imported_tmp = import_module(imp)
except ImportError:
if allow_failed_imports:
logger.warning(f'{imp} failed to import and is ignored.')
imported_tmp = None
else:
raise ImportError
imported.append(imported_tmp)
if single_import:
imported = imported[0]
return imported
def validate_py_syntax(filename):
with open(filename, 'r', encoding='utf-8') as f:
# Setting encoding explicitly to resolve coding issue on windows
content = f.read()
try:
ast.parse(content)
except SyntaxError as e:
raise SyntaxError('There are syntax errors in config '
f'file {filename}: {e}')

33
modelscope/utils/registry.py
View File

@@ -1,7 +1,9 @@
# Copyright (c) Alibaba, Inc. and its affiliates.
import inspect
from typing import List, Tuple, Union
from modelscope.utils.import_utils import requires
from modelscope.utils.logger import get_logger
default_group = 'default'
@@ -52,9 +54,14 @@ class Registry(object):
def _register_module(self,
group_key=default_group,
module_name=None,
module_cls=None):
module_cls=None,
requirements=None):
assert isinstance(group_key,
str), 'group_key is required and must be str'
if requirements is not None:
requires(module_cls, requirements)
if group_key not in self._modules:
self._modules[group_key] = dict()
@@ -70,23 +77,11 @@ class Registry(object):
self._modules[group_key][module_name] = module_cls
module_cls.group_key = group_key
if module_name in self._modules[default_group]:
if id(self._modules[default_group][module_name]) == id(module_cls):
return
else:
logger.warning(f'{module_name} is already registered in '
f'{self._name}[{default_group}] and will '
'be overwritten')
logger.warning(f'{self._modules[default_group][module_name]}'
f'to {module_cls}')
# also register module in the default group for faster access
# only by module name
self._modules[default_group][module_name] = module_cls
def register_module(self,
group_key: str = default_group,
module_name: str = None,
module_cls: type = None):
module_cls: type = None,
requirements: Union[List, Tuple] = None):
""" Register module
Example:
@@ -110,17 +105,18 @@ class Registry(object):
default group name is 'default'
module_name: Module name
module_cls: Module class object
requirements: Module necessary requirements
"""
if not (module_name is None or isinstance(module_name, str)):
raise TypeError(f'module_name must be either of None, str,'
f'got {type(module_name)}')
if module_cls is not None:
self._register_module(
group_key=group_key,
module_name=module_name,
module_cls=module_cls)
module_cls=module_cls,
requirements=requirements)
return module_cls
# if module_cls is None, should return a decorator function
@@ -128,7 +124,8 @@ class Registry(object):
self._register_module(
group_key=group_key,
module_name=module_name,
module_cls=module_cls)
module_cls=module_cls,
requirements=requirements)
return module_cls
return _register

5
requirements.txt
View File

@@ -1,6 +1 @@
-r requirements/runtime.txt
-r requirements/pipeline.txt
-r requirements/multi-modal.txt
-r requirements/nlp.txt
-r requirements/audio.txt
-r requirements/cv.txt

7
requirements/audio.txt
View File

@@ -1,10 +1,5 @@
#tts
h5py
https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/TTS/requirements/pytorch_wavelets-1.3.0-py3-none-any.whl
https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/TTS/requirements/ttsfrd-0.0.2-cp36-cp36m-linux_x86_64.whl; python_version=='3.6'
https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/TTS/requirements/ttsfrd-0.0.2-cp37-cp37m-linux_x86_64.whl; python_version=='3.7'
https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/TTS/requirements/ttsfrd-0.0.2-cp38-cp38-linux_x86_64.whl; python_version=='3.8'
https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/TTS/requirements/ttsfrd-0.0.2-cp39-cp39-linux_x86_64.whl; python_version=='3.9'
inflect
keras
librosa
@@ -14,6 +9,7 @@ nara_wpe
numpy
protobuf>3,<=3.20
ptflops
pytorch_wavelets==1.3.0
PyWavelets>=1.0.0
scikit-learn
SoundFile>0.10
@@ -24,4 +20,5 @@ torch
torchaudio
torchvision
tqdm
ttsfrd==0.0.2
unidecode

6
requirements/multi-modal.txt
View File

@@ -1,8 +1,6 @@
datasets
einops
fairseq==maas
ftfy>=6.0.3
https://jirenmr.oss-cn-zhangjiakou.aliyuncs.com/ofa/fairseq-maas-py3-none-any.whl
https://jirenmr.oss-cn-zhangjiakou.aliyuncs.com/ofa/ofa-0.0.2-py3-none-any.whl
ofa==0.0.2
pycocoevalcap>=1.2
pycocotools>=2.0.4
rouge_score

1
requirements/nlp.txt
View File

@@ -1,3 +1,4 @@
http://ait-public.oss-cn-hangzhou-zmf.aliyuncs.com/jizhu/en_core_web_sm-2.3.1.tar.gz
https://alinlp.alibaba-inc.com/pypi/sofa-1.0.5-py3-none-any.whl
sofa==1.0.5
spacy>=2.3.5

6
requirements/pipeline.txt
View File

@@ -1,6 +0,0 @@
#https://atp-modelzoo-sh.oss-cn-shanghai.aliyuncs.com/release/package/whl/easynlp-0.0.4-py2.py3-none-any.whl
# tensorflow
#--find-links https://download.pytorch.org/whl/torch_stable.html
# torch<1.10,>=1.8.0
# torchaudio
# torchvision

10
requirements/runtime.txt
View File

@@ -1,16 +1,18 @@
addict
datasets
easydict
einops
filelock>=3.3.0
numpy
opencv-python-headless
opencv-python
Pillow>=6.2.0
protobuf>3,<=3.20
pyyaml
requests
requests==2.27.1
scipy
setuptools==58.0.4
setuptools
tokenizers<=0.10.3
torch
tqdm>=4.64.0
transformers<=4.16.2
transformers<=4.16.2,>=4.10.3
yapf

13
setup.py
View File

@@ -5,6 +5,8 @@ import shutil
import subprocess
from setuptools import find_packages, setup
from modelscope.utils.constant import Fields
def readme():
with open('README.md', encoding='utf-8') as f:
@@ -169,6 +171,16 @@ if __name__ == '__main__':
pack_resource()
os.chdir('package')
install_requires, deps_link = parse_requirements('requirements.txt')
extra_requires = {}
all_requires = []
for field in dir(Fields):
if field.startswith('_'):
continue
extra_requires[field], _ = parse_requirements(
f'requirements/{field}.txt')
all_requires.append(extra_requires[field])
extra_requires['all'] = all_requires
setup(
name='model-scope',
version=get_version(),
@@ -193,5 +205,6 @@ if __name__ == '__main__':
license='Apache License 2.0',
tests_require=parse_requirements('requirements/tests.txt'),
install_requires=install_requires,
extras_require=extra_requires,
dependency_links=deps_link,
zip_safe=False)

11
tests/pipelines/test_base.py
View File

@@ -8,6 +8,7 @@ import PIL
from modelscope.pipelines import Pipeline, pipeline
from modelscope.pipelines.builder import PIPELINES, add_default_pipeline_info
from modelscope.pipelines.outputs import OutputKeys
from modelscope.utils.constant import Tasks
from modelscope.utils.logger import get_logger
from modelscope.utils.registry import default_group
@@ -68,28 +69,28 @@ class CustomPipelineTest(unittest.TestCase):
outputs['filename'] = inputs['url']
img = inputs['img']
new_image = img.resize((img.width // 2, img.height // 2))
outputs['output_png'] = np.array(new_image)
outputs[OutputKeys.OUTPUT_IMG] = np.array(new_image)
return outputs
def postprocess(self, inputs: Dict[str, Any]) -> Dict[str, Any]:
return inputs
self.assertTrue('custom-image' in PIPELINES.modules[default_group])
self.assertTrue('custom-image' in PIPELINES.modules[dummy_task])
add_default_pipeline_info(dummy_task, 'custom-image', overwrite=True)
pipe = pipeline(pipeline_name='custom-image')
pipe = pipeline(task=dummy_task, pipeline_name='custom-image')
pipe2 = pipeline(dummy_task)
self.assertTrue(type(pipe) is type(pipe2))
img_url = 'data/test/images/image1.jpg'
output = pipe(img_url)
self.assertEqual(output['filename'], img_url)
self.assertEqual(output['output_png'].shape, (318, 512, 3))
self.assertEqual(output[OutputKeys.OUTPUT_IMG].shape, (318, 512, 3))
outputs = pipe([img_url for i in range(4)])
self.assertEqual(len(outputs), 4)
for out in outputs:
self.assertEqual(out['filename'], img_url)
self.assertEqual(out['output_png'].shape, (318, 512, 3))
self.assertEqual(out[OutputKeys.OUTPUT_IMG].shape, (318, 512, 3))
if __name__ == '__main__':

3
tests/pipelines/test_image_captioning.py
View File

@@ -3,6 +3,7 @@
import unittest
from modelscope.pipelines import pipeline
from modelscope.pipelines.outputs import OutputKeys
from modelscope.utils.constant import Tasks
from modelscope.utils.test_utils import test_level
@@ -15,7 +16,7 @@ class ImageCaptionTest(unittest.TestCase):
Tasks.image_captioning,
model='damo/ofa_image-caption_coco_large_en')
result = img_captioning('data/test/images/image_captioning.png')
print(result['caption'])
print(result[OutputKeys.CAPTION])
if __name__ == '__main__':

11
tests/pipelines/test_image_matting.py
View File

@@ -9,6 +9,7 @@ import cv2
from modelscope.fileio import File
from modelscope.msdatasets import MsDataset
from modelscope.pipelines import pipeline
from modelscope.pipelines.outputs import OutputKeys
from modelscope.utils.constant import ModelFile, Tasks
from modelscope.utils.test_utils import test_level
@@ -29,7 +30,7 @@ class ImageMattingTest(unittest.TestCase):
img_matting = pipeline(Tasks.image_matting, model=tmp_dir)
result = img_matting('data/test/images/image_matting.png')
cv2.imwrite('result.png', result['output_png'])
cv2.imwrite('result.png', result[OutputKeys.OUTPUT_IMG])
@unittest.skipUnless(test_level() >= 1, 'skip test in current test level')
def test_run_with_dataset(self):
@@ -41,7 +42,7 @@ class ImageMattingTest(unittest.TestCase):
img_matting = pipeline(Tasks.image_matting, model=self.model_id)
# note that for dataset output, the inference-output is a Generator that can be iterated.
result = img_matting(dataset)
cv2.imwrite('result.png', next(result)['output_png'])
cv2.imwrite('result.png', next(result)[OutputKeys.OUTPUT_IMG])
print(f'Output written to {osp.abspath("result.png")}')
@unittest.skipUnless(test_level() >= 0, 'skip test in current test level')
@@ -49,7 +50,7 @@ class ImageMattingTest(unittest.TestCase):
img_matting = pipeline(Tasks.image_matting, model=self.model_id)
result = img_matting('data/test/images/image_matting.png')
cv2.imwrite('result.png', result['output_png'])
cv2.imwrite('result.png', result[OutputKeys.OUTPUT_IMG])
print(f'Output written to {osp.abspath("result.png")}')
@unittest.skipUnless(test_level() >= 2, 'skip test in current test level')
@@ -57,7 +58,7 @@ class ImageMattingTest(unittest.TestCase):
img_matting = pipeline(Tasks.image_matting)
result = img_matting('data/test/images/image_matting.png')
cv2.imwrite('result.png', result['output_png'])
cv2.imwrite('result.png', result[OutputKeys.OUTPUT_IMG])
print(f'Output written to {osp.abspath("result.png")}')
@unittest.skipUnless(test_level() >= 2, 'skip test in current test level')
@@ -67,7 +68,7 @@ class ImageMattingTest(unittest.TestCase):
img_matting = pipeline(Tasks.image_matting, model=self.model_id)
result = img_matting(dataset)
for i in range(10):
cv2.imwrite(f'result_{i}.png', next(result)['output_png'])
cv2.imwrite(f'result_{i}.png', next(result)[OutputKeys.OUTPUT_IMG])
print(
f'Output written to dir: {osp.dirname(osp.abspath("result_0.png"))}'
)

80
tests/pipelines/test_key_word_spotting.py
View File

@@ -15,8 +15,8 @@ from modelscope.utils.test_utils import test_level
KWSBP_URL = 'https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/KWS/tools/kwsbp'
POS_WAV_FILE = '20200707_spk57db_storenoise52db_40cm_xiaoyun_sox_6.wav'
POS_WAV_URL = 'https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/KWS/pos_testset/' + POS_WAV_FILE
POS_WAV_FILE = 'data/test/audios/kws_xiaoyunxiaoyun.wav'
BOFANGYINYUE_WAV_FILE = 'data/test/audios/kws_bofangyinyue.wav'
POS_TESTSETS_FILE = 'pos_testsets.tar.gz'
POS_TESTSETS_URL = 'https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/KWS/pos_testsets.tar.gz'
@@ -47,12 +47,8 @@ class KeyWordSpottingTest(unittest.TestCase):
# wav, neg_testsets, pos_testsets, roc
kws_set = 'wav'
# downloading wav file
wav_file_path = os.path.join(self.workspace, POS_WAV_FILE)
if not os.path.exists(wav_file_path):
r = requests.get(POS_WAV_URL)
with open(wav_file_path, 'wb') as f:
f.write(r.content)
# get wav file
wav_file_path = POS_WAV_FILE
# downloading kwsbp
kwsbp_file_path = os.path.join(self.workspace, 'kwsbp')
@@ -70,6 +66,7 @@ class KeyWordSpottingTest(unittest.TestCase):
self.assertTrue(preprocessor is not None)
kwsbp_16k_pipline = pipeline(
task=Tasks.key_word_spotting,
pipeline_name=Pipelines.kws_kwsbp,
model=model,
preprocessor=preprocessor)
@@ -91,9 +88,73 @@ class KeyWordSpottingTest(unittest.TestCase):
"""
if kws_result.__contains__('keywords'):
print('test_run_with_wav keywords: ', kws_result['keywords'])
print('test_run_with_wav confidence: ', kws_result['confidence'])
print('test_run_with_wav detected result: ', kws_result['detected'])
print('test_run_with_wav wave time(seconds): ', kws_result['wav_time'])
@unittest.skipUnless(test_level() >= 0, 'skip test in current test level')
def test_run_with_wav_by_customized_keywords(self):
# wav, neg_testsets, pos_testsets, roc
kws_set = 'wav'
# get wav file
wav_file_path = BOFANGYINYUE_WAV_FILE
# downloading kwsbp
kwsbp_file_path = os.path.join(self.workspace, 'kwsbp')
if not os.path.exists(kwsbp_file_path):
r = requests.get(KWSBP_URL)
with open(kwsbp_file_path, 'wb') as f:
f.write(r.content)
model = Model.from_pretrained(self.model_id)
self.assertTrue(model is not None)
cfg_preprocessor = dict(
type=Preprocessors.wav_to_lists, workspace=self.workspace)
preprocessor = build_preprocessor(cfg_preprocessor, Fields.audio)
self.assertTrue(preprocessor is not None)
# customized keyword if you need.
# full settings eg.
# keywords = [
# {'keyword':'你好电视', 'threshold': 0.008},
# {'keyword':'播放音乐', 'threshold': 0.008}
# ]
keywords = [{'keyword': '播放音乐'}]
kwsbp_16k_pipline = pipeline(
task=Tasks.key_word_spotting,
pipeline_name=Pipelines.kws_kwsbp,
model=model,
preprocessor=preprocessor,
keywords=keywords)
self.assertTrue(kwsbp_16k_pipline is not None)
kws_result = kwsbp_16k_pipline(
kws_type=kws_set, wav_path=[wav_file_path, None])
self.assertTrue(kws_result.__contains__('detected'))
"""
kws result json format example:
{
'wav_count': 1,
'kws_set': 'wav',
'wav_time': 9.132938,
'keywords': ['播放音乐'],
'detected': True,
'confidence': 0.660368
}
"""
if kws_result.__contains__('keywords'):
print('test_run_with_wav_by_customized_keywords keywords: ',
kws_result['keywords'])
print('test_run_with_wav_by_customized_keywords confidence: ',
kws_result['confidence'])
print('test_run_with_wav_by_customized_keywords detected result: ',
kws_result['detected'])
print('test_run_with_wav_by_customized_keywords wave time(seconds): ',
kws_result['wav_time'])
@unittest.skipUnless(test_level() >= 1, 'skip test in current test level')
def test_run_with_pos_testsets(self):
# wav, neg_testsets, pos_testsets, roc
@@ -133,6 +194,7 @@ class KeyWordSpottingTest(unittest.TestCase):
self.assertTrue(preprocessor is not None)
kwsbp_16k_pipline = pipeline(
task=Tasks.key_word_spotting,
pipeline_name=Pipelines.kws_kwsbp,
model=model,
preprocessor=preprocessor)
@@ -204,6 +266,7 @@ class KeyWordSpottingTest(unittest.TestCase):
self.assertTrue(preprocessor is not None)
kwsbp_16k_pipline = pipeline(
task=Tasks.key_word_spotting,
pipeline_name=Pipelines.kws_kwsbp,
model=model,
preprocessor=preprocessor)
@@ -298,6 +361,7 @@ class KeyWordSpottingTest(unittest.TestCase):
self.assertTrue(preprocessor is not None)
kwsbp_16k_pipline = pipeline(
task=Tasks.key_word_spotting,
pipeline_name=Pipelines.kws_kwsbp,
model=model,
preprocessor=preprocessor)

3
tests/pipelines/test_person_image_cartoon.py
View File

@@ -7,6 +7,7 @@ import cv2
from modelscope.pipelines import pipeline
from modelscope.pipelines.base import Pipeline
from modelscope.pipelines.outputs import OutputKeys
from modelscope.utils.constant import Tasks
from modelscope.utils.test_utils import test_level
@@ -22,7 +23,7 @@ class ImageCartoonTest(unittest.TestCase):
def pipeline_inference(self, pipeline: Pipeline, input_location: str):
result = pipeline(input_location)
if result is not None:
cv2.imwrite('result.png', result['output_png'])
cv2.imwrite('result.png', result[OutputKeys.OUTPUT_IMG])
print(f'Output written to {osp.abspath("result.png")}')
@unittest.skip('deprecated, download model from model hub instead')

3
tests/pipelines/test_text_to_speech.py
View File

@@ -12,7 +12,7 @@ from modelscope.metainfo import Pipelines, Preprocessors
from modelscope.models import Model
from modelscope.pipelines import pipeline
from modelscope.preprocessors import build_preprocessor
from modelscope.utils.constant import Fields
from modelscope.utils.constant import Fields, Tasks
from modelscope.utils.logger import get_logger
from modelscope.utils.test_utils import test_level
@@ -43,6 +43,7 @@ class TextToSpeechSambertHifigan16kPipelineTest(unittest.TestCase):
self.assertTrue(voc is not None)
sambert_tts = pipeline(
task=Tasks.text_to_speech,
pipeline_name=Pipelines.sambert_hifigan_16k_tts,
config_file='',
model=[am, voc],

22
tests/utils/test_check_requirements.py Normal file
View File

@@ -0,0 +1,22 @@
# Copyright (c) Alibaba, Inc. and its affiliates.
import unittest
from typing import List, Union
from modelscope.utils.check_requirements import NLPModuleNotFoundError, get_msg
from modelscope.utils.constant import Fields
class ImportUtilsTest(unittest.TestCase):
def test_type_module_not_found(self):
with self.assertRaises(NLPModuleNotFoundError) as ctx:
try:
import not_found
except ModuleNotFoundError as e:
raise NLPModuleNotFoundError(e)
self.assertTrue(get_msg(Fields.nlp) in ctx.exception.msg.msg)
if __name__ == '__main__':
unittest.main()