Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2026-07-09 20:09:17 +02:00
Code Issues Packages Projects Releases Wiki Activity

[Backport] Docker fixes from release/1.38 (#1735)

Browse Source 
* Upgrade numpy to 2.x for 1.38 Docker images

- Replace deprecated numpy aliases (np.math.ceil → math.ceil, np.Inf → np.inf)
- Upgrade Docker constraints: numpy>=2.0, cython>=3.0, remove scipy upper bound

* Add ipywidgets dependency to Docker images

* update docker

* fix

* fix

* fix

* fix cpu image

* fix(docker): force numpy>=2.0 after evalscope install

ms-opencompass pulls numpy<2.0, downgrading numpy from 2.x to 1.26.4.
Force reinstall numpy>=2.0 after pip install .[eval] to restore it.

* add metax dockerfile and its requirements for ms-swift 4.2.x (#1734)

* Update npu dockerfile (#1736)

* fix(docker): remove unrelated added docker files

---------

Co-authored-by: Jintao Huang <huangjintao.hjt@alibaba-inc.com>
Co-authored-by: dwd <dwd1044898101@gmail.com>
Co-authored-by: addsubmuldiv <zyh13227@163.com>
This commit is contained in:
Yunlin Mao
2026-06-26 17:10:49 +08:00
committed by GitHub
parent 39583ec639
commit 020cad8229
8 changed files with 47 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
.github/workflows/docker-image.yml vendored
View File

@@ -27,9 +27,9 @@ on:
description: 'Other params in --xxx xxx'
required: false
python_version:
description: 'Python version to use, default is 3.11.11'
description: 'Python version to use, default is 3.12.13'
required: false
default: '3.11.11'
default: '3.12.13'
run-name: Docker-${{ inputs.modelscope_branch }}-${{ inputs.image_type }}-${{ inputs.workflow_name }}-${{ inputs.python_version }}-by-@${{ github.actor }}

9
docker/Dockerfile.ascend
View File

@@ -87,15 +87,16 @@ RUN pip uninstall ms-swift modelscope -y && pip install --no-cache-dir pip==23.*
if [ "$INSTALL_MS_DEPS" = "True" ]; then \
pip install --no-cache-dir omegaconf==2.0.6 && \
pip install 'editdistance==0.8.1' && \
pip install --no-cache-dir 'cython<=0.29.36' versioneer 'numpy<2.0' && \
pip install --no-cache-dir 'cython>=3.0' versioneer 'numpy>=2.0' && \
pip install --no-cache-dir -r /var/modelscope/framework.txt && \
pip install --no-cache-dir -r /var/modelscope/audio.txt -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html && \
pip install --no-cache-dir -r /var/modelscope/tests.txt && \
pip install --no-cache-dir -r /var/modelscope/server.txt && \
pip install --no-cache-dir https://modelscope.oss-cn-beijing.aliyuncs.com/packages/imageio_ffmpeg-0.4.9-py3-none-any.whl --no-dependencies --force && \
pip install --no-cache-dir 'scipy<1.13.0' && \
pip install --no-cache-dir 'scipy' && \
pip install --no-cache-dir funtextprocessing typeguard==2.13.3 scikit-learn && \
pip install --no-cache-dir 'decord>=0.6.0' mpi4py paint_ldm ipykernel fasttext && \
pip install --no-cache-dir ipywidgets && \
pip install --no-cache-dir 'blobfile>=1.0.5' && \
pip uninstall MinDAEC -y && \
pip install https://modelscope.oss-cn-beijing.aliyuncs.com/releases/dependencies/MinDAEC-0.0.2-py3-none-any.whl && \
@@ -115,7 +116,9 @@ RUN source /usr/local/Ascend/ascend-toolkit/set_env.sh && \
pip install --no-cache-dir -U funasr scikit-learn && \
pip install --no-cache-dir -U qwen_vl_utils qwen_omni_utils librosa 'timm>=0.9.0' transformers accelerate peft trl safetensors && \
cd /ms-swift && pip install --no-cache-dir -e '.[llm]' && \
pip install --no-cache-dir -e '.[eval]' && pip install evalscope -U --no-dependencies && pip install ms-agent -U --no-dependencies && \
pip install --no-cache-dir -e '.[eval]' && pip install evalscope -U --no-dependencies && \
pip install --no-cache-dir "numpy>=2.0" --force-reinstall --no-deps && \
pip install ms-agent -U --no-dependencies && \
cd /tmp && GIT_LFS_SKIP_SMUDGE=1 git clone -b {modelscope_branch} --single-branch https://github.com/modelscope/modelscope.git && \
cd modelscope && pip install . -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html && \
cd / && rm -fr /tmp/modelscope && \

19
docker/Dockerfile.ubuntu
View File

@@ -25,15 +25,16 @@ RUN pip uninstall ms-swift modelscope -y && pip --no-cache-dir install pip==23.*
if [ "$INSTALL_MS_DEPS" = "True" ]; then \
pip --no-cache-dir install omegaconf==2.0.6 && \
pip install 'editdistance==0.8.1' && \
pip install --no-cache-dir 'cython<=0.29.36' versioneer 'numpy<2.0' && \
pip install --no-cache-dir 'cython>=3.0' versioneer 'numpy>=2.0' && \
pip install --no-cache-dir -r /var/modelscope/framework.txt && \
pip install --no-cache-dir -r /var/modelscope/audio.txt -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html && \
pip install --no-cache-dir -r /var/modelscope/tests.txt && \
pip install --no-cache-dir -r /var/modelscope/server.txt && \
pip install --no-cache-dir https://modelscope.oss-cn-beijing.aliyuncs.com/packages/imageio_ffmpeg-0.4.9-py3-none-any.whl --no-dependencies --force && \
pip install --no-cache-dir 'scipy<1.13.0' && \
pip install --no-cache-dir 'scipy' && \
pip install --no-cache-dir funtextprocessing typeguard==2.13.3 scikit-learn -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html && \
pip install --no-cache-dir 'decord>=0.6.0' mpi4py paint_ldm ipykernel fasttext -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html && \
pip install --no-cache-dir ipywidgets && \
pip install --no-cache-dir 'blobfile>=1.0.5' && \
pip uninstall MinDAEC -y && \
pip install https://modelscope.oss-cn-beijing.aliyuncs.com/releases/dependencies/MinDAEC-0.0.2-py3-none-any.whl && \
@@ -51,7 +52,9 @@ RUN bash /tmp/install.sh {version_args} && \
pip install --no-cache-dir -U qwen_vl_utils qwen_omni_utils librosa timm transformers accelerate peft trl safetensors && \
cd /tmp && GIT_LFS_SKIP_SMUDGE=1 git clone -b {swift_branch} --single-branch https://github.com/modelscope/ms-swift.git && \
cd ms-swift && pip install .[llm] && \
pip install .[eval] && pip install evalscope -U --no-dependencies && pip install ms-agent -U --no-dependencies && \
pip install .[eval] && pip install evalscope -U --no-dependencies && \
pip install --no-cache-dir "numpy>=2.0" --force-reinstall --no-deps && \
pip install ms-agent -U --no-dependencies && \
cd / && rm -fr /tmp/ms-swift && pip cache purge; \
cd /tmp && GIT_LFS_SKIP_SMUDGE=1 git clone -b {modelscope_branch} --single-branch https://github.com/modelscope/modelscope.git && \
cd modelscope && pip install . -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html && \
@@ -66,16 +69,16 @@ RUN bash /tmp/install.sh {version_args} && \
RUN if [ "$IMAGE_TYPE" = "gpu" ]; then \
pip install --no-cache-dir math_verify "gradio<5.33" "deepspeed<0.18" ray -U && \
pip install --no-cache-dir liger_kernel wandb swanlab nvitop pre-commit "transformers" "trl<0.25" "peft<0.18" huggingface-hub -U && \
pip install --no-cache-dir math_verify "gradio<5.33" "deepspeed<0.19" ray -U && \
pip install --no-cache-dir mcore-bridge -i https://pypi.org/simple/ -U && \
pip install --no-cache-dir liger_kernel wandb swanlab nvitop pre-commit "transformers<5.9" "trl<1.0" "peft<0.20" huggingface-hub -U && \
pip install --no-cache-dir --no-build-isolation transformer_engine[pytorch]; \
cd /tmp && GIT_LFS_SKIP_SMUDGE=1 git clone https://github.com/NVIDIA/apex && \
cd apex && pip install -v --disable-pip-version-check --no-cache-dir --no-build-isolation --config-settings "--build-option=--cpp_ext" --config-settings "--build-option=--cuda_ext" ./ && \
cd / && rm -fr /tmp/apex && pip cache purge; \
pip install --no-cache-dir "megatron-core==0.15.*"; \
pip uninstall autoawq -y; \
pip install --no-cache-dir "megatron-core==0.16.*" -U; \
elif [ "$IMAGE_TYPE" = "cpu" ]; then \
pip install --no-cache-dir huggingface-hub transformers peft diffusers -U; \
pip install --no-cache-dir huggingface-hub "transformers<5.9" peft diffusers -U; \
else \
pip install "transformers<5.0" "tokenizers<0.22" "trl<0.23" "diffusers<0.35" --no-dependencies; \
fi

3
docker/Dockerfile.ubuntu.old
View File

@@ -1,5 +1,8 @@
FROM {base_image}
ARG CUR_TIME={cur_time}
RUN echo $CUR_TIME
RUN cd /tmp && GIT_LFS_SKIP_SMUDGE=1 git clone -b {modelscope_branch} --single-branch https://github.com/modelscope/modelscope.git && \
cd modelscope && pip install . -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html && \
cd / && rm -fr /tmp/modelscope && pip cache purge;

33
docker/build_image.py
View File

@@ -3,6 +3,7 @@ import os
import platform
import re
import subprocess
from copy import copy
from datetime import datetime
from typing import Any
@@ -26,9 +27,9 @@ class Builder:
# A mirrored image of nvidia/cuda:12.4.0-devel-ubuntu22.04
args.base_image = 'nvidia/cuda:12.8.1-cudnn-devel-ubuntu22.04'
if not args.torch_version:
args.torch_version = '2.9.1'
args.torchaudio_version = '2.9.1'
args.torchvision_version = '0.24.1'
args.torch_version = '2.10.0'
args.torchaudio_version = '2.10.0'
args.torchvision_version = '0.25.0'
if not args.optimum_version:
args.optimum_version = '2.0.0'
if not args.tf_version:
@@ -36,7 +37,7 @@ class Builder:
if not args.cuda_version:
args.cuda_version = '12.8.1'
if not args.vllm_version:
args.vllm_version = '0.15.1'
args.vllm_version = '0.19.1'
if not args.lmdeploy_version:
args.lmdeploy_version = '0.11.0'
if not args.autogptq_version:
@@ -141,6 +142,7 @@ class OldCPUImageBuilder(Builder):
content = content.replace('{base_image}', old_cpu_image)
content = content.replace('{modelscope_branch}',
self.args.modelscope_branch)
content = content.replace('{cur_time}', formatted_time)
return content
def image(self) -> str:
@@ -198,6 +200,7 @@ class OldGPUImageBuilder(Builder):
content = content.replace('{base_image}', old_gpu_image)
content = content.replace('{modelscope_branch}',
self.args.modelscope_branch)
content = content.replace('{cur_time}', formatted_time)
return content
def image(self) -> str:
@@ -340,6 +343,15 @@ class StableCPUImageBuilder(Builder):
class StableGPUImageBuilder(Builder):
"""Dependencies will be stable versions"""
def init_args(self, args: Any) -> Any:
if not args.torch_version:
args.torch_version = '2.10.0'
args.torchaudio_version = '2.10.0'
args.torchvision_version = '0.25.0'
if not args.vllm_version:
args.vllm_version = '0.19.1'
return super().init_args(args)
def generate_dockerfile(self) -> str:
meta_file = './docker/install.sh'
with open('docker/Dockerfile.extra_install', 'r') as f:
@@ -401,11 +413,6 @@ RUN pip install --no-cache-dir -U icecream soundfile pybind11 py-spy
class LatestGPUImageBuilder(StableGPUImageBuilder):
"""Dependencies will be latest versions"""
def init_args(self, args: Any) -> Any:
if not args.vllm_version:
args.vllm_version = '0.16.0'
return super().init_args(args)
def generate_dockerfile(self) -> str:
meta_file = './docker/install.sh'
with open('docker/Dockerfile.extra_install', 'r') as f:
@@ -418,8 +425,7 @@ RUN pip install --no-cache-dir -U icecream soundfile pybind11 py-spy
version_args = (
f'{self.args.torch_version} {self.args.torchvision_version} {self.args.torchaudio_version} '
f'{self.args.vllm_version} {self.args.lmdeploy_version} {self.args.autogptq_version} '
f'{self.args.optimum_version}'
f'{self.args.flashattn_version}')
f'{self.args.flashattn_version} {self.args.optimum_version}')
with open('docker/Dockerfile.ubuntu', 'r') as f:
content = f.read()
content = content.replace('{base_image}', self.args.base_image)
@@ -427,7 +433,7 @@ RUN pip install --no-cache-dir -U icecream soundfile pybind11 py-spy
content = content.replace('{meta_file}', meta_file)
content = content.replace('{version_args}', version_args)
content = content.replace('{cur_time}', formatted_time)
content = content.replace('{install_ms_deps}', 'True')
content = content.replace('{install_ms_deps}', 'False')
content = content.replace('{image_type}', 'gpu')
content = content.replace('{torch_version}',
self.args.torch_version)
@@ -570,7 +576,7 @@ RUN pip install --no-cache-dir -U icecream soundfile pybind11 py-spy
parser = argparse.ArgumentParser()
parser.add_argument('--base_image', type=str, default=None)
parser.add_argument('--image_type', type=str)
parser.add_argument('--python_version', type=str, default='3.11.11')
parser.add_argument('--python_version', type=str, default='3.12.13')
parser.add_argument('--ubuntu_version', type=str, default='22.04')
parser.add_argument('--torch_version', type=str, default=None)
parser.add_argument('--torchvision_version', type=str, default=None)
@@ -607,4 +613,5 @@ else:
raise ValueError(f'Unsupported image_type: {args.image_type}')
for builder in builder_cls:
args = copy(args)
builder(args, args.dry_run)()

5
docker/install.sh
View File

@@ -7,14 +7,11 @@ vllm_version=${4:-0.6.0}
lmdeploy_version=${5:-0.6.1}
autogptq_version=${6:-0.7.1}
flashattn_version=${7:-2.7.1.post4}
optimum_version=${8:-2.0.0}
pip uninstall -y torch torchvision torchaudio
pip install --no-cache-dir torch==$torch_version torchvision==$torchvision_version torchaudio==$torchaudio_version
pip install --no-cache-dir torch==$torch_version torchvision==$torchvision_version torchaudio==$torchaudio_version
pip install --no-cache-dir tiktoken transformers_stream_generator bitsandbytes deepspeed torchmetrics decord optimum openai-whisper
# pip install https://github.com/Dao-AILab/flash-attention/releases/download/v2.6.3/flash_attn-2.6.3+cu123torch2.4cxx11abiTRUE-cp310-cp310-linux_x86_64.whl
@@ -24,8 +21,6 @@ MAX_JOBS=16 pip install --no-cache-dir flash_attn==$flashattn_version --no-build
pip install --no-cache-dir triton -U && pip cache purge
pip install --no-cache-dir optimum==$optimum_version
if [[ "$(printf '%s\n' "0.6.0" "$vllm_version" | sort -V | head -n1)" = "0.6.0" ]]; then
# vllm_version is >= 0.6.0
pip install --no-cache-dir vllm==$vllm_version && pip cache purge

2
modelscope/models/cv/video_depth_estimation/models/model_checkpoint.py
View File

@@ -40,7 +40,7 @@ class ModelCheckpoint:
self.kth_best_model = ''
self.best = 0
# Monitoring modes
torch_inf = torch.tensor(np.Inf)
torch_inf = torch.tensor(np.inf)
mode_dict = {
'min': (torch_inf, 'min'),
'max': (-torch_inf, 'max'),

7
modelscope/pipelines/multi_modal/soonet_video_temporal_grounding_pipeline.py
View File

@@ -1,5 +1,6 @@
# Copyright 2022-2023 The Alibaba Fundamental Vision Team Authors. All rights reserved.
import math
import os
from typing import Any, Dict
@@ -144,12 +145,12 @@ class SOONetVideoTemporalGroundingPipeline(Pipeline):
start_ts, end_ts, scale_boundaries = list(), list(), [0]
ori_video_length = len(imgs)
pad_video_length = int(
np.math.ceil(ori_video_length / self.max_anchor_length)
math.ceil(ori_video_length / self.max_anchor_length)
* self.max_anchor_length)
for i in range(self.config.nscales):
anchor_length = self.config.snippet_length * (2**i)
pad_feat_length = pad_video_length // anchor_length
nfeats = np.math.ceil(ori_video_length / anchor_length)
nfeats = math.ceil(ori_video_length / anchor_length)
s_times = np.arange(0, nfeats).astype(np.float32) * (
anchor_length // self.fps)
e_times = np.arange(1, nfeats + 1).astype(np.float32) * (
@@ -180,7 +181,7 @@ class SOONetVideoTemporalGroundingPipeline(Pipeline):
#
ori_video_length, feat_dim = video_feats.shape
pad_video_length = int(
np.math.ceil(ori_video_length / self.max_anchor_length)
math.ceil(ori_video_length / self.max_anchor_length)
* self.max_anchor_length)
pad_video_feats = torch.zeros((pad_video_length, feat_dim),
dtype=float)