Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2025-12-16 16:27:45 +01:00
Code Issues Packages Projects Releases Wiki Activity
1,711 Commits 262 Branches 106 Tags
25a7398aeab1a36466fd5e5278317f7de9032d19
Commit Graph

17 Commits

Author SHA1 Message Date
mulin.lyh
25a7398aea docker file py38 and py37 compatible merge 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12722824
* debug

* add missing deps

* add stanza

* fix numpy issue

* numpy version

* tf case

* numpy <= 1.22.0

* fix tf 1.15.5 case

* add python38 and python37 compatible docker file

* cv add fvcore

* add easycv

* remove debug code

* fix style issue

* fix python3.7 python3.8 compatible issue

* comment

* modify run config

* scipy<=1.7.3

* revert scipy<=1.7.3

* fix compatible bug

* fix compatible bug

* add easycv

* update version: 1.6.1rc0
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12754942
fix python3.8 and python3.8 compatible issue

* numpy version change

* numpy version

* numpy version

* fix numpy version

* fix numpy version

* update version: 1.6.1rc0
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12754942
fix python3.8 and python3.8 compatible issue

* restore setup.py

* restore setup.py

* add build base image

* fix style

* numpy version fix

* optimize build image, split base and modelscope image

* fix style issue

* modify example path

* modify examples folder

* add SETUPTOOLS_USE_DISTUTILS=stdlib comments
 2023-06-08 10:01:58 +08:00
mulin.lyh
86128cd3a8 [to #47612577]feat: image supoort 3d models, python3.8 test version 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11542009


* mmcv-full<=1.7.0 for mmdet3d

* fix code style issue

* add speechbrain==0.5.7

* add compatible comments
 2023-02-07 02:28:20 +00:00
wenmeng.zwm
ed859e5274 Title: merge master-github and fix conflict 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11370549
 2023-01-10 14:03:08 +08:00
mulin.lyh
04b9704b48 [to #47190649]fix: merge v1.1 docker file 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11371422

    * v1.1 docker file modify
 2023-01-10 14:02:15 +08:00
wenmeng.zwm
9ce750f4a9 merge master-github and fix conflict 2023-01-10 11:12:37 +08:00
xingjun.wxj
43edddd31f [to #42322933] msdataset module refactor and add 1230 features 
1. 优化本地数据集加载链路  
2. local与remote解耦，无网络环境下也可以使用SDK  
3. 升级hf datasets及其相关依赖到最新版(2.7.0+)
4. 解决元数据感知不到数据文件变更的问题  
5. 系统分层设计
6. 本地缓存管理问题  
7. 优化error log输出信息  
8. 支持streaming load	
* a. 支持数据文件为zip格式的streaming
* b. 支持Image/Text/Audio/Biodata等格式数据集的iter
* c. 兼容训练数据在meta中的历史数据集的streaming load
* d. 支持数据文件为文件夹格式的streaming load

9. finetune任务串接进一步规范
* a. 避免出现to_hf_dataset这种使用，将常用的tf相关的func封装起来  
* b. 去掉了跟hf混用的一些逻辑，统一包装到MsDataset里面

10. 超大数据集场景优化
* a. list oss objects: 直接拉取meta中的csv mapping，不需要做 list_oss_objects的api调用（前述提交已实现）
* b. 优化sts过期加载问题（前述提交已实现）

11. 支持dataset_name格式为：namespace/dataset_name的输入方式

参考Aone链接： https://aone.alibaba-inc.com/v2/project/1162242/task/46262894
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11264406
 2023-01-10 07:01:34 +08:00
chenxujun
d510683475 Update modelscope_env_init.sh 
Fix typo
 2022-12-03 06:59:31 +08:00
mulin.lyh
eb37921ad6 [to #45915958]feat: v1.0.2 docker file update, add deepspeed, apex, uncore 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10617428
 2022-11-11 11:03:00 +08:00
zfm104435
08327d5af9 在modelscope 基础镜像中添加 jupyter lab 插件，支持静默关闭功能 
在modelscope 基础镜像中添加 jupyter lab 插件，用于收集用户在notebook 中的鼠标、键盘事件，来支持静默关闭功能。
主要改变如下：
1. 新增jupyterlab_active_log 插件的源码和编译后的文件
2. 修改Dockerfile.ubuntu 文件，将上述源码文件拷贝的jupyter的插件目录中。
3. 修改pip和conda 的镜像地址为阿里云的
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10690152
 2022-11-10 22:10:13 +08:00
mulin.lyh
ac07b719e9 [to #45546922]feat: add fasttext package 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10431169

    * [to #45546922]feat: add fasttext package
 2022-10-17 20:51:58 +08:00
mulin.lyh
c51b74c2ea [to #45220645]fix: fix ffmpeg mp4 encoder bug 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10284398

    * [to #45220645]fix: fix ffmpeg mp4 encoder bug
 2022-09-28 13:36:09 +08:00
mulin.lyh
ad6bb1e7d9 [to #44790143]fix: add ipythonkernel to image for dsw 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10049527

    * add ipykernel to image for dsw
 2022-09-07 20:51:15 +08:00
wenmeng.zwm
ba74cdf97e [to #43878347] Rename runtime.txt to framework.txt 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10000642

    * rename runtime.txt  to framework.txt
 2022-09-03 12:10:16 +08:00
mulin.lyh
01bb751425 [to #43653669]feat: auto build docker images 
auto build docker images
aone 任务： https://test.aone.alibaba-inc.com/jobs/1824567?buildId=143470479
修改任务分支
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9566518

    * [to #43653669]feat: auto build docker images
 2022-08-22 16:01:42 +08:00
wenmeng.zwm
8e51a073a6 [to #42966122] requirements enchanment and self-host repo support 
* add self-hosted repo:
* add extra requirements for different field and reduce necessary requirements
* update docker file with so required by audio
* add requirements checker which will be used later when implement lazy import
* remove repeated requirements and replace opencv-python-headless with opencv-python

example usage:
```shell
pip install model_scope[all] -f https://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/release/maas/repo.html
pip install model_scope[cv] -f https://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/release/maas/repo.html
pip install model_scope[nlp] -f https://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/release/maas/repo.html
pip install model_scope[audio] -f https://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/release/maas/repo.html
pip install model_scope[multi-modal] -f https://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/release/maas/repo.html

```
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9211383
 2022-07-01 16:38:06 +08:00
wenmeng.zwm
4f7928bb6e [to #42362853] formalize the output of pipeline and make pipeline reusable 
* format pipeline output and check it
* fix UT
* add docstr to clarify the difference between model.postprocess and pipeline.postprocess

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9051405
 2022-06-16 11:15:09 +08:00
wenmeng.zwm
67086e26f9 [to #42362932] feat: docker_support 
* add dockerfile
* uninstall opencv-python-headless
* update develop doc
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9020302
 2022-06-14 16:29:12 +08:00