Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2025-12-25 20:49:37 +01:00
Code Issues Packages Projects Releases Wiki Activity
1,636 Commits 263 Branches 106 Tags
9ff4ab1cd825df574266f93bf550ce00a1a3a67b
Commit Graph

21 Commits

Author SHA1 Message Date
hemu.zp
b3e7eca817 lmit mmdet version 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12339472
 2023-04-15 17:08:55 +08:00
yuze.zyz
2650d37094 Fix the log level and return value of push_to_hub 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12338218
 2023-04-15 16:42:35 +08:00
hemu.zp
a4bafd8b25 Add requirements for jsonplus 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12331254
 2023-04-14 14:05:36 +08:00
hemu.zp
4e78f611e6 Add sortedcontainers to requirements 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12310786
 2023-04-12 22:26:56 +08:00
jiangnana.jnn
6f9d59f469 fix jsonplus, support ndarray 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12056076

* fix jsonplus

* add unittest

* fix json dump np float
 2023-03-21 18:10:10 +08:00
yuze.zyz
4a0cb67e66 fix numpy dependency 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11978446
 2023-03-14 13:31:22 +08:00
mulin.lyh
e54694690f [to #46993990]feat: run ci cases base on code diff to reduct ci test time 2023-02-06 08:00:19 +00:00
Yingda Chen
af62b3e9ad limit pyarrow version 2023-01-31 11:35:09 +08:00
yingda.chen
2e029a1eda update requirment and fix some typos 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11479462
 2023-01-31 01:23:56 +00:00
xingjun.wxj
5a01eca834 [to #42322933] fix datasets version incompatible issue 
最近release的datasets==2.9.0版本在IterableDataset等类和方法上，与老版本(<=2.8.0)不兼容；修改requirements将其版本限制在2.7.0~2.8.0

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11476999

* fix datasets==2.8.0 and datasets==2.9.0 incompatible issue
 2023-01-29 11:13:09 +00:00
xingjun.wxj
43edddd31f [to #42322933] msdataset module refactor and add 1230 features 
1. 优化本地数据集加载链路  
2. local与remote解耦，无网络环境下也可以使用SDK  
3. 升级hf datasets及其相关依赖到最新版(2.7.0+)
4. 解决元数据感知不到数据文件变更的问题  
5. 系统分层设计
6. 本地缓存管理问题  
7. 优化error log输出信息  
8. 支持streaming load	
* a. 支持数据文件为zip格式的streaming
* b. 支持Image/Text/Audio/Biodata等格式数据集的iter
* c. 兼容训练数据在meta中的历史数据集的streaming load
* d. 支持数据文件为文件夹格式的streaming load

9. finetune任务串接进一步规范
* a. 避免出现to_hf_dataset这种使用，将常用的tf相关的func封装起来  
* b. 去掉了跟hf混用的一些逻辑，统一包装到MsDataset里面

10. 超大数据集场景优化
* a. list oss objects: 直接拉取meta中的csv mapping，不需要做 list_oss_objects的api调用（前述提交已实现）
* b. 优化sts过期加载问题（前述提交已实现）

11. 支持dataset_name格式为：namespace/dataset_name的输入方式

参考Aone链接： https://aone.alibaba-inc.com/v2/project/1162242/task/46262894
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11264406
 2023-01-10 07:01:34 +08:00
wenmeng.zwm
070ec00720 remove opencv-python from framework requirements and remove easynlp from nlp default requirements 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11119626
 2022-12-16 14:27:23 +08:00
wenmeng.zwm
4dd99b8f6e Revert "move opencv dependency from framwork to cv " 
This reverts commit e970a6eb43.
 2022-11-30 18:29:03 +08:00
yingda.chen
e970a6eb43 move opencv dependency from framwork to cv 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10708368
 2022-11-12 20:09:48 +08:00
班扬
79c44a6810 add event tracking 2022-11-01 15:41:01 +08:00
班扬
84032f90e3 add event tracking 2022-11-01 15:34:58 +08:00
Yingda Chen
3464324f6b [to #42322933] limit datasets version for now 2022-10-31 22:15:37 +08:00
mulin.lyh
7257f6c6fb [to #45631658]feat support eas deploy 
服务端文档链接(可能需要登录)：
https://test.modelscope.cn/api/v1/deployer/docs
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10478609
 2022-10-24 15:12:48 +08:00
wenmeng.zwm
275f8b4323 Revert "[to #45071449] fix setup error " 
This reverts commit a26e6e3869.
 2022-10-14 13:55:09 +08:00
wenmeng.zwm
a26e6e3869 [to #45071449] fix setup error 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10196007
 2022-10-12 17:33:03 +08:00
wenmeng.zwm
ba74cdf97e [to #43878347] Rename runtime.txt to framework.txt 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10000642

    * rename runtime.txt  to framework.txt
 2022-09-03 12:10:16 +08:00