mirror of
https://github.com/modelscope/modelscope.git
synced 2025-12-24 20:19:22 +01:00
f7f7eb21dced72762ed26a74bf7baa2be58822c6
1. Change the logic of using fast tokenizer from mode to user arguments and tokenizer_config.json
This is to fix the problem of RANER must use fast tokenizer in some special models.
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10488982
Introduction
ModelScope library is targeted to support training, evaluation and inference for the state of the art models provided by Mind and further support third-party models provided by users outside alibaba.
Design doc
Please refer to alidoc link
Development doc
Please refer to develop.md
ChangeLog
- 20/05/2022 First release version
Refer to change_log.md for more details
Description
Languages
Python
98.7%
Cuda
0.8%
C++
0.4%