Mirrors/modelscope
1
0
Fork 0
mirror of https://github.com/modelscope/modelscope.git synced 2025-12-25 20:49:37 +01:00
Code Issues Packages Projects Releases Wiki Activity

add kwargs for save_vocabulary

Browse Source 
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13508568
This commit is contained in:
lukeming.lkm
2023-08-02 18:12:11 +08:00
committed by wenmeng.zwm
parent dece963ad5
commit 1ce78d648d
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
modelscope/models/nlp/qwen/tokenization.py
View File

@@ -150,7 +150,7 @@ class QWenTokenizer(PreTrainedTokenizer):
format(len(ids), self.max_len))
return ids
def save_vocabulary(self, save_directory: str) -> Tuple[str]:
def save_vocabulary(self, save_directory: str, **kwargs) -> Tuple[str]:
"""
Save only the vocabulary of the tokenizer (vocabulary + added tokens).