examples/pytorch/llm/utils/__init__.py

from .dataset import DATASET_MAPPER, get_dataset, process_dataset
from .models import MODEL_MAPPER, get_model_tokenizer
from .utils import (DEFAULT_PROMPT, MyMetric, data_collate_fn, get_T_max,
                    get_work_dir, inference, parse_args, plot_images,
                    print_example, print_model_info, read_tensorboard_file,
                    seed_everything, show_freeze_layers, stat_dataset,
                    tensorboard_smoothing, tokenize_function)
add qwen 7b base and chat 添加QWen 7b base模型和chat模型及相关pipelines Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13482235 * add qwen 7b base and chat * fix logger * update examples, lint test * add unittest for qwen base and chat * rename qwen to qwen-7b * resolve imports and add a registry to text-generation * reset load model from pretrained * fix precheck * skip qwen test case now * remove strange file 2023-08-02 09:25:21 +08:00			`from .dataset import DATASET_MAPPER, get_dataset, process_dataset`
			`from .models import MODEL_MAPPER, get_model_tokenizer`
			`from .utils import (DEFAULT_PROMPT, MyMetric, data_collate_fn, get_T_max,`
			`get_work_dir, inference, parse_args, plot_images,`
			`print_example, print_model_info, read_tensorboard_file,`
			`seed_everything, show_freeze_layers, stat_dataset,`
			`tensorboard_smoothing, tokenize_function)`