Commit Graph

7 Commits

Author SHA1 Message Date
Yingda Chen
2b1c839918 Format llm pipeline (#1094)
* format llm pipeline

Co-authored-by: Yingda Chen <yingda.chen@alibaba-inc.com>
2024-11-22 20:04:59 +08:00
hemu.zp
80f76ca475 Support stream output for transformers model
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/13271136
* support stream for transformers model

* set test_level >= 2

* support hf model and chatglm2

* remove streaming_output for chatglm2
2023-07-25 17:41:32 +08:00
hemu.zp
96c2d42f09 Add StreamingMixin
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/12445731
* StreamingMixin poc

* update design

* Merge branch 'master' into feat/StreamingMixin

* add dicstr

* make postprocessor input consistent
2023-06-08 19:40:14 +08:00
hemu.zp
429cfee826 Support run text generation pipeline with args
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11937122
2023-03-10 09:48:10 +08:00
hemu.zp
fd0099c92d [to #42322933] Refactor megatron-util
Rename import lib 'megatron' to 'megatron_util' and add error message for users when import failed.

Use initialize_megatron as a unified initialization entry in megatron-util, which can accept configuration input of ConfigDict in MaaS-lib.

Wrap the initialization process into the utils/megatron_utils.py file, add default parameters for the existing large model to be compatible with the uploaded configuration file.

The version of megatron_cfg currently supports v3 (default), v1 and moe.
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/11231840
2022-12-29 15:01:45 +08:00
yuze.zyz
707cbef013 [to #42322933]Fix bug in daily UT
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10491891
2022-10-22 23:25:18 +08:00
hemu.zp
3b1f1a0252 [to #42322933] Add GPT3 tensor parallel inference
添加基于 Megatron-v3 的 GPT3 tensor 并行的推理代码
复用 DistributedPipeline 与 megatron-util
适用模型:1.3B/2.7B/13B 参数的 GPT-3 预训练生成模型
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10416721
2022-10-18 15:58:33 +08:00