Zobrazeno 1 - 1
of 1
pro vyhledávání: '"Guo, Hefei"'
Autor:
Jin, Yibo, Wang, Tao, Lin, Huimin, Song, Mingyang, Li, Peiyang, Ma, Yipeng, Shan, Yicheng, Yuan, Zhengfan, Li, Cailong, Sun, Yajing, Wu, Tiandeng, Chu, Xing, Huan, Ruizhi, Ma, Li, You, Xiao, Zhou, Wenting, Ye, Yunpeng, Liu, Wen, Xu, Xiangkun, Zhang, Yongsheng, Dong, Tiantian, Zhu, Jiawei, Wang, Zhe, Ju, Xijian, Song, Jianxun, Cheng, Haoliang, Li, Xiaojing, Ding, Jiandong, Guo, Hefei, Zhang, Zhengyong
Serving disaggregated large language models (LLMs) over tens of thousands of xPU devices (GPUs or NPUs) with reliable performance faces multiple challenges. 1) Ignoring the diversity (various prefixes and tidal requests), treating all the prompts in
Externí odkaz:
http://arxiv.org/abs/2408.08147