Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent

Autor: Sun, Xingwu, Chen, Yanfeng, Huang, Yiqing, Xie, Ruobing, Zhu, Jiaqi, Zhang, Kai, Li, Shuaipeng, Yang, Zhen, Han, Jonny, Shu, Xiaobo, Bu, Jiahao, Chen, Zhongzhi, Huang, Xuemeng, Lian, Fengzong, Yang, Saiyong, Yan, Jianfeng, Zeng, Yuyuan, Ren, Xiaoqin, Yu, Chao, Wu, Lulu, Mao, Yue, Xia, Jun, Yang, Tao, Zheng, Suncong, Wu, Kan, Jiao, Dian, Xue, Jinbao, Zhang, Xipeng, Wu, Decheng, Liu, Kai, Wu, Dengpeng, Xu, Guanghui, Chen, Shaohua, Chen, Shuang, Feng, Xiao, Hong, Yigeng, Zheng, Junqiang, Xu, Chengcheng, Li, Zongwei, Kuang, Xiong, Hu, Jianglu, Chen, Yiqi, Deng, Yuchi, Li, Guiyang, Liu, Ao, Zhang, Chenchen, Hu, Shihui, Zhao, Zilong, Wu, Zifan, Ding, Yao, Wang, Weichao, Liu, Han, Wang, Roberts, Fei, Hao, Yu, Peijie, Zhao, Ze, Cao, Xun, Wang, Hai, Xiang, Fusheng, Huang, Mengyuan, Xiong, Zhiyuan, Hu, Bin, Hou, Xuebin, Jiang, Lei, Ma, Jianqiang, Wu, Jiajia, Deng, Yaping, Shen, Yi, Wang, Qian, Liu, Weijie, Liu, Jie, Chen, Meng, Dong, Liang, Jia, Weiwen, Chen, Hu, Liu, Feifei, Yuan, Rui, Xu, Huilin, Yan, Zhenxiang, Cao, Tengfei, Hu, Zhichao, Feng, Xinhua, Du, Dong, Yu, Tinghao, Tao, Yangyu, Zhang, Feng, Zhu, Jianchen, Xu, Chengzhong, Li, Xirui, Zha, Chong, Ouyang, Wen, Xia, Yinben, Li, Xiang, He, Zekun, Chen, Rongpeng, Song, Jiawei, Chen, Ruibin, Jiang, Fan, Zhao, Chongqing, Wang, Bo, Gong, Hao, Gan, Rong, Hu, Winston, Kang, Zhanhui, Yang, Yong, Liu, Yuhong, Wang, Di, Jiang, Jie

In this paper, we introduce Hunyuan-Large, which is currently the largest open-source Transformer-based mixture of experts model, with a total of 389 billion parameters and 52 billion activation parameters, capable of handling up to 256K tokens. We c

Externí odkaz: http://arxiv.org/abs/2411.02265

Stochastic Multi-objective Multi-trip AMR Routing Problem with Time Windows

Autor: Cheng, Lulu, Zhao, Ning, Wu, Kan

In recent years, with the rapidly aging population, alleviating the pressure on medical staff has become a critical issue. To improve the work efficiency of medical staff and reduce the risk of infection, we consider the multi-trip autonomous mobile

Externí odkaz: http://arxiv.org/abs/2406.13168

Human-machine cooperation: optimization of drug retrieval sequencing in automated drug dispensing systems

Autor: Yuan, Mengge, Wu, Kan, Zhao, Ning

Automated drug dispensing systems (ADDSs) are increasingly in demand in today's pharmacies, primarily driven by the growing ageing population. Recognizing the practical challenges faced by pharmacies implementing ADDSs, this study aims to optimize th

Externí odkaz: http://arxiv.org/abs/2312.11306

FP8-LM: Training FP8 Large Language Models

Autor: Peng, Houwen, Wu, Kan, Wei, Yixuan, Zhao, Guoshuai, Yang, Yuxiang, Liu, Ze, Xiong, Yifan, Yang, Ziyue, Ni, Bolin, Hu, Jingcheng, Li, Ruihang, Zhang, Miaosen, Li, Chen, Ning, Jia, Wang, Ruizhe, Zhang, Zheng, Liu, Shuguang, Chau, Joe, Hu, Han, Cheng, Peng

In this paper, we explore FP8 low-bit data formats for efficient training of large language models (LLMs). Our key insight is that most variables, such as gradients and optimizer states, in LLM training can employ low-precision data formats without c

Externí odkaz: http://arxiv.org/abs/2310.18313

Stochastic scheduling of autonomous mobile robots at hospitals

Autor: Cheng, Lulu, Zhao, Ning, Yuan, Mengge, Wu, Kan

This paper studies the scheduling of autonomous mobile robots (AMRs) at hospitals where the stochastic travel times and service times of AMRs are affected by the surrounding environment. The routes of AMRs are planned to minimize the daily cost of th

Externí odkaz: http://arxiv.org/abs/2309.12318