Výsledky vyhledávání - "Zhao, Rongqian"

Report

Expert-Token Resonance: Redefining MoE Routing through Affinity-Driven Active Selection

Autor: Li, Jing, Sun, Zhijie, Lin, Dachao, He, Xuan, Lin, Yi, Zheng, Binfan, Zeng, Li, Zhao, Rongqian, Chen, Xin

Mixture-of-Experts (MoE) architectures have emerged as a paradigm-shifting approach for large language models (LLMs), offering unprecedented computational efficiency. However, these architectures grapple with challenges of token distribution imbalanc

Externí odkaz: http://arxiv.org/abs/2406.00023

Zobrazit plný text záznamu

Report

WindGP: Efficient Graph Partitioning on Heterogenous Machines

Autor: Zeng, Li, Huang, Haohan, Zheng, Binfan, Yang, Kang, Shao, Shengcheng, Zhou, Jinhua, Xie, Jun, Zhao, Rongqian, Chen, Xin

Graph Partitioning is widely used in many real-world applications such as fraud detection and social network analysis, in order to enable the distributed graph computing on large graphs. However, existing works fail to balance the computation cost an

Externí odkaz: http://arxiv.org/abs/2403.00331

Zobrazit plný text záznamu

Report

LocMoE: A Low-Overhead MoE for Large Language Model Training

Autor: Li, Jing, Sun, Zhijie, He, Xuan, Zeng, Li, Lin, Yi, Li, Entong, Zheng, Binfan, Zhao, Rongqian, Chen, Xin

The Mixtures-of-Experts (MoE) model is a widespread distributed and integrated learning method for large language models (LLM), which is favored due to its ability to sparsify and expand models efficiently. However, the performance of MoE is limited

Externí odkaz: http://arxiv.org/abs/2401.13920

Zobrazit plný text záznamu

Report

RaftGP: Random Fast Graph Partitioning

Autor: Gao, Yu, Qin, Meng, Ding, Yibin, Zeng, Li, Zhang, Chaorui, Zhang, Weixi, Han, Wei, Zhao, Rongqian, Bai, Bo

Graph partitioning (GP), a.k.a. community detection, is a classic problem that divides the node set of a graph into densely-connected blocks. Following prior work on the IEEE HPEC Graph Challenge benchmark and recent advances in graph machine learnin

Externí odkaz: http://arxiv.org/abs/2312.01560

Zobrazit plný text záznamu