Výsledky vyhledávání - "Liu Xuandong"

Report

Rina: Enhancing Ring-AllReduce with In-network Aggregation in Distributed Model Training

Autor: Chen, Zixuan, Liu, Xuandong, Li, Minglin, Hu, Yinfan, Mei, Hao, Xing, Huifeng, Wang, Hao, Shi, Wanxin, Liu, Sen, Xu, Yang

Parameter Server (PS) and Ring-AllReduce (RAR) are two widely utilized synchronization architectures in multi-worker Deep Learning (DL), also referred to as Distributed Deep Learning (DDL). However, PS encounters challenges with the ``incast'' issue,

Externí odkaz: http://arxiv.org/abs/2407.19721

Zobrazit plný text záznamu

Report

OSP: Boosting Distributed Model Training with 2-stage Synchronization

Autor: Chen, Zixuan, Shi, Lei, Liu, Xuandong, Li, Jiahui, Liu, Sen, Xu, Yang

Distributed deep learning (DDL) is a promising research area, which aims to increase the efficiency of training deep learning tasks with large size of datasets and models. As the computation capability of DDL nodes continues to increase, the network

Externí odkaz: http://arxiv.org/abs/2306.16926

Zobrazit plný text záznamu

Report

Boosting Distributed Machine Learning Training Through Loss-tolerant Transmission Protocol

Autor: Chen, Zixuan, Shi, Lei, Liu, Xuandong, Ai, Xin, Liu, Sen, Xu, Yang

Distributed Machine Learning (DML) systems are utilized to enhance the speed of model training in data centers (DCs) and edge nodes. The Parameter Server (PS) communication architecture is commonly employed, but it faces severe long-tail latency caus

Externí odkaz: http://arxiv.org/abs/2305.04279

Zobrazit plný text záznamu