Zobrazeno 1 - 10
of 169
pro vyhledávání: '"Sridharan, Srinivas"'
Autor:
Yoo, Jinsun, Won, William, Cowan, Meghan, Jiang, Nan, Klenk, Benjamin, Sridharan, Srinivas, Krishna, Tushar
The explosion of machine learning model size has led to its execution on distributed clusters at a very large scale. Many works have tried to optimize the process of producing collective algorithms and running collective communications, which act as
Externí odkaz:
http://arxiv.org/abs/2408.11008
Autor:
Sridharan, Srinivas, Heo, Taekyung, Feng, Louis, Wang, Zhaodong, Bergeron, Matt, Fu, Wenyin, Zheng, Shengbao, Coutinho, Brian, Rashidi, Saeed, Man, Changhai, Krishna, Tushar
Benchmarking and co-design are essential for driving optimizations and innovation around ML models, ML software, and next-generation hardware. Full workload benchmarks, e.g. MLPerf, play an essential role in enabling fair comparison across different
Externí odkaz:
http://arxiv.org/abs/2305.14516
Autor:
Won, William, Heo, Taekyung, Rashidi, Saeed, Sridharan, Srinivas, Srinivasan, Sudarshan, Krishna, Tushar
As deep learning models and input data are scaling at an unprecedented rate, it is inevitable to move towards distributed training platforms to fit the model and increase training throughput. State-of-the-art approaches and techniques, such as wafer-
Externí odkaz:
http://arxiv.org/abs/2303.14006
Autor:
Liang, Mingyu, Fu, Wenyin, Feng, Louis, Lin, Zhongyi, Panakanti, Pavani, Zheng, Shengbao, Sridharan, Srinivas, Delimitrou, Christina
Building large AI fleets to support the rapidly growing DL workloads is an active research topic for modern cloud providers. Generating accurate benchmarks plays an essential role in designing the fast-paced software and hardware solutions in this sp
Externí odkaz:
http://arxiv.org/abs/2301.04122
Autor:
Khan, Tarannum, Rashidi, Saeed, Sridharan, Srinivas, Shurpali, Pallavi, Akella, Aditya, Krishna, Tushar
RDMA over Converged Ethernet (RoCE) has gained significant attraction for datacenter networks due to its compatibility with conventional Ethernet-based fabric. However, the RDMA protocol is efficient only on (nearly) lossless networks, emphasizing th
Externí odkaz:
http://arxiv.org/abs/2207.10898
Themis: A Network Bandwidth-Aware Collective Scheduling Policy for Distributed Training of DL Models
Distributed training is a solution to reduce DNN training time by splitting the task across multiple NPUs (e.g., GPU/TPU). However, distributed training adds communication overhead between the NPUs in order to synchronize the gradients and/or activat
Externí odkaz:
http://arxiv.org/abs/2110.04478
Autor:
Mudigere, Dheevatsa, Hao, Yuchen, Huang, Jianyu, Jia, Zhihao, Tulloch, Andrew, Sridharan, Srinivas, Liu, Xing, Ozdal, Mustafa, Nie, Jade, Park, Jongsoo, Luo, Liang, Yang, Jie Amy, Gao, Leon, Ivchenko, Dmytro, Basant, Aarti, Hu, Yuxi, Yang, Jiyan, Ardestani, Ehsan K., Wang, Xiaodong, Komuravelli, Rakesh, Chu, Ching-Hsiang, Yilmaz, Serhat, Li, Huayu, Qian, Jiyuan, Feng, Zhuobo, Ma, Yinbin, Yang, Junjie, Wen, Ellie, Li, Hong, Yang, Lin, Sun, Chonglin, Zhao, Whitney, Melts, Dimitry, Dhulipala, Krishna, Kishore, KR, Graf, Tyler, Eisenman, Assaf, Matam, Kiran Kumar, Gangidi, Adi, Chen, Guoqiang Jerry, Krishnan, Manoj, Nayak, Avinash, Nair, Krishnakumar, Muthiah, Bharath, khorashadi, Mahmoud, Bhattacharya, Pallab, Lapukhov, Petr, Naumov, Maxim, Mathews, Ajit, Qiao, Lin, Smelyanskiy, Mikhail, Jia, Bill, Rao, Vijay
Deep learning recommendation models (DLRMs) are used across many business-critical services at Facebook and are the single largest AI application in terms of infrastructure demand in its data-centers. In this paper we discuss the SW/HW co-designed so
Externí odkaz:
http://arxiv.org/abs/2104.05158
Autor:
Rashidi, Saeed, Denton, Matthew, Sridharan, Srinivas, Srinivasan, Sudarshan, Suresh, Amoghavarsha, Ni, Jade, Krishna, Tushar
Deep Learning (DL) training platforms are built by interconnecting multiple DL accelerators (e.g., GPU/TPU) via fast, customized interconnects with 100s of gigabytes (GBs) of bandwidth. However, as we identify in this work, driving this bandwidth is
Externí odkaz:
http://arxiv.org/abs/2007.00156
Autor:
Naumov, Maxim, Kim, John, Mudigere, Dheevatsa, Sridharan, Srinivas, Wang, Xiaodong, Zhao, Whitney, Yilmaz, Serhat, Kim, Changkyu, Yuen, Hector, Ozdal, Mustafa, Nair, Krishnakumar, Gao, Isabel, Su, Bor-Yiing, Yang, Jiyan, Smelyanskiy, Mikhail
Large-scale training is important to ensure high performance and accuracy of machine-learning models. At Facebook we use many different models, including computer vision, video and language models. However, in this paper we focus on the deep learning
Externí odkaz:
http://arxiv.org/abs/2003.09518
The deep neural networks (DNNs) have been enormously successful in tasks that were hitherto in the human-only realm such as image recognition, and language translation. Owing to their success the DNNs are being explored for use in ever more sophistic
Externí odkaz:
http://arxiv.org/abs/1906.08168