Zobrazeno 1 - 2
of 2
pro vyhledávání: '"Chrzaszcz, Bart"'
Autor:
Alabed, Sami, Belov, Daniel, Chrzaszcz, Bart, Franco, Juliana, Grewe, Dominik, Maclaurin, Dougal, Molloy, James, Natan, Tom, Norman, Tamara, Pan, Xiaoyue, Paszke, Adam, Rink, Norman A., Schaarschmidt, Michael, Sitdikov, Timur, Swietlik, Agnieszka, Vytiniotis, Dimitrios, Wee, Joel
Training of modern large neural networks (NN) requires a combination of parallelization strategies encompassing data, model, or optimizer sharding. When strategies increase in complexity, it becomes necessary for partitioning tools to be 1) expressiv
Externí odkaz:
http://arxiv.org/abs/2401.11202
Autor:
Alabed, Sami, Grewe, Dominik, Franco, Juliana, Chrzaszcz, Bart, Natan, Tom, Norman, Tamara, Rink, Norman A., Vytiniotis, Dimitrios, Schaarschmidt, Michael
Large neural network models are commonly trained through a combination of advanced parallelism strategies in a single program, multiple data (SPMD) paradigm. For example, training large transformer models requires combining data, model, and pipeline
Externí odkaz:
http://arxiv.org/abs/2210.06352