Výsledky vyhledávání - "Beaumont, Olivier"

Report

Rockmate: an Efficient, Fast, Automatic and Generic Tool for Re-materialization in PyTorch

Autor: Zhao, Xunyi, Hellard, Théotime Le, Eyraud, Lionel, Gusak, Julia, Beaumont, Olivier

We propose Rockmate to control the memory requirements when training PyTorch DNN models. Rockmate is an automatic tool that starts from the model code and generates an equivalent model, using a predefined amount of memory for activations, at the cost

Externí odkaz: http://arxiv.org/abs/2307.01236

Zobrazit plný text záznamu

Report

Survey on Large Scale Neural Network Training

Autor: Gusak, Julia, Cherniuk, Daria, Shilova, Alena, Katrutsa, Alexander, Bershatsky, Daniel, Zhao, Xunyi, Eyraud-Dubois, Lionel, Shlyazhko, Oleg, Dimitrov, Denis, Oseledets, Ivan, Beaumont, Olivier

Modern Deep Neural Networks (DNNs) require significant memory to store weight, activations, and other intermediate tensors during training. Hence, many models do not fit one GPU device or can be trained using only a small per-GPU batch size. This sur

Externí odkaz: http://arxiv.org/abs/2202.10435

Zobrazit plný text záznamu

Report

I/O-Optimal Algorithms for Symmetric Linear Algebra Kernels

Autor: Beaumont, Olivier, Eyraud-Dubois, Lionel, Vérité, Mathieu, Langou, Julien

In this paper, we consider two fundamental symmetric kernels in linear algebra: the Cholesky factorization and the symmetric rank-$k$ update (SYRK), with the classical three nested loops algorithms for these kernels. In addition, we consider a machin

Externí odkaz: http://arxiv.org/abs/2202.10217

Zobrazit plný text záznamu

Report

Geometric Deep Reinforcement Learning for Dynamic DAG Scheduling

Autor: Grinsztajn, Nathan, Beaumont, Olivier, Jeannot, Emmanuel, Preux, Philippe

In practice, it is quite common to face combinatorial optimization problems which contain uncertainty along with non-determinism and dynamicity. These three properties call for appropriate algorithms; reinforcement learning (RL) is dealing with them

Externí odkaz: http://arxiv.org/abs/2011.04333

Zobrazit plný text záznamu

Report

Optimal checkpointing for heterogeneous chains: how to train deep neural networks with limited memory

Autor: Herrmann, Julien, Beaumont, Olivier, Eyraud-Dubois, Lionel, Hermann, Julien, Joly, Alexis, Shilova, Alena

This paper introduces a new activation checkpointing method which allows to significantly decrease memory usage when training Deep Neural Networks with the back-propagation algorithm. Similarly to checkpoint-ing techniques coming from the literature

Externí odkaz: http://arxiv.org/abs/1911.13214

Zobrazit plný text záznamu

Report

Scheduling on Two Types of Resources: a Survey

Autor: Beaumont, Olivier, Canon, Louis-claude, Eyraud-Dubois, Lionel, Lucarelli, Giorgio, Marchal, Loris, Mommessin, Clément, Simon, Bertrand, Trystram, Denis

Publikováno v: ACM Computing Survey, Vol. 53, No. 3, 2020

The evolution in the design of modern parallel platforms leads to revisit the scheduling jobs on distributed heterogeneous resources. The goal of this survey is to present the main existing algorithms, to classify them based on their underlying princ

Externí odkaz: http://arxiv.org/abs/1909.11365

Zobrazit plný text záznamu

Report

Training on the Edge: The why and the how

Autor: Kukreja, Navjot, Shilova, Alena, Beaumont, Olivier, Huckelheim, Jan, Ferrier, Nicola, Hovland, Paul, Gorman, Gerard

Edge computing is the natural progression from Cloud computing, where, instead of collecting all data and processing it centrally, like in a cloud computing environment, we distribute the computing power and try to do as much processing as possible,

Externí odkaz: http://arxiv.org/abs/1903.03051

Zobrazit plný text záznamu