Zobrazeno 1 - 2
of 2
pro vyhledávání: '"Shamshoum, Yara"'
We introduce CompAct, a technique that reduces peak memory utilization on GPU by 25-30% for pretraining and 50% for fine-tuning of LLMs. Peak device memory is a major limiting factor in training LLMs, with various recent works aiming to reduce model
Externí odkaz:
http://arxiv.org/abs/2410.15352
Many recent works use machine learning models to solve various complex algorithmic problems. However, these models attempt to reach a solution without considering the problem's required computational complexity, which can be detrimental to their abil
Externí odkaz:
http://arxiv.org/abs/2406.02187