Zobrazeno 1 - 2
of 2
pro vyhledávání: '"Tyukin, Georgy"'
The inference demand for LLMs has skyrocketed in recent months, and serving models with low latencies remains challenging due to the quadratic input length complexity of the attention layers. In this work, we investigate the effect of dropping MLP an
Externí odkaz:
http://arxiv.org/abs/2407.15516
Autor:
Tyukin, Georgy
Large Language Models are growing in size, and we expect them to continue to do so, as larger models train quicker. However, this increase in size will severely impact inference costs. Therefore model compression is important, to retain the performan
Externí odkaz:
http://arxiv.org/abs/2404.05741