Zobrazeno 1 - 2
of 2
pro vyhledávání: '"Pathria, Ribhu"'
Autor:
Dey, Nolan, Soboleva, Daria, Al-Khateeb, Faisal, Yang, Bowen, Pathria, Ribhu, Khachane, Hemant, Muhammad, Shaheer, Zhiming, Chen, Myers, Robert, Steeves, Jacob Robert, Vassilieva, Natalia, Tom, Marvin, Hestness, Joel
We introduce the Bittensor Language Model, called "BTLM-3B-8K", a new state-of-the-art 3 billion parameter open-source language model. BTLM-3B-8K was trained on 627B tokens from the SlimPajama dataset with a mixture of 2,048 and 8,192 context lengths
Externí odkaz:
http://arxiv.org/abs/2309.11568
Autor:
Dey, Nolan, Gosal, Gurpreet, Zhiming, Chen, Khachane, Hemant, Marshall, William, Pathria, Ribhu, Tom, Marvin, Hestness, Joel
We study recent research advances that improve large language models through efficient pre-training and scaling, and open datasets and tools. We combine these advances to introduce Cerebras-GPT, a family of open compute-optimal language models scaled
Externí odkaz:
http://arxiv.org/abs/2304.03208