Zobrazeno 1 - 10
of 733
pro vyhledávání: '"Zhou, TianYi"'
What makes a difference in the post-training of LLMs? We investigate the training patterns of different layers in large language models (LLMs), through the lens of gradient, when training with different responses and initial models. We are specifical
Externí odkaz:
http://arxiv.org/abs/2410.23743
Sharpness-Aware Minimization (SAM) has been demonstrated to improve the generalization performance of overparameterized models by seeking flat minima on the loss landscape through optimizing model parameters that incur the largest loss within a neigh
Externí odkaz:
http://arxiv.org/abs/2410.22656
Incremental learning (IL) aims to acquire new knowledge from current tasks while retaining knowledge learned from previous tasks. Replay-based IL methods store a set of exemplars from previous tasks in a buffer and replay them when learning new tasks
Externí odkaz:
http://arxiv.org/abs/2410.15372
Evaluating large language models (LLMs) is costly: it requires the generation and examination of LLM outputs on a large-scale benchmark of various tasks. This paper investigates how to efficiently reduce the tasks used to benchmark LLMs without affec
Externí odkaz:
http://arxiv.org/abs/2410.13804
Low-quality or scarce data has posed significant challenges for training deep neural networks in practice. While classical data augmentation cannot contribute very different new data, diffusion models opens up a new door to build self-evolving AI by
Externí odkaz:
http://arxiv.org/abs/2410.13674
Autor:
Chen, Lichang, Hu, Hexiang, Zhang, Mingda, Chen, Yiwen, Wang, Zifeng, Li, Yandong, Shyam, Pranav, Zhou, Tianyi, Huang, Heng, Yang, Ming-Hsuan, Gong, Boqing
We introduce OmnixR, an evaluation suite designed to benchmark SoTA Omni-modality Language Models, such as GPT-4o and Gemini. Evaluating OLMs, which integrate multiple modalities such as text, vision, and audio, presents unique challenges. Particular
Externí odkaz:
http://arxiv.org/abs/2410.12219
Autor:
Li, Ziyue, Zhou, Tianyi
While large language models (LLMs) excel on generation tasks, their decoder-only architecture often limits their potential as embedding models if no further representation finetuning is applied. Does this contradict their claim of generalists? To ans
Externí odkaz:
http://arxiv.org/abs/2410.10814
Autor:
Zhou, Siyu, Zhou, Tianyi, Yang, Yijun, Long, Guodong, Ye, Deheng, Jiang, Jing, Zhang, Chengqi
Can large language models (LLMs) directly serve as powerful world models for model-based agents? While the gaps between the prior knowledge of LLMs and the specified environment's dynamics do exist, our study reveals that the gaps can be bridged by a
Externí odkaz:
http://arxiv.org/abs/2410.07484
Recent advancements of large language models (LLMs) have led to claims of AI surpassing humans in natural language processing (NLP) tasks such as textual understanding and reasoning. This work investigates these assertions by introducing CAIMIRA, a n
Externí odkaz:
http://arxiv.org/abs/2410.06524
Autor:
Ding, Mucong, Deng, Chenghao, Choo, Jocelyn, Wu, Zichu, Agrawal, Aakriti, Schwarzschild, Avi, Zhou, Tianyi, Goldstein, Tom, Langford, John, Anandkumar, Anima, Huang, Furong
While generalization over tasks from easy to hard is crucial to profile language models (LLMs), the datasets with fine-grained difficulty annotations for each problem across a broad range of complexity are still blank. Aiming to address this limitati
Externí odkaz:
http://arxiv.org/abs/2409.18433