Zobrazeno 1 - 10
of 157
pro vyhledávání: '"Shao, Yijia"'
While language model (LM)-powered chatbots and generative search engines excel at answering concrete queries, discovering information in the terrain of unknown unknowns remains challenging for users. To emulate the common educational scenario where c
Externí odkaz:
http://arxiv.org/abs/2408.15232
Language models are aligned to emulate the collective voice of many, resulting in outputs that align with no one in particular. Steering LLMs away from generic output is possible through supervised finetuning or RLHF, but requires prohibitively large
Externí odkaz:
http://arxiv.org/abs/2406.00888
When writing and talking, people sometimes pause to think. Although reasoning-focused works have often framed reasoning as a method of answering questions or completing agentic tasks, reasoning is implicit in almost all written text. For example, thi
Externí odkaz:
http://arxiv.org/abs/2403.09629
We study how to apply large language models to write grounded and organized long-form articles from scratch, with comparable breadth and depth to Wikipedia pages. This underexplored problem poses new challenges at the pre-writing stage, including how
Externí odkaz:
http://arxiv.org/abs/2402.14207
Publikováno v:
ICLR 2024
Class incremental learning (CIL) is a challenging setting of continual learning, which learns a series of tasks sequentially. Each task consists of a set of unique classes. The key feature of CIL is that no task identifier (or task-id) is provided at
Externí odkaz:
http://arxiv.org/abs/2309.15048
Despite the great success of pre-trained language models, it is still a challenge to use these models for continual learning, especially for the class-incremental learning (CIL) setting due to catastrophic forgetting (CF). This paper reports our find
Externí odkaz:
http://arxiv.org/abs/2306.12619
Commonsense reasoning is omnipresent in human communications and thus is an important feature for open-domain dialogue systems. However, evaluating commonsense in dialogue systems is still an open challenge. We take the first step by focusing on even
Externí odkaz:
http://arxiv.org/abs/2305.07797
Publikováno v:
ICLR 2023
Language models (LMs) have been instrumental for the rapid advance of natural language processing. This paper studies continual pre-training of LMs, in particular, continual domain-adaptive pre-training (or continual DAP-training). Existing research
Externí odkaz:
http://arxiv.org/abs/2302.03241
Domain-adaptive pre-training (or DA-training for short), also known as post-training, aims to train a pre-trained general-purpose language model (LM) using an unlabeled corpus of a particular domain to adapt the LM so that end-tasks in the domain can
Externí odkaz:
http://arxiv.org/abs/2301.08986
Transformers are widely used in NLP tasks. However, current approaches to leveraging transformers to understand language expose one weak spot: Number understanding. In some scenarios, numbers frequently occur, especially in semi-structured data like
Externí odkaz:
http://arxiv.org/abs/2212.02691