Building for Tomorrow: Assessing the Temporal Persistence of Text Classifiers
| Autor: | Rabab Alkhalifa, Elena Kochkina, Arkaitz Zubiaga |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2022 |
| Předmět: |
FOS: Computer and information sciences
Artificial Intelligence (cs.AI) Computer Science - Computation and Language Computer Science - Artificial Intelligence Media Technology Library and Information Sciences Management Science and Operations Research Computation and Language (cs.CL) Computer Science Applications Information Systems |
| Popis: | Performance of text classification models tends to drop over time due to changes in data, which limits the lifetime of a pretrained model. Therefore an ability to predict a model's ability to persist over time can help design models that can be effectively used over a longer period of time. In this paper, we provide a thorough discussion into the problem, establish an evaluation setup for the task. We look at this problem from a practical perspective by assessing the ability of a wide range of language models and classification algorithms to persist over time, as well as how dataset characteristics can help predict the temporal stability of different models. We perform longitudinal classification experiments on three datasets spanning between 6 and 19 years, and involving diverse tasks and types of data. By splitting the longitudinal datasets into years, we perform a comprehensive set of experiments by training and testing across data that are different numbers of years apart from each other, both in the past and in the future. This enables a gradual investigation into the impact of the temporal gap between training and test sets on the classification performance, as well as measuring the extent of the persistence over time. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|