POLITICS: Pretraining with Same-story Article Comparison for Ideology Prediction and Stance Detection

Autor:	Liu, Yujian, Zhang, Xinliang Frederick, Wegsman, David, Beauchamp, Nick, Wang, Lu
Rok vydání:	2022
Předmět:	Computer Science - Computation and Language
Druh dokumentu:	Working Paper
Popis:	Ideology is at the core of political science research. Yet, there still does not exist general-purpose tools to characterize and predict ideology across different genres of text. To this end, we study Pretrained Language Models using novel ideology-driven pretraining objectives that rely on the comparison of articles on the same story written by media of different ideologies. We further collect a large-scale dataset, consisting of more than 3.6M political news articles, for pretraining. Our model POLITICS outperforms strong baselines and the previous state-of-the-art models on ideology prediction and stance detection tasks. Further analyses show that POLITICS is especially good at understanding long or formally written texts, and is also robust in few-shot learning scenarios. Comment: Findings of NAACL'22. The first two authors contribute equally
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2205.00619 Zobrazit plný text záznamu View this record from Arxiv