GPT is an effective tool for multilingual psychological text analysis.

Autor:	Rathje S; Department of Psychology, New York University, New York, NY 10003., Mirea DM; Department of Psychology, Princeton University, Princeton, NJ 08540., Sucholutsky I; Department of Computer Science, Princeton University, Princeton, NJ 08540., Marjieh R; Department of Psychology, Princeton University, Princeton, NJ 08540., Robertson CE; Department of Psychology, New York University, New York, NY 10003., Van Bavel JJ; Department of Psychology, New York University, New York, NY 10003.; Center for Neural Science, New York University, New York, NY 10003.; Department of Strategy and Management, Norwegian School of Economics, Bergen 5045, Norway.
Jazyk:	angličtina
Zdroj:	Proceedings of the National Academy of Sciences of the United States of America [Proc Natl Acad Sci U S A] 2024 Aug 20; Vol. 121 (34), pp. e2308950121. Date of Electronic Publication: 2024 Aug 12.
DOI:	10.1073/pnas.2308950121
Abstrakt:	The social and behavioral sciences have been increasingly using automated text analysis to measure psychological constructs in text. We explore whether GPT, the large-language model (LLM) underlying the AI chatbot ChatGPT, can be used as a tool for automated psychological text analysis in several languages. Across 15 datasets ( n = 47,925 manually annotated tweets and news headlines), we tested whether different versions of GPT (3.5 Turbo, 4, and 4 Turbo) can accurately detect psychological constructs (sentiment, discrete emotions, offensiveness, and moral foundations) across 12 languages. We found that GPT ( r = 0.59 to 0.77) performed much better than English-language dictionary analysis ( r = 0.20 to 0.30) at detecting psychological constructs as judged by manual annotators. GPT performed nearly as well as, and sometimes better than, several top-performing fine-tuned machine learning models. Moreover, GPT's performance improved across successive versions of the model, particularly for lesser-spoken languages, and became less expensive. Overall, GPT may be superior to many existing methods of automated text analysis, since it achieves relatively high accuracy across many languages, requires no training data, and is easy to use with simple prompts (e.g., "is this text negative?") and little coding experience. We provide sample code and a video tutorial for analyzing text with the GPT application programming interface. We argue that GPT and other LLMs help democratize automated text analysis by making advanced natural language processing capabilities more accessible, and may help facilitate more cross-linguistic research with understudied languages. Competing Interests: Competing interests statement:The authors declare no competing interest.
Databáze:	MEDLINE
Externí odkaz:	Zobrazit plný text záznamu