Multilingual Large Language Models Are Not (Yet) Code-Switchers

Autor:	Zhang, Ruochen, Cahyawijaya, Samuel, Cruz, Jan Christian Blaise, Winata, Genta Indra, Aji, Alham Fikri
Rok vydání:	2023
Předmět:	Computer Science - Computation and Language Computer Science - Artificial Intelligence
Druh dokumentu:	Working Paper
Popis:	Multilingual Large Language Models (LLMs) have recently shown great capabilities in a wide range of tasks, exhibiting state-of-the-art performance through zero-shot or few-shot prompting methods. While there have been extensive studies on their abilities in monolingual tasks, the investigation of their potential in the context of code-switching (CSW), the practice of alternating languages within an utterance, remains relatively uncharted. In this paper, we provide a comprehensive empirical analysis of various multilingual LLMs, benchmarking their performance across four tasks: sentiment analysis, machine translation, summarization and word-level language identification. Our results indicate that despite multilingual LLMs exhibiting promising outcomes in certain tasks using zero or few-shot prompting, they still underperform in comparison to fine-tuned models of much smaller scales. We argue that current "multilingualism" in LLMs does not inherently imply proficiency with code-switching texts, calling for future research to bridge this discrepancy. Comment: Accepted at EMNLP 2023
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2305.14235 Zobrazit plný text záznamu View this record from Arxiv