Text data augmentation and pre-trained Language Model for enhancing text classification of low-resource languages.
| Autor: | Ziyaden A; Kazakh-British Technical University, Almaty, Kazakhstan.; Institute of Information and Computational Technologies, Almaty, Kazakhstan., Yelenov A; Institute of Information and Computational Technologies, Almaty, Kazakhstan.; Nazarbayev University, Astana, Kazakhstan., Hajiyev F; School of Information Technologies and Engineering, ADA University, Baku, Azerbaijan., Rustamov S; School of Information Technologies and Engineering, ADA University, Baku, Azerbaijan., Pak A; Kazakh-British Technical University, Almaty, Kazakhstan.; Institute of Information and Computational Technologies, Almaty, Kazakhstan. |
|---|---|
| Jazyk: | angličtina |
| Zdroj: | PeerJ. Computer science [PeerJ Comput Sci] 2024 Mar 29; Vol. 10, pp. e1974. Date of Electronic Publication: 2024 Mar 29 (Print Publication: 2024). |
| DOI: | 10.7717/peerj-cs.1974 |
| Abstrakt: | Background: In the domain of natural language processing (NLP), the development and success of advanced language models are predominantly anchored in the richness of available linguistic resources. Languages such as Azerbaijani, which is classified as a low-resource, often face challenges arising from limited labeled datasets, consequently hindering effective model training. Methodology: The primary objective of this study was to enhance the effectiveness and generalization capabilities of news text classification models using text augmentation techniques. In this study, we solve the problem of working with low-resource languages using translations using the Facebook mBart50 model, as well as the Google Translate API and a combination of mBart50 and Google Translate thus expanding the capabilities when working with text. Results: The experimental outcomes reveal a promising uptick in classification performance when models are trained on the augmented dataset compared with their counterparts using the original data. This investigation underscores the immense potential of combined data augmentation strategies to bolster the NLP capabilities of underrepresented languages. As a result of our research, we have published our labeled text classification dataset and pre-trained RoBERTa model for the Azerbaijani language. Competing Interests: The authors declare that they have no competing interests. (© 2024 Ziyaden et al.) |
| Databáze: | MEDLINE |
| Externí odkaz: |
|