| Autor: |
Tula, Debapriya, Shreyas, M. S., Reddy, Viswanatha, Sahu, Pranjal, Doddapaneni, Sumanth, Potluri, Prathyush, Sukumaran, Rohan, Patwa, Parth |
| Zdroj: |
SN Computer Science; September 2022, Vol. 3 Issue: 5 |
| Abstrakt: |
Over the past decade, we have seen exponential growth in online content fueled by social media platforms. Data generation of this scale comes with the caveat of insurmountable offensive content in it. The complexity of identifying offensive content is exacerbated by the usage of multiple modalities (image, language, etc.), code mixed language and more. Moreover, even after careful sampling and annotation of offensive content, there will always exist a significant class imbalance between offensive and non-offensive content. In this paper, we introduce a novel code-mixing index (CMI) based focal loss which circumvents two challenges (1) code-mixing in languages (2) class imbalance problem for Dravidian language offence detection. We also replace the conventional dot product-based classifier with the cosine-based classifier which results in a boost in performance. Further, we use multilingual models that help transfer characteristics learnt across languages to work effectively with low-resourced languages. It is also important to note that our model handles instances of mixed script (say usage of Latinand Dravidian—Tamilscript) as well. To summarize, our model can handle offensive language detection in a low-resource, class imbalanced, multilingual and code mixed setting. The code is publicly available at https://github.com/Debapriya-Tula/EACL2021-DravidianTask-Bitions. |
| Databáze: |
Supplemental Index |
| Externí odkaz: |
|
Full text from SpringerLink