A Comparative Study of Traditional, Ensemble and Neural Network-Based Natural Language Processing Algorithms

Autor: Essen, Achraf Chikhi, Seyed Sahand Mohammadi Ziabari, Jan-Willem van
Jazyk: angličtina
Rok vydání: 2023
Předmět:
Zdroj: Journal of Risk and Financial Management; Volume 16; Issue 7; Pages: 327
ISSN: 1911-8074
DOI: 10.3390/jrfm16070327
Popis: Accurate data analysis is an important part of data-driven financial audits. Given the increased data availability and various systems from which audit files are generated, RCSFI provides a way for standardization on behalf of analysis. This research attempted to automate this hierarchical text classification task in order to save financial auditors time and avoid errors. Several studies have shown that ensemble-based models and neural-network-based natural language processing (NLP) techniques achieved encouraging results for classification problems in various domains. However, there has been limited empirical research comparing the performance of both of the aforementioned techniques in a hierarchical multi-class classification setting. Moreover, neural-network- based NLP techniques have commonly been applied to English datasets and not to Dutch financial datasets. Additionally, this research took the implementation of hierarchical approaches into account for the traditional and ensemble-based models and found that the performance did not increase when implementing the included hierarchical approaches. DistilBERT achieved the highest scores on level 1-2-3-4 and outperformed the traditional and ensemble-based models. The model obtained a F1 of 94.50% for level 1-2-3-4. DistilBERT also outperformed BERTje at level 1-2-3-4 despite BERTje being specifically pre-trained on Dutch datasets.
Databáze: OpenAIRE