L3iTC at the FinLLM Challenge Task: Quantization for Financial Text Classification & Summarization

Autor:	Pontes, Elvys Linhares, González-Gallardo, Carlos-Emiliano, Benjannet, Mohamed, Qu, Caryn, Doucet, Antoine
Rok vydání:	2024
Předmět:	Computer Science - Computation and Language Computer Science - Computational Engineering Finance and Science
Druh dokumentu:	Working Paper
Popis:	This article details our participation (L3iTC) in the FinLLM Challenge Task 2024, focusing on two key areas: Task 1, financial text classification, and Task 2, financial text summarization. To address these challenges, we fine-tuned several large language models (LLMs) to optimize performance for each task. Specifically, we used 4-bit quantization and LoRA to determine which layers of the LLMs should be trained at a lower precision. This approach not only accelerated the fine-tuning process on the training data provided by the organizers but also enabled us to run the models on low GPU memory. Our fine-tuned models achieved third place for the financial classification task with an F1-score of 0.7543 and secured sixth place in the financial summarization task on the official test datasets. Comment: Joint Workshop of the 8th Financial Technology and Natural Language Processing (FinNLP) and the 1st Agent AI for Scenario Planning (AgentScen), 2024
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2408.03033 Zobrazit plný text záznamu View this record from Arxiv