Advanced ingestion process powered by LLM parsing for RAG system

Autor:	Perez, Arnau, Vizcaino, Xavier
Rok vydání:	2024
Předmět:	Computer Science - Computation and Language Computer Science - Artificial Intelligence Computer Science - Information Retrieval
Druh dokumentu:	Working Paper
Popis:	Retrieval Augmented Generation (RAG) systems struggle with processing multimodal documents of varying structural complexity. This paper introduces a novel multi-strategy parsing approach using LLM-powered OCR to extract content from diverse document types, including presentations and high text density files both scanned or not. The methodology employs a node-based extraction technique that creates relationships between different information types and generates context-aware metadata. By implementing a Multimodal Assembler Agent and a flexible embedding strategy, the system enhances document comprehension and retrieval capabilities. Experimental evaluations across multiple knowledge bases demonstrate the approach's effectiveness, showing improvements in answer relevancy and information faithfulness. Comment: 12 pages, 3 figures
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2412.15262 Zobrazit plný text záznamu View this record from Arxiv