VARCO-VISION: Expanding Frontiers in Korean Vision-Language Models

Autor:	Ju, Jeongho, Kim, Daeyoung, Park, SunYoung, Kim, Youngjune
Rok vydání:	2024
Předmět:	Computer Science - Computer Vision and Pattern Recognition Computer Science - Computation and Language
Druh dokumentu:	Working Paper
Popis:	In this paper, we introduce an open-source Korean-English vision-language model (VLM), VARCO-VISION. We incorporate a step-by-step training strategy that allows a model learn both linguistic and visual information while preserving the backbone model's knowledge. Our model demonstrates outstanding performance in diverse settings requiring bilingual image-text understanding and generation abilities compared to models of similar size. VARCO-VISION is also capable of grounding, referring, and OCR, expanding its usage and potential applications for real-world scenarios. In addition to the model, we release five Korean evaluation datasets, including four closed-set and one openset benchmarks. We anticipate that our milestone will broaden the opportunities for AI researchers aiming to train VLMs. VARCO-VISION is available at https://huggingface.co/NCSOFT/VARCO-VISION-14B. Comment: 24 pages, 15 figures, 4 tables. Model weights at https://huggingface.co/NCSOFT/VARCO-VISION-14B. Benchmarks released at NCSOFT's HuggingFace repositories (K-MMBench, K-SEED, K-MMStar, K-DTCBench, K-LLaVA-W). VARCO-VISION is an open-source Korean-English VLM with OCR, grounding, and referring capabilities
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2411.19103 Zobrazit plný text záznamu View this record from Arxiv