LLM as an Art Director (LaDi): Using LLMs to improve Text-to-Media Generators

Autor:	Roush, Allen, Zakirov, Emil, Shirokov, Artemiy, Lunina, Polina, Gane, Jack, Duffy, Alexander, Basil, Charlie, Whitcomb, Aber, Benedetto, Jim, DeWolfe, Chris
Rok vydání:	2023
Předmět:	Computer Science - Computation and Language Computer Science - Artificial Intelligence Computer Science - Computer Vision and Pattern Recognition
Druh dokumentu:	Working Paper
Popis:	Recent advancements in text-to-image generation have revolutionized numerous fields, including art and cinema, by automating the generation of high-quality, context-aware images and video. However, the utility of these technologies is often limited by the inadequacy of text prompts in guiding the generator to produce artistically coherent and subject-relevant images. In this paper, We describe the techniques that can be used to make Large Language Models (LLMs) act as Art Directors that enhance image and video generation. We describe our unified system for this called "LaDi". We explore how LaDi integrates multiple techniques for augmenting the capabilities of text-to-image generators (T2Is) and text-to-video generators (T2Vs), with a focus on constrained decoding, intelligent prompting, fine-tuning, and retrieval. LaDi and these techniques are being used today in apps and platforms developed by Plai Labs. Comment: 12 pages, System Demonstration/Industry paper. Preprint
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2311.03716 Zobrazit plný text záznamu View this record from Arxiv