A Blueprint Architecture of Compound AI Systems for Enterprise

Autor:	Kandogan, Eser, Rahman, Sajjadur, Bhutani, Nikita, Zhang, Dan, Chen, Rafael Li, Mitra, Kushan, Gurajada, Sairam, Pezeshkpour, Pouya, Iso, Hayate, Feng, Yanlin, Kim, Hannah, Shen, Chen, Wang, Jin, Hruschka, Estevam
Rok vydání:	2024
Předmět:	Computer Science - Databases Computer Science - Artificial Intelligence
Druh dokumentu:	Working Paper
Popis:	Large Language Models (LLMs) have showcased remarkable capabilities surpassing conventional NLP challenges, creating opportunities for use in production use cases. Towards this goal, there is a notable shift to building compound AI systems, wherein LLMs are integrated into an expansive software infrastructure with many components like models, retrievers, databases and tools. In this paper, we introduce a blueprint architecture for compound AI systems to operate in enterprise settings cost-effectively and feasibly. Our proposed architecture aims for seamless integration with existing compute and data infrastructure, with ``stream'' serving as the key orchestration concept to coordinate data and instructions among agents and other components. Task and data planners, respectively, break down, map, and optimize tasks and data to available agents and data sources defined in respective registries, given production constraints such as accuracy and latency. Comment: Compound AI Systems Workshop at the Data+AI Summit 2024
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2406.00584 Zobrazit plný text záznamu View this record from Arxiv