Semantic Approach to Quantifying the Consistency of Diffusion Model Image Generation

Autor:	Bent, Brinnae
Rok vydání:	2024
Předmět:	Computer Science - Computer Vision and Pattern Recognition Computer Science - Artificial Intelligence Computer Science - Human-Computer Interaction Computer Science - Machine Learning
Druh dokumentu:	Working Paper
Popis:	In this study, we identify the need for an interpretable, quantitative score of the repeatability, or consistency, of image generation in diffusion models. We propose a semantic approach, using a pairwise mean CLIP (Contrastive Language-Image Pretraining) score as our semantic consistency score. We applied this metric to compare two state-of-the-art open-source image generation diffusion models, Stable Diffusion XL and PixArt-{\alpha}, and we found statistically significant differences between the semantic consistency scores for the models. Agreement between the Semantic Consistency Score selected model and aggregated human annotations was 94%. We also explored the consistency of SDXL and a LoRA-fine-tuned version of SDXL and found that the fine-tuned model had significantly higher semantic consistency in generated images. The Semantic Consistency Score proposed here offers a measure of image generation alignment, facilitating the evaluation of model architectures for specific tasks and aiding in informed decision-making regarding model selection. Comment: Accepted to 2024 CVPR 3rd Explainable AI for Computer Vision (XAI4CV) Workshop
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2404.08799 Zobrazit plný text záznamu View this record from Arxiv