AutoLife: Automatic Life Journaling with Smartphones and LLMs

Autor:	Xu, Huatao, Tong, Panron, Li, Mo, Srivastava, Mani
Rok vydání:	2024
Předmět:	Computer Science - Artificial Intelligence Computer Science - Computation and Language Computer Science - Human-Computer Interaction
Druh dokumentu:	Working Paper
Popis:	This paper introduces a novel mobile sensing application - life journaling - designed to generate semantic descriptions of users' daily lives. We present AutoLife, an automatic life journaling system based on commercial smartphones. AutoLife only inputs low-cost sensor data (without photos or audio) from smartphones and can automatically generate comprehensive life journals for users. To achieve this, we first derive time, motion, and location contexts from multimodal sensor data, and harness the zero-shot capabilities of Large Language Models (LLMs), enriched with commonsense knowledge about human lives, to interpret diverse contexts and generate life journals. To manage the task complexity and long sensing duration, a multilayer framework is proposed, which decomposes tasks and seamlessly integrates LLMs with other techniques for life journaling. This study establishes a real-life dataset as a benchmark and extensive experiment results demonstrate that AutoLife produces accurate and reliable life journals. Comment: 13 pages
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2412.15714 Zobrazit plný text záznamu View this record from Arxiv