Simple Mechanisms for Representing, Indexing and Manipulating Concepts

Autor:	Li, Yuanzhi, Meka, Raghu, Panigrahy, Rina, Shah, Kulin
Rok vydání:	2023
Předmět:	Computer Science - Machine Learning Computer Science - Computation and Language Statistics - Machine Learning
Druh dokumentu:	Working Paper
Popis:	Deep networks typically learn concepts via classifiers, which involves setting up a model and training it via gradient descent to fit the concept-labeled data. We will argue instead that learning a concept could be done by looking at its moment statistics matrix to generate a concrete representation or signature of that concept. These signatures can be used to discover structure across the set of concepts and could recursively produce higher-level concepts by learning this structure from those signatures. When the concepts are `intersected', signatures of the concepts can be used to find a common theme across a number of related `intersected' concepts. This process could be used to keep a dictionary of concepts so that inputs could correctly identify and be routed to the set of concepts involved in the (latent) generation of the input. Comment: 19 pages
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2310.12143 Zobrazit plný text záznamu View this record from Arxiv