How to measure uncertainty in uncertainty sampling for active learning
| Autor: | Eyke Hüllermeier, Vu-Linh Nguyen, Mohammad Hossein Shaker |
|---|---|
| Přispěvatelé: | Uncertainty in Artificial Intelligence |
| Jazyk: | angličtina |
| Rok vydání: | 2022 |
| Předmět: |
Active learning
Epistemic uncertainty Uncertainty sampling business.industry Computer science Aleatoric uncertainty Probabilistic logic Credal uncertainty Sampling (statistics) 02 engineering and technology Machine learning computer.software_genre Measure (mathematics) Artificial Intelligence 020204 information systems 0202 electrical engineering electronic engineering information engineering Entropy (information theory) 020201 artificial intelligence & image processing Artificial intelligence Aleatoric music business computer Software |
| Zdroj: | Machine Learning, 111, 89-122. Springer |
| ISSN: | 0885-6125 |
| Popis: | Various strategies for active learning have been proposed in the machine learning literature. In uncertainty sampling, which is among the most popular approaches, the active learner sequentially queries the label of those instances for which its current prediction is maximally uncertain. The predictions as well as the measures used to quantify the degree of uncertainty, such as entropy, are traditionally of a probabilistic nature. Yet, alternative approaches to capturing uncertainty in machine learning, alongside with corresponding uncertainty measures, have been proposed in recent years. In particular, some of these measures seek to distinguish different sources and to separate different types of uncertainty, such as the reducible (epistemic) and the irreducible (aleatoric) part of the total uncertainty in a prediction. The goal of this paper is to elaborate on the usefulness of such measures for uncertainty sampling, and to compare their performance in active learning. To this end, we instantiate uncertainty sampling with different measures, analyze the properties of the sampling strategies thus obtained, and compare them in an experimental study. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
Full text from SpringerLink