Learning from multiple annotators using kernel alignment
Autor: | Álvaro-Ángel Orozco-Gutierrez, J. Gil-Gonzalez, Andrés Marino Álvarez-Meza |
---|---|
Rok vydání: | 2018 |
Předmět: |
Ground truth
Training set Computer science business.industry Supervised learning 02 engineering and technology Machine learning computer.software_genre Oracle Kernel alignment Kernel method Artificial Intelligence 020204 information systems Signal Processing 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Convex combination Computer Vision and Pattern Recognition Artificial intelligence business computer Software |
Zdroj: | Pattern Recognition Letters. 116:150-156 |
ISSN: | 0167-8655 |
Popis: | In a typical supervised learning scenario, it is supposed that there is an oracle who gives the correct label (also known as gold standard or ground truth) for each instance available in the training set. Nevertheless, for many real-world problems, instead of the gold standard, we have access to some annotations (possibly noisy) provided by multiple annotators with different unknown levels of expertise. Then, it is not appropriate to use trivial methods, i.e., majority voting, to estimate the actual label from the annotations due to this way assumes homogeneity in the performance of the labelers. Here, we introduce a new kernel alignment-based annotator relevance analysis–(KAAR) approach to code each annotator expertise as an averaged matching between the input features and the expert labels. So, a new sample label is predicted as a convex combination of classifiers adopting the achieved KAAR-based coding. Experimental results show that our methodology can estimate the performance of annotators even if the gold standard is not available, defeating state-of-the-art techniques. |
Databáze: | OpenAIRE |
Externí odkaz: |