How Good Is Crude MDL for Solving the Bias-Variance Dilemma? An Empirical Investigation Based on Bayesian Networks
| Autor: | Elva Maria Novoa-Del-Toro, Alejandro Guerra-Hernández, Luis Alonso Nava-Fernández, Karina Gutiérrez-Fragoso, Héctor-Gabriel Acosta-Mesa, Efrén Mezura-Montes, Patricia González-Gaspar, Nicandro Cruz-Ramírez, Guillermo de Jesús Hoyos-Rivera, Vicente Josué Aguilera-Rueda, Maria-Yaneli Ameca-Alducin, Rocío Erandi Barrientos-Martínez |
|---|---|
| Rok vydání: | 2014 |
| Předmět: |
Computer and Information Sciences
lcsh:Medicine Overfitting Machine learning computer.software_genre Bioinformatics Bias Goodness of fit Bayesian information criterion Statistical Methods lcsh:Science Minimum description length Probability Physics Multidisciplinary business.industry Applied Mathematics Model selection lcsh:R Software Engineering Bayesian network Bayes Theorem Random Variables Variance (accounting) Probability Theory Probability Distribution Databases as Topic Physical Sciences Engineering and Technology lcsh:Q Artificial intelligence Akaike information criterion business computer Algorithms Mathematics Statistics (Mathematics) Research Article |
| Zdroj: | PLoS ONE PLoS ONE, Vol 9, Iss 3, p e92866 (2014) |
| ISSN: | 1932-6203 |
| DOI: | 10.1371/journal.pone.0092866 |
| Popis: | The bias-variance dilemma is a well-known and important problem in Machine Learning. It basically relates the generalization capability (goodness of fit) of a learning method to its corresponding complexity. When we have enough data at hand, it is possible to use these data in such a way so as to minimize overfitting (the risk of selecting a complex model that generalizes poorly). Unfortunately, there are many situations where we simply do not have this required amount of data. Thus, we need to find methods capable of efficiently exploiting the available data while avoiding overfitting. Different metrics have been proposed to achieve this goal: the Minimum Description Length principle (MDL), Akaike's Information Criterion (AIC) and Bayesian Information Criterion (BIC), among others. In this paper, we focus on crude MDL and empirically evaluate its performance in selecting models with a good balance between goodness of fit and complexity: the so-called bias-variance dilemma, decomposition or tradeoff. Although the graphical interaction between these dimensions (bias and variance) is ubiquitous in the Machine Learning literature, few works present experimental evidence to recover such interaction. In our experiments, we argue that the resulting graphs allow us to gain insights that are difficult to unveil otherwise: that crude MDL naturally selects balanced models in terms of bias-variance, which not necessarily need be the gold-standard ones. We carry out these experiments using a specific model: a Bayesian network. In spite of these motivating results, we also should not overlook three other components that may significantly affect the final model selection: the search procedure, the noise rate and the sample size. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|