A Comparison of AutoML Tools for Machine Learning, Deep Learning and XGBoost
| Autor: | Luís Ferreira, Paulo Cortez, Carlos Martins, Pedro Miguel Pires, André Luiz Pilastri |
|---|---|
| Přispěvatelé: | Universidade do Minho |
| Rok vydání: | 2021 |
| Předmět: |
Computer science
02 engineering and technology Indústria inovação e infraestruturas Machine learning computer.software_genre Software 0202 electrical engineering electronic engineering information engineering Science & Technology Neural Architecture Search (NAS) business.industry Deep learning Supervised learning Automated Machine Learning (AutoML) Automated Deep Learning (AutoDL) Ciências Naturais::Ciências da Computação e da Informação Benchmarking Classification Lexicographical order Regression Task (computing) Benchmark (computing) 020201 artificial intelligence & image processing Artificial intelligence Supervised Learning business computer |
| Zdroj: | IJCNN |
| Popis: | This paper presents a benchmark of supervised Automated Machine Learning (AutoML) tools. Firstly, we an- alyze the characteristics of eight recent open-source AutoML tools (Auto-Keras, Auto-PyTorch, Auto-Sklearn, AutoGluon, H2O AutoML, rminer, TPOT and TransmogrifAI) and describe twelve popular OpenML datasets that were used in the benchmark (divided into regression, binary and multi-class classification tasks). Then, we perform a comparison study with hundreds of computational experiments based on three scenarios: General Machine Learning (GML), Deep Learning (DL) and XGBoost (XGB). To select the best tool, we used a lexicographic approach, considering first the average prediction score for each task and then the computational effort. The best predictive results were achieved for GML, which were further compared with the best OpenML public results. Overall, the best GML AutoML tools obtained competitive results, outperforming the best OpenML models in five datasets. These results confirm the potential of the general-purpose AutoML tools to fully automate the Machine Learning (ML) algorithm selection and tuning. Opti-Edge: 5G Digital Services Optimization at the Edge, Individual Project, NUP: POCI-01-0247-FEDER-045220, co-funded by the Incentive System for Research and Technological Development, from the Thematic Operational Program Competitiveness of the national framework program - Portugal2020 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|