MOABB: trustworthy algorithm benchmarking for BCIs
| Autor: | Alexandre Barachant, Vinay Jayaram |
|---|---|
| Rok vydání: | 2018 |
| Předmět: |
FOS: Computer and information sciences
Databases Factual Computer science Interface (Java) 0206 medical engineering Computer Science - Human-Computer Interaction Biomedical Engineering 02 engineering and technology Human-Computer Interaction (cs.HC) Machine Learning Set (abstract data type) 03 medical and health sciences Cellular and Molecular Neuroscience 0302 clinical medicine Software Humans Preprocessor Brain–computer interface Software suite business.industry Reproducibility of Results Electroencephalography Benchmarking 020601 biomedical engineering Brain-Computer Interfaces business Algorithm Algorithms 030217 neurology & neurosurgery Decoding methods |
| Zdroj: | Journal of Neural Engineering. 15:066011 |
| ISSN: | 1741-2552 1741-2560 |
| DOI: | 10.1088/1741-2552/aadea0 |
| Popis: | Objective Brain-computer interface (BCI) algorithm development has long been hampered by two major issues: small sample sets and a lack of reproducibility. We offer a solution to both of these problems via a software suite that streamlines both the issues of finding and preprocessing data in a reliable manner, as well as that of using a consistent interface for machine learning methods. Approach By building on recent advances in software for signal analysis implemented in the MNE toolkit, and the unified framework for machine learning offered by the scikit-learn project, we offer a system that can improve BCI algorithm development. This system is fully open-source under the BSD licence and available at https://github.com/NeuroTechX/moabb. Main results We analyze a set of state-of-the-art decoding algorithms across 12 open access datasets, including over 250 subjects. Our results show that even for the best methods, there are datasets which do not show significant improvements, and further that many previously validated methods do not generalize well outside the datasets they were tested on. Significance Our analysis confirms that BCI algorithms validated on single datasets are not representative, highlighting the need for more robust validation in the machine learning for BCIs community. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|