| Popis: |
In the last three decades, neural networks have evolved from an academic topic to a common scientific computing tool. CRAN currently hosts around 80 packages (May 2020) that involve neural network modeling; some offering more than one algorithm. However, to our knowledge, there is no comprehensive study which tests the accuracy, the reliability, and the ease-of-use of those NN packages. In this paper, we test a large number of packages against a common set of datasets with varying levels of complexity to benchmark and rank them with statistical metrics. We restrict our evaluation to single hidden-layer perceptrons that perform regression. We ignore packages for classification and other specialized purposes. This leaves us with approximately 60 package:algorithm pairs to test. The criteria used in our benchmark were: (i) accuracy, i.e. the ability to find the global minima on 13 datasets, measured by the Root Mean Square Error (RMSE) in a fixed number of iterations; (ii) speed of the training algorithm; (iii) availability of helpful utilities; (iv) quality of the documentation. We have given a score for each evaluation criterion to compare all package:algorithm pairs in a global table. Overall, 15 pairs are considered accurate and reliable and are recommended for daily usage. Other packages are either less accurate, slow, difficult to use, or have poor or zero documentation. |
