fsdaSAS: A Package for Robust Regression for Very Large Datasets Including the Batch Forward Search
| Autor: | Francesca Torti, Aldo Corbellini, Anthony C. Atkinson |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
Multivariate statistics
approximate analysis Computer science Robust statistics linked plots computer.software_genre 01 natural sciences Robust regression 010104 statistics & probability Grubbs' test for outliers big data Robustness (computer science) 0502 economics and business media_common.cataloged_instance HA Statistics QA Mathematics 0101 mathematics European union 050205 econometrics media_common Statistics 05 social sciences Estimator Regression HA1-4737 robust regression monitoring Data mining computer |
| Zdroj: | Stats, Vol 4, Iss 22, Pp 327-347 (2021) Stats Volume 4 Issue 2 Pages 22-347 |
| ISSN: | 2571-905X |
| DOI: | 10.3390/stats4020022 |
| Popis: | The forward search (FS) is a general method of robust data fitting that moves smoothly from very robust to maximum likelihood estimation. The regression procedures are included in the MATLAB toolbox FSDA. The work on a SAS version of the FS originates from the need for the analysis of large datasets expressed by law enforcement services operating in the European Union that use our SAS software for detecting data anomalies that may point to fraudulent customs returns. Specific to our SAS implementation, the fsdaSAS package, we describe the approximation used to provide fast analyses of large datasets using an FS which progresses through the inclusion of batches of observations, rather than progressing one observation at a time. We do, however, test for outliers one observation at a time. We demonstrate that our SAS implementation becomes appreciably faster than the MATLAB version as the sample size increases and is also able to analyse larger datasets. The series of fits provided by the FS leads to the adaptive data-dependent choice of maximally efficient robust estimates. This also allows the monitoring of residuals and parameter estimates for fits of differing robustness levels. We mention that our fsdaSAS also applies the idea of monitoring to several robust estimators for regression for a range of values of breakdown point or nominal efficiency, leading to adaptive values for these parameters. We have also provided a variety of plots linked through brushing. Further programmed analyses include the robust transformations of the response in regression. Our package also provides the SAS community with methods of monitoring robust estimators for multivariate data, including multivariate data transformations. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|