MRQAR: A generic MapReduce framework to discover quantitative association rules in big data problems
| Autor: | Martín, D., Martínez Ballesteros, María del Mar, García Gil, D., Alcalá Fernández, J., Herrera, F., Riquelme Santos, José Cristóbal |
|---|---|
| Přispěvatelé: | Universidad de Sevilla. Departamento de Lenguajes y Sistemas Informáticos, Universidad de Sevilla. TIC-254: Data Science and Big Data Lab, Ministerio de Economía y Competitividad (MINECO). España |
| Rok vydání: | 2018 |
| Předmět: | |
| Zdroj: | idUS. Depósito de Investigación de la Universidad de Sevilla Fundacion Sancho el Sabio Fundazioa (FSS) |
| Popis: | Many algorithms have emerged to address the discovery of quantitative association rules from datasets in the last years. However, this task is becoming a challenge because the processing power of most existing techniques is not enough to handle the large amount of data generated nowadays. These vast amounts of data are known as Big Data. A number of previous studies have been focused on mining boolean or nominal association rules from Big Data problems, nevertheless, the data in real-world applications usually consist of quantitative values and designing data mining algorithms able to extract quantitative association rules presents a challenge to workers in this research field. In spite of the fact that we can find classical methods to discover boolean or nominal association rules in the most well-known repositories of Big Data algorithms, such repositories do not provide methods to discover quantitative association rules. Indeed, no methodologies have been proposed in the literature without prior discretization in Big Data. Hence, this work proposes MRQAR, a new generic parallel framework to discover quantitative association rules in large amounts of data, designed following the MapReduce paradigm using Apache Spark. MRQAR performs an incremental learning able to run any sequential quantitative association rule algorithm in Big Data problems without needing to redesign such algorithms. As a case study, we have integrated the multiobjective evolutionary algorithm MOPNAR into MRQAR to validate the generic MapReduce framework proposed in this work. The results obtained in the experimental study performed on five Big Data problems prove the capability of MRQAR to obtain reduced set of high quality rules in reasonable time. Ministerio de Economía y Competitividad TIN2017-89517-P Ministerio de Economía y Competitividad TIN2014-55894-C2-1-R Ministerio de Economía y Competitividad TIN2017-88209-C2-2-R |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|