jMetalSP: A framework for dynamic multi-objective big data optimization
| Autor: | Juan J. Durillo, José García-Nieto, José A. Cordero, Antonio J. Nebro, Cristóbal Barba-González, José F. Aldana-Montes, Ismael Navas-Delgado |
|---|---|
| Přispěvatelé: | Universidad de Sevilla. Departamento de Ciencias de la Computación e Inteligencia Artificial, Ministerio de Educación y Ciencia (MEC). España, Junta de Andalucía |
| Rok vydání: | 2018 |
| Předmět: |
Optimization problem
jMetal business.industry Computer science Distributed computing Big data Big Data optimization 020206 networking & telecommunications Metaheuristics 02 engineering and technology computer.software_genre Multi-objective optimization Travelling salesman problem Dynamic problem Software Framework Computer cluster 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Data mining business Metaheuristic computer Software |
| Zdroj: | idUS. Depósito de Investigación de la Universidad de Sevilla instname idUS: Depósito de Investigación de la Universidad de Sevilla Universidad de Sevilla (US) |
| ISSN: | 1568-4946 |
| Popis: | Multi-objective metaheuristics have become popular techniques for dealing with complex optimization problems composed of a number of conflicting functions. Nowadays, we are in the Big Data era, so metaheuristics must be able to solve dynamic problems that may vary over time due to the processing and analysis of several streaming data sources. As this is a new field, there is a need for software platforms to solve dynamic multi-objective Big Data optimization problems. In this paper, we present jMetalSP, which combines the multi-objective optimization features of the jMetal framework with the streaming facilities of the Apache Spark cluster computing system. Thus, existing state-of-the-art multi-objective metaheuristics can be easily adapted to deal with dynamic optimization problems that are fed by multiple streaming data sources. Moreover, these algorithms can take advantage of the parallel computing features of Spark. We describe the architecture of jMetalSP and show how it can be used to solve a dynamic bi-objective instance of the Traveling Salesman Problem (TSP) based on New York City's real-time traffic data. We have also carried out an experimental study to assess the performance of the resultant jMetalSP application in a Hadoop cluster composed of 100 nodes. Ministerio de Educación y Ciencia TIN2014-58304-R Junta de Andalucía P11-TIC-7529 Junta de Andalucía P12-TIC-1519 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
Full Text from ScienceDirect