Dataset popularity prediction for caching of CMS big data
| Autor: | Nicola Tonellotto, Raffaele Perego, Marco Meoni |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2018 |
| Předmět: |
Computer Networks and Communications
Computer science Data management Big data CPU time 02 engineering and technology computer.software_genre Capacity planning 020204 information systems Machine learning 0202 electrical engineering electronic engineering information engineering Dataset popularity Cache algorithms Caching strategies business.industry Classification Grid computing Hardware and Architecture Scalability 020201 artificial intelligence & image processing Data mining Cache business computer Software Information Systems |
| Zdroj: | Journal of grid computing 16 (2018): 211–228. doi:10.1007/s10723-018-9436-4 info:cnr-pdr/source/autori:Meoni M.; Perego R.; Tonellotto N./titolo:Dataset popularity prediction for caching of CMS big data/doi:10.1007%2Fs10723-018-9436-4/rivista:Journal of grid computing/anno:2018/pagina_da:211/pagina_a:228/intervallo_pagine:211–228/volume:16 |
| DOI: | 10.1007/s10723-018-9436-4 |
| Popis: | The Compact Muon Solenoid (CMS) experiment at the European Organization for Nuclear Research (CERN) deploys its data collections, simulation and analysis activities on a distributed computing infrastructure involving more than 70 sites worldwide. The historical usage data recorded by this large infrastructure is a rich source of information for system tuning and capacity planning. In this paper we investigate how to leverage machine learning on this huge amount of data in order to discover patterns and correlations useful to enhance the overall efficiency of the distributed infrastructure in terms of CPU utilization and task completion time. In particular we propose a scalable pipeline of components built on top of the Spark engine for large-scale data processing, whose goal is collecting from different sites the dataset access logs, organizing them into weekly snapshots, and training, on these snapshots, predictive models able to forecast which datasets will become popular over time. The high accuracy achieved indicates the ability of the learned model to correctly separate popular datasets from unpopular ones. Dataset popularity predictions are then exploited within a novel data caching policy, called PPC (Popularity Prediction Caching). We evaluate the performance of PPC against popular caching policy baselines like LRU (Least Recently Used). The experiments conducted on large traces of real dataset accesses show that PPC outperforms LRU reducing the number of cache misses up to 20% in some sites. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|