Exploring large scale security system reproducibility with the LESS simulator
| Autor: | John Sonchack, Adam J. Aviv |
|---|---|
| Rok vydání: | 2016 |
| Předmět: |
Ground truth
Scale (ratio) Computer Networks and Communications Computer science Network security business.industry 020206 networking & telecommunications 02 engineering and technology Limited availability Internet topology computer.software_genre Hardware and Architecture Software deployment Factor (programming language) 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Data mining Safety Risk Reliability and Quality Autonomous system (mathematics) business computer Software Simulation computer.programming_language |
| Zdroj: | Journal of Computer Security. 24:645-665 |
| ISSN: | 1875-8924 0926-227X |
| Popis: | Many network security systems analyze large scale data collected from multiple collaborating domains or aggregated network vantage points. Scale is clearly beneficial for these systems, however it also makes them difficult to design and test. Large scale data sets can be difficult to acquire and may not contain important meta-information (e.g. ground truth). Further, their limited availability can make it extremely difficult to understand how well experimental results would reproduce in differ- ent conditions, or at different networks. In this article, we discuss using simulation to overcome these challenges. We present an augmented version of LESS, our recently proposed agent based simulator for evaluating large scale network security systems. LESS uses publicly available data sets and high level parameters to generate synthetic traffic that models large scale, multi- network scenarios. Essentially, LESS allows researchers to "scale up" the data and statistics about networks and attacks that they have access to, so that they can be used to test large scale network security systems. Researchers can also tune LESS's high level parameters to better understand the sensitivities of their systems, and the reproducibility of their results. The version of LESS that we discuss in this article is extended to allow researchers to study an additional factor of system performance related to reproducibility: deployment location; by modeling the global Internet topology at the Autonomous System level. We demonstrate the applicability and benefits of LESS by tuning it with publicly available traces and then using generated records to reproduce and extend results from several recently proposed large scale security systems. In new experiments, we use LESS to study how deployment location affects large scale security systems. Our results demonstrate that LESS can evoke realistic performance from these systems with minimal tuning and provide insight into the network and topological factors that may affect the reproducibility of their evaluations. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|