Fallout: Distributed systems testing as a service
| Autor: | Jake Luciani, Sean McCarthy, Pushkala Pattabhiraman, Matt Fleming, Guy Bolton King |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
FOS: Computer and information sciences
Service (systems architecture) business.industry Computer science Distributed computing Databases (cs.DB) Benchmarking Automation Variety (cybernetics) Software Computer Science - Databases Computer Science - Distributed Parallel and Cluster Computing Work (electrical) Scalability Distributed Parallel and Cluster Computing (cs.DC) Architecture business |
| Zdroj: | BenchCouncil Transactions on Benchmarks, Standards and Evaluations. 1:100010 |
| ISSN: | 2772-4859 |
| DOI: | 10.1016/j.tbench.2021.100010 |
| Popis: | All modern distributed systems list performance and scalability as their core strengths. Given that optimal performance requires carefully selecting configuration options, and typical cluster sizes can range anywhere from 2 to 300 nodes, it is rare for any two clusters to be exactly the same. Validating the behavior and performance of distributed systems in this large configuration space is challenging without automation that stretches across the software stack. In this paper we present Fallout, an open-source distributed systems testing service that automatically provisions and configures distributed systems and clients, supports running a variety of workloads and benchmarks, and generates performance reports based on collected metrics for visual analysis. We have been running the Fallout service internally at DataStax for over 5 years and have recently open sourced it to support our work with Apache Cassandra, Pulsar, and other open source projects. We describe the architecture of Fallout along with the evolution of its design and the lessons we learned operating this service in a dynamic environment where teams work on different products and favor different benchmarking tools. Comment: Submitted to 2021 BenchCouncil International Symposium on Benchmarking, Measuring and Optimizing (Bench'21) |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|