Wormhole: a novel big data platform for 100 Gbit/s network monitoring and beyond
| Autor: | Rafael Leira, Ivan Gonzalez, Lluis Gifre, Jorge E. López de Vergara, Javier Aracil |
|---|---|
| Rok vydání: | 2019 |
| Předmět: |
business.industry
Computer science Network packet Big data 020206 networking & telecommunications Throughput 02 engineering and technology Network monitoring Encryption 020204 information systems Computer data storage 0202 electrical engineering electronic engineering information engineering The Internet business Distributed File System Computer network |
| Zdroj: | ICIN 2019 22nd Conference on Innovation in Clouds, Internet and Networks and Workshops (ICIN) |
| Popis: | Internet measurement and analysis is increasingly challenging as the Internet evolves, primarily due to changing-trends, speed increments or new protocols and ciphers. As such, ad-hoc monitoring equipment comes in handy, albeit cost-effectiveness impedes deployment at a very large scale. As an alternative, big data-based distributed architectures are being proposed for network monitoring and analysis. However, in light of the high throughput currently offered by 100 Gbit/s links, it turns out that state-of-the-art big data solutions fall short of capacity, unless a huge amount of computers are used. In order to effectively tackle that issue, we have created Wormhole: a streaming engine that circumvents existing limitations by distributing the input messages/packets coherently among different off-the-shelf analysis equipment, thus reducing costs and equipment. Should the incoming data rate be larger than the system throughput, a distributed file system can be used for temporary data storage, for subsequent filtering and in-depth analysis. The proposed solution provides on-line real-time monitoring metrics with the ability to gain further insights when required. The prototyped architecture is able to deal with 100 Gbit/s networks and can be easily scaled up to higher rates by just adding more computing nodes and/or by trimming encrypted packet payloads. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|