A Data Compacting Technique to reduce the NetFlow size in Botnet Detection with BotCluster
| Autor: | Chun-Yu Wang, Ce-Kuen Shieh, Jyh-Biau Chang, Ta-Chun Lo, Shih-Hao Fuh, Feng-Min Cho, Yu-Cheng Chen, Qi-Jun Cheng |
|---|---|
| Rok vydání: | 2019 |
| Předmět: |
DBSCAN
Computer science business.industry Pipeline (computing) Feature vector Big data 020206 networking & telecommunications 02 engineering and technology computer.software_genre Data redundancy NetFlow 0202 electrical engineering electronic engineering information engineering Overhead (computing) 020201 artificial intelligence & image processing Data mining business computer |
| Zdroj: | BDCAT |
| DOI: | 10.1145/3365109.3368778 |
| Popis: | Big data analytics helps us to find potentially valuable knowledge, but as the size of the dataset increases, the computing cost also grows exponentially. In our previous work, BotCluster, we had designed a pre-processing filtering pipeline, including whitelist filter and flow loss-response rate (FLR) filter, for data reduction, which intended to wipe out irrelative noises and reduce the computing overhead. However, we still face a data redundancy phenomenon in which some of the same feature vectors repeatedly emerged. In this paper, we propose a data compacting approach aimed to reduce the input volume and keep enough representative feature vectors to fit DBSCAN's (Density-based spatial clustering of applications with noise) criteria. It purges the redundant vectors according to a purging threshold and keeps the primary representatives. Experimental results have shown that the average data reduction ratio is about 81.34%, while the precision has only slightly decreased by 1.6% on average, and the results still have 99.88% of IPs overlapped with the previous system. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|