P-Gram: Positional N-Gram for the Clustering of Machine-Generated Messages
| Autor: | Jean-Guy Schneider, Zeinab Farahmandpour, Arafat Hossain, Christopher Leckie, Jun Han, Jiaojiao Jiang, Steve Versteeg |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2019 |
| Předmět: |
General Computer Science
Computer science General Engineering Keyword extraction 020206 networking & telecommunications 02 engineering and technology Intrusion detection system computer.software_genre Weighting n-gram Machine-generated messages 0202 electrical engineering electronic engineering information engineering Entropy (information theory) 020201 artificial intelligence & image processing General Materials Science Data mining lcsh:Electrical engineering. Electronics. Nuclear engineering Electrical and Electronic Engineering Cluster analysis computer lcsh:TK1-9971 clustering |
| Zdroj: | IEEE Access, Vol 7, Pp 88504-88516 (2019) |
| ISSN: | 2169-3536 |
| Popis: | An IT system generates messages for other systems or users to consume, through direct interaction or as system logs. Automatically identifying the types of these machine-generated messages has many applications, such as intrusion detection and system behavior discovery. Among various heuristic methods for automatically identifying message types, the clustering methods based on keyword extraction have been quite effective. However, these methods still suffer from keyword misidentification problems, i.e., some keyword occurrences are wrongly identified as payload and some strings in the payload are wrongly identified as keyword occurrences, leading to the misidentification of the message types. In this paper, we propose a new machine language processing (MLP) approach, called P-gram, specifically designed for identifying keywords in, and subsequently clustering, machine-generated messages. First, we introduce a novel concept and technique, positional n-gram, for message keywords extraction. By associating the position as meta-data with each n-gram, we can more accurately discern which n-grams are keywords of a message and which n-grams are parts of the payload information. Then, the positional keywords are used as features to cluster the messages, and an entropy-based positional weighting method is devised to measure the importance or weight of the positional keywords to each message. Finally, a general centroid clustering method, K-Medoids, is used to leverage the importance of the keywords and cluster messages into groups reflecting their types. We evaluate our method on a range of machine-generated (text and binary) messages from the real-world systems and show that our method achieves higher accuracy than the current state-of-the-art tools. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|