LogFlow: Simplified Log Analysis for Large Scale Systems
| Autor: | Benoit Pelletier, Noel De Palma, Thomas Ropars, Marc Platini |
|---|---|
| Přispěvatelé: | Université Grenoble Alpes (UGA), Laboratoire d'Informatique de Grenoble (LIG), Centre National de la Recherche Scientifique (CNRS)-Université Grenoble Alpes (UGA)-Institut polytechnique de Grenoble - Grenoble Institute of Technology (Grenoble INP ), Efficient and Robust Distributed Systems (ERODS ), Université Grenoble Alpes (UGA)-Centre National de la Recherche Scientifique (CNRS)-Université Grenoble Alpes (UGA)-Institut polytechnique de Grenoble - Grenoble Institute of Technology (Grenoble INP ), Bull atos technologies |
| Jazyk: | angličtina |
| Rok vydání: | 2021 |
| Předmět: |
0209 industrial biotechnology
Parsing Scale (ratio) Computer science Cloud systems 02 engineering and technology Orders of magnitude (volume) computer.software_genre 020901 industrial engineering & automation Recurrent neural network [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG] 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing State (computer science) Data mining [INFO.INFO-OS]Computer Science [cs]/Operating Systems [cs.OS] Layer (object-oriented design) [INFO.INFO-DC]Computer Science [cs]/Distributed Parallel and Cluster Computing [cs.DC] computer ComputingMilieux_MISCELLANEOUS |
| Zdroj: | ICDCN '21: International Conference on Distributed Computing and Networking 2021 ICDCN '21: International Conference on Distributed Computing and Networking 2021, Jan 2021, Nara, Japan. pp.116-125, ⟨10.1145/3427796.3427808⟩ ICDCN |
| DOI: | 10.1145/3427796.3427808⟩ |
| Popis: | Distributed infrastructures generate huge amount of logs that can provide useful information about the state of system, but that can be challenging to analyze. The paper presents LogFlow, a tool to help human operators in the analysis of logs by automatically constructing graphs of correlations between log entries. The core of LogFlow is an interpretable predictive model based on a Recurrent Neural Network augmented with a state-of-the-art attention layer from which correlations between log entries are deduced. To be able to deal with huge amount of data, LogFlow also relies on a new log parser algorithm that can be orders of magnitude faster than best existing log parsers. Experiments run with several system logs generated by Supercomputers and Cloud systems show that LogFlow is able to achieve more than 96% of accuracy in most cases. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|