Knowledge Distillation for Anomaly Detection

Autor: Pol, Adrian Alan, Govorkova, Ekaterina, Gronroos, Sonja, Chernyavskaya, Nadezda, Harris, Philip, Pierini, Maurizio, Ojalvo, Isobel, Elmer, Peter
Rok vydání: 2023
Předmět:
Druh dokumentu: Working Paper
Popis: Unsupervised deep learning techniques are widely used to identify anomalous behaviour. The performance of such methods is a product of the amount of training data and the model size. However, the size is often a limiting factor for the deployment on resource-constrained devices. We present a novel procedure based on knowledge distillation for compressing an unsupervised anomaly detection model into a supervised deployable one and we suggest a set of techniques to improve the detection sensitivity. Compressed models perform comparably to their larger counterparts while significantly reducing the size and memory footprint.
Databáze: arXiv