Feature Map Transformation for Multi-sensor Fusion in Object Detection Networks for Autonomous Driving

Autor: Julien Vitay, Enrico Schroder, Mirko Mählisch, Sascha Braun, Fred H. Hamker
Rok vydání: 2019
Předmět:
Zdroj: Advances in Intelligent Systems and Computing ISBN: 9783030177973
DOI: 10.1007/978-3-030-17798-0_12
Popis: We present a general framework for fusing pre-trained object detection networks for multiple sensor modalities in autonomous cars at an intermediate stage. The key innovation is an autoencoder-inspired Transformer module which transforms perspective as well as feature activation characteristics from one sensor modality to another. Transformed feature maps can be combined with those of a modality-native feature extractor to enhance performance and reliability through a simple fusion scheme. Our approach is not limited to specific object detection network types. Compared to other methods, our framework allows fusion of pre-trained object detection networks and fuses sensor modalities at a single stage, resulting in a modular and traceable architecture. We show effectiveness of the proposed scheme by fusing camera and Lidar information to detect objects using our own as well as the KITTI dataset.
Databáze: OpenAIRE