RF-inpainter:multimodal image inpainting based on vision and radio signals

Autor: Chen, C. (Cheng), Nishio, T. (Takayuki), Bennis, M. (Mehdi), Park, J. (Jihong)
Jazyk: angličtina
Rok vydání: 2022
Předmět:
Popis: This study demonstrates the feasibility of image inpainting using both visual information and radio frequency (RF) signals. Recent developments in imaging and vision-based technologies using RF signals have revealed the potential of leveraging multimodal information to enhance image inpainting performance. In this context, we propose RF-Inpainter—a novel inpainting method that integrates visual and wireless information by fusing defective RGB images with received signal strength indicator (RSSI) using a deep auto-encoder model. The inpainting performance of RF-Inpainter is evaluated using experimentally obtained images and RSSI datasets in an indoor environment. Image-only inpainting and RSSI-only inpainting models are used as baselines to illustrate the superiority of RF-Inpainter over inpainting methods based on a single modality. The results establish that RF-Inpainter generates satisfactory inpainted images in most experimental scenarios, achieving a maximum improvement of 36.4% and 14.6% in terms of mean peak signal-to-noise ratio (PSNR) and mean structural similarity index (SSIM), respectively.
Databáze: OpenAIRE