SPEAR: Receiver-to-Receiver Acoustic Neural Warping Field

Autor:	He, Yuhang, Xu, Shitong, Zhong, Jia-Xing, Shin, Sangyun, Trigoni, Niki, Markham, Andrew
Rok vydání:	2024
Předmět:	Computer Science - Sound Computer Science - Artificial Intelligence Electrical Engineering and Systems Science - Audio and Speech Processing
Druh dokumentu:	Working Paper
Popis:	We present SPEAR, a continuous receiver-to-receiver acoustic neural warping field for spatial acoustic effects prediction in an acoustic 3D space with a single stationary audio source. Unlike traditional source-to-receiver modelling methods that require prior space acoustic properties knowledge to rigorously model audio propagation from source to receiver, we propose to predict by warping the spatial acoustic effects from one reference receiver position to another target receiver position, so that the warped audio essentially accommodates all spatial acoustic effects belonging to the target position. SPEAR can be trained in a data much more readily accessible manner, in which we simply ask two robots to independently record spatial audio at different positions. We further theoretically prove the universal existence of the warping field if and only if one audio source presents. Three physical principles are incorporated to guide SPEAR network design, leading to the learned warping field physically meaningful. We demonstrate SPEAR superiority on both synthetic, photo-realistic and real-world dataset, showing the huge potential of SPEAR to various down-stream robotic tasks. Comment: 9 pages, 5 figures in main paper
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2406.11006 Zobrazit plný text záznamu View this record from Arxiv