Musical Speech: A Transformer-based Composition Tool

Autor:	d'Eon, Jason, Dumpala, Sri Harsha, Sastry, Chandramouli Shama, Oore, Dani, Oore, Sageev
Rok vydání:	2021
Předmět:	Computer Science - Sound Computer Science - Computer Vision and Pattern Recognition Computer Science - Machine Learning Electrical Engineering and Systems Science - Audio and Speech Processing
Druh dokumentu:	Working Paper
Popis:	In this paper, we propose a new compositional tool that will generate a musical outline of speech recorded/provided by the user for use as a musical building block in their compositions. The tool allows any user to use their own speech to generate musical material, while still being able to hear the direct connection between their recorded speech and the resulting music. The tool is built on our proposed pipeline. This pipeline begins with speech-based signal processing, after which some simple musical heuristics are applied, and finally these pre-processed signals are passed through Transformer models trained on new musical tasks. We illustrate the effectiveness of our pipeline -- which does not require a paired dataset for training -- through examples of music created by musicians making use of our tool. Comment: NeurIPS 2020 Demonstration Track; extended for PMLR
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2108.01043 Zobrazit plný text záznamu View this record from Arxiv