A Case for Reproducibility in MIR: Replication of ‘A Highly Robust Audio Fingerprinting System’

Autor: Joren Six, Federica Bressan, Marc Leman
Jazyk: angličtina
Rok vydání: 2018
Předmět:
Zdroj: Transactions of the International Society for Music Information Retrieval, Vol 1, Iss 1, Pp 56-67 (2018)
Druh dokumentu: article
ISSN: 2514-3298
DOI: 10.5334/tismir.4
Popis: Claims made in many Music Information Retrieval (MIR) publications are hard to verify due to the fact that (i) often only a textual description is made available and code remains unpublished – leaving many implementation issues uncovered; (ii) copyrights on music limit the sharing of datasets; and (iii) incentives to put effort into reproducible research – publishing and documenting code and specifics on data – is lacking. In this article the problems around reproducibility are illustrated by replicating an MIR work. The system and evaluation described in ‘A Highly Robust Audio Fingerprinting System’ is replicated as closely as possible. The replication is done with several goals in mind: to describe difficulties in replicating the work and subsequently reflect on guidelines around reproducible research. Added contributions are the verification of the reported work, a publicly available implementation and an evaluation method that is reproducible.
Databáze: Directory of Open Access Journals