Evaluation of video activity localizations integrating quality and quantity measurements

Autor: Charles-Edmond Bichot, Julien Mille, Oya Celiktutan, Bulent Sankur, Eric Lombardi, Emmanuel Dellandréa, Christophe Garcia, Gonen Eren, Christian Wolf, Moez Baccouche, Mingyuan Jiu, Emre Dogan
Přispěvatelé: Extraction de Caractéristiques et Identification (imagine), Laboratoire d'InfoRmatique en Image et Systèmes d'information (LIRIS), Institut National des Sciences Appliquées de Lyon (INSA Lyon), Université de Lyon-Institut National des Sciences Appliquées (INSA)-Université de Lyon-Institut National des Sciences Appliquées (INSA)-Centre National de la Recherche Scientifique (CNRS)-Université Claude Bernard Lyon 1 (UCBL), Université de Lyon-École Centrale de Lyon (ECL), Université de Lyon-Université Lumière - Lyon 2 (UL2)-Institut National des Sciences Appliquées de Lyon (INSA Lyon), Université de Lyon-Université Lumière - Lyon 2 (UL2), Department of Electrical and Electronic Engineering [Istanbul], Boǧaziçi üniversitesi = Boğaziçi University [Istanbul], Department of Computer Engineering, Galatasaray Universitesi (GSU), Boğaziçi University [Istanbul]
Rok vydání: 2014
Předmět:
Zdroj: Computer Vision and Image Understanding
Computer Vision and Image Understanding, Elsevier, 2014, 127, pp.14-30. ⟨10.1016/j.cviu.2014.06.014⟩
ISSN: 1077-3142
1090-235X
DOI: 10.1016/j.cviu.2014.06.014
Popis: International audience; Evaluating the performance of computer vision algorithms is classically done by reporting classification error or accuracy, if the problem at hand is the classification of an object in an image, the recognition of an activity in a video or the categorization and labeling of the image or video. If in addition the detection of an item in an image or a video, and/or its localization are required, frequently used metrics are Recall and Precision, as well as ROC curves. These metrics give quantitative performance values which are easy to understand and to interpret even by non-experts. However, an inherent problem is the dependency of quantitative performance measures on the quality constraints that we need impose on the detection algorithm. In particular, an important quality parameter of these measures is the spatial or spatio-temporal overlap between a ground-truth item and a detected item, and this needs to be taken into account when interpreting the results. We propose a new performance metric addressing and unifying the qualitative and quantitative aspects of the performance measures. The performance of a detection and recognition algorithm is illustrated intuitively by performance graphs which present quantitative performance values, like Recall, Precision and F-Score, depending on quality constraints of the detection. In order to compare the performance of different computer vision algorithms, a representative single performance measure is computed from the graphs, by integrating out all quality parameters. The evaluation method can be applied to different types of activity detection and recognition algorithms. The performance metric has been tested on several activity recognition algorithms participating in the ICPR 2012 HARL competition.
Databáze: OpenAIRE
Externí odkaz: