A Comprehensive Analysis of Crowdsourcing for Subjective Evaluation of Tone Mapping Operators
| Autor: | Ali Ak, Abhishek Goswami, Patrick Le Callet, Frederic Dufaux, Wolf Hauser |
|---|---|
| Přispěvatelé: | Laboratoire des Sciences du Numérique de Nantes (LS2N), Université de Nantes - Faculté des Sciences et des Techniques, Université de Nantes (UN)-Université de Nantes (UN)-École Centrale de Nantes (ECN)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique Bretagne-Pays de la Loire (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT), DxO Labs, Laboratoire des signaux et systèmes (L2S), CentraleSupélec-Université Paris-Saclay-Centre National de la Recherche Scientifique (CNRS), Université Paris-Sud - Paris 11 (UP11)-CentraleSupélec-Centre National de la Recherche Scientifique (CNRS), IMT Atlantique Bretagne-Pays de la Loire (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-Université de Nantes - UFR des Sciences et des Techniques (UN UFR ST), Université de Nantes (UN)-Université de Nantes (UN)-École Centrale de Nantes (ECN)-Centre National de la Recherche Scientifique (CNRS), Image Perception Interaction (IPI) |
| Rok vydání: | 2020 |
| Předmět: |
Ground truth
Image quality Computer science business.industry media_common.quotation_subject Environment controlled Fidelity Tone mapping Machine learning computer.software_genre Crowdsourcing Rendering (computer graphics) [INFO.INFO-TS]Computer Science [cs]/Signal and Image Processing [INFO.INFO-TI]Computer Science [cs]/Image Processing [eess.IV] Artificial intelligence business [SPI.SIGNAL]Engineering Sciences [physics]/Signal and Image processing computer High dynamic range media_common |
| Zdroj: | Image Quality and System Performance, IS&T International Symposium on Electronic Imaging (EI 2021) IQSP Image Quality and System Performance, IS&T International Symposium on Electronic Imaging (EI 2021), Jan 2021, San Francisco, United States |
| DOI: | 10.2352/issn.2470-1173.2021.9.iqsp-262 |
| Popis: | International audience; Tone mapping operators (TMO) are pivotal in rendering High Dynamic Range (HDR) content on limited dynamic range media. Analysing the quality of tone mapped images depends on several objective factors and a combination of several subjective factors like aesthetics, fidelity etc. Objective Image quality assessment (IQA) metrics are often used to evaluate TMO quality but they do not always reflect the ground truth. A robust alternative to objective IQA metrics is subjective quality assessment. Although, subjective experiments provide accurate results, they can be time-consuming and expensive to conduct. Over the last decade, crowdsourcing experiments have become more popular for collecting large amount of data within a shorter period of time for a lesser cost. Although they provide more data requiring less resources, lack of controlled environment for the experiment results in noisy data. In this work 1 , we propose a comprehensive analysis of crowdsourcing experiments with two different groups of participants. Our contributions include a comparative study and a collection of methods to detect unreliable participants in crowdsourcing experiments in a TMO quality evaluation scenario. These methods can be utilized by the scientific community to increase the reliability of the gathered data. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|