Crowdsourcing-Based Web Accessibility Evaluation with Golden Maximum Likelihood Inference
Autor: | Andreas Artmeier, Ye Wang, Can Wang, Shuyi Song, Keyue Shi, Zhi Yu, Jiajun Bu |
---|---|
Rok vydání: | 2018 |
Předmět: |
Evaluation system
Computer Networks and Communications business.industry Computer science Maximum likelihood 05 social sciences Inference 02 engineering and technology Crowdsourcing Data science Task (project management) Human-Computer Interaction User experience design 020204 information systems Web page 0202 electrical engineering electronic engineering information engineering 0501 psychology and cognitive sciences business 050107 human factors Social Sciences (miscellaneous) Web accessibility |
Zdroj: | Proceedings of the ACM on Human-Computer Interaction. 2:1-21 |
ISSN: | 2573-0142 |
DOI: | 10.1145/3274432 |
Popis: | Web accessibility evaluation examines how well websites comply with accessibility guidelines which help people with disabilities to perceive, navigate and contribute to the Web. This demanding task usually requires manual assessment by experts with many years of training and experience. However, not enough experts are available to carry out the increasing number of evaluation projects while non-experts often have different opinions about the presence of accessibility barriers. Addressing these issues, we introduce a crowdsourcing system with a novel truth inference algorithm to derive reliable and accurate assessments from conflicting opinions of evaluators. Extensive evaluation on 23,901 complex tasks assessed by 50 people with and without disabilities shows that our approach outperforms state of the art approaches. In addition, we conducted surveys to identify frequent barriers that people with disabilities are facing in their daily lives and the difficulty to access Web pages when they encounter these barriers. The frequencies and severities of barriers correlate with their derived importance in our evaluation project. |
Databáze: | OpenAIRE |
Externí odkaz: |