Effects of annotation quality on model performance
| Autor: | Indrek Seppo, Khaled Alhazmi, Martin Simon, Walaa Alsumari, Lara Podkuiko |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
0209 industrial biotechnology
Training set business.industry Computer science media_common.quotation_subject 02 engineering and technology Object (computer science) Machine learning computer.software_genre Object detection Data set Annotation 020901 industrial engineering & automation Homogeneous Data quality 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Quality (business) Artificial intelligence business computer media_common |
| Zdroj: | ICAIIC |
| Popis: | Supervised machine learning generally requires pre-labelled data. Although there are several open access and pre-annotated datasets available for training machine learning algorithms, most contain a limited number of object classes, which may not be suitable for specific tasks. As previously available pre-annotated data is not usually sufficient for custom models, most of the real world applications require collecting and preparing training data. There is an obvious trade-off between annotation quality and quantity. Time and resources can be allocated for ensuring superior data quality or for increasing the quantity of the annotated data. We test the degree of the detrimental effect caused by the annotation errors. We conclude that while the results deteriorate if annotations are erroneous; the effect – at least while using relatively homogeneous sequential video data – is limited. The benefits from the increased annotated data set size (created by using imperfect auto-annotation methods) outweighs the deterioration caused by annotated data. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|