Search and Explore Strategies for Interactive Analysis of Real-Life Image Collections with Unknown and Unique Categories
| Autor: | Gisolf, F., Geradts, Z., Worring, M., Lokoč, J., Schoeffmann, K., Li, X., Patras, I., Skopal, T., Mezaris, V., Vrochidis, S. |
|---|---|
| Přispěvatelé: | IvI Research (FNWI), Intelligent Sensory Information Systems (IVI, FNWI), Multimedia Analytics Lab (IvI, FNWI), Video & Image Sense Lab (IvI, FNWI), Operations Management (ABS, FEB) |
| Rok vydání: | 2021 |
| Předmět: |
Information retrieval
business.industry Computer science User modeling ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION 020207 software engineering 02 engineering and technology Interactive analysis Convolutional neural network Image (mathematics) User studies Ask price Analytics 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing business Cluster analysis |
| Zdroj: | MultiMedia Modeling ISBN: 9783030678340 MMM (2) MultiMedia Modeling: 27th International Conference, MMM 2021 : Prague, Czech Republic, June 22–24, 2021 : proceedings, 2, 244-255 |
| Popis: | Many real-life image collections contain image categories that are unique to that specific image collection and have not been seen before by any human expert analyst nor by a machine. This prevents supervised machine learning to be effective and makes evaluation of such an image collection inefficient. Real-life collections ask for a multimedia analytics solution where the expert performs search and explores the image collection, supported by machine learning algorithms. We propose a method that covers both exploration and search strategies for such complex image collections. Several strategies are evaluated through an artificial user model. Two user studies were performed with experts and students respectively to validate the proposed method. As evaluation of such a method can only be done properly in a real-life application, the proposed method is applied on the MH17 airplane crash photo database on which we have expert knowledge. To show that the proposed method also helps with other image collections an image collection created with the Open Image Database is used. We show that by combining image features extracted with a convolutional neural network pretrained on ImageNet 1k, intelligent use of clustering, a well chosen strategy and expert knowledge, an image collection such as the MH17 airplane crash photo database can be interactively structured into relevant dynamically generated categories, allowing the user to analyse an image collection efficiently. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|