AI-Based Semantic Multimedia Indexing and Retrieval for Social Media on Smartphones
| Autor: | Matthias Hemmje, Stefan Wagenpfeil, Paul Mc Kevitt, Felix Engel |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2021 |
| Předmět: |
semantic indexing
multimedia retrieval Computer science Feature vector media_common.quotation_subject graph encoding 02 engineering and technology Semantics Image (mathematics) framework 020204 information systems 0202 electrical engineering electronic engineering information engineering Social media Quality (business) media_common Information retrieval lcsh:T58.5-58.64 lcsh:Information technology Search engine indexing GMAF AI4MMRA Feature (computer vision) AI MMFVGF MMFVGG Graph (abstract data type) 020201 artificial intelligence & image processing Information Systems graph code multimedia feature vector graph |
| Zdroj: | Information, Vol 12, Iss 43, p 43 (2021) Information Volume 12 Issue 1 |
| ISSN: | 2078-2489 |
| Popis: | To cope with the growing number of multimedia assets on smartphones and social media, an integrated approach for semantic indexing and retrieval is required. Here, we introduce a generic framework to fuse existing image and video analysis tools and algorithms into a unified semantic annotation, indexing and retrieval model resulting in a multimedia feature vector graph representing various levels of media content, media structures and media features. Utilizing artificial intelligence (AI) and machine learning (ML), these feature representations can provide accurate semantic indexing and retrieval. Here, we provide an overview of the generic multimedia analysis framework (GMAF) and the definition of a multimedia feature vector graph framework (MMFVGF). We also introduce AI4MMRA to detect differences, enhance semantics and refine weights in the feature vector graph. To address particular requirements on smartphones, we introduce an algorithm for fast indexing and retrieval of graph structures. Experiments to prove efficiency, effectiveness and quality of the algorithm are included. All in all, we describe a solution for highly flexible semantic indexing and retrieval that offers unique potential for applications such as social media or local applications on smartphones. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|