Semantic highlight retrieval
| Autor: | Yen-Chen Lin, Kuo-Hao Zeng, Min Sun, Ali Farhadi |
|---|---|
| Rok vydání: | 2016 |
| Předmět: |
Vocabulary
Information retrieval Computer science media_common.quotation_subject InformationSystems_INFORMATIONSTORAGEANDRETRIEVAL 02 engineering and technology 010501 environmental sciences Semantics 01 natural sciences Visualization Task (project management) Bridging (programming) 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Representation (mathematics) 0105 earth and related environmental sciences media_common |
| Zdroj: | ICIP |
| DOI: | 10.1109/icip.2016.7532982 |
| Popis: | Finding highlights relevant to a text query in unedited videos has become increasingly important due to their unprecedented growth. We refer this task as semantic highlight retrieval and propose a query-dependent video representation for retrieving a variety of highlights. Our method consist of two parts: (1) “viralets”, a mid-level representation bridging between visual and semantic spaces; (2) a novel Semantically-Modulation (SM) procedure to make viralets query-dependent (referred to as SM viralets). Given SM viralets, we train a single highlight ranker to predict the highlightness of clips with respect to a variety of queries, whereas existing approaches can be applied only in a few predefined domains. We collect a viral video dataset1 including users' comments, highlights, and/or original videos. Among a database with 1189 (13% highlights) clips, our highlight ranker achieves 41.2% recall at top-10 retrieved clips. It is significantly higher than a state-of-the-art domain-specific highlight ranker and its extension. Similarly, our method also outperforms all baseline methods on the video highlight dataset. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|