Aligning movies with scripts by exploiting temporal ordering constraints
| Autor: | Iftekhar Naim, Daniel Gildea, Henry Kautz, Abdullah Al Mamun, Young Chol Song, Jiebo Luo |
|---|---|
| Rok vydání: | 2016 |
| Předmět: |
Conditional random field
business.industry Computer science 02 engineering and technology 010501 environmental sciences computer.software_genre 01 natural sciences Annotation Discriminative model Scripting language 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Artificial intelligence Face detection business Hidden Markov model Cluster analysis computer Natural language processing 0105 earth and related environmental sciences |
| Zdroj: | ICPR |
| DOI: | 10.1109/icpr.2016.7899895 |
| Popis: | Scripts provide rich textual annotation of movies, including dialogs, character names, and other situational descriptions. Exploiting such rich annotations requires aligning the sentences in the scripts with the corresponding video frames. Previous work on aligning movies with scripts predominantly relies on time-aligned closed-captions or subtitles, which are not always available. In this paper, we focus on automatically aligning faces in movies with their corresponding character names in scripts without requiring closed-captions/subtitles. We utilize the intuition that faces in a movie generally appear in the same sequential order as their names are mentioned in the script. We first apply standard techniques for face detection and tracking, and cluster similar face tracks together. Next, we apply a generative Hidden Markov Model (HMM) and a discriminative Latent Conditional Random Field (LCRF) to align the clusters of face tracks with the corresponding character names. Our alignment models (especially LCRF) significantly outperform the previous state-of-the-art on two different movie datasets and for a wide range of face clustering algorithms. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|