Topic identification based on document coherence and spectral analysis
| Autor: | Paul-Armand Verhaegen, Joris D'hondt, Joost Duflou, Joris Vertommen, Dirk Cattrysse |
|---|---|
| Rok vydání: | 2011 |
| Předmět: |
Information Systems and Management
Information retrieval Computer science business.industry Graph partition Document clustering Graph Computer Science Applications Theoretical Computer Science Text mining Artificial Intelligence Control and Systems Engineering Coherence (signal processing) Spectral analysis business Software |
| Zdroj: | Information Sciences. 181:3783-3797 |
| ISSN: | 0020-0255 |
| DOI: | 10.1016/j.ins.2011.04.044 |
| Popis: | In a world with vast information overload, well-optimized retrieval of relevant information has become increasingly important. Dividing large, multiple topic spanning documents into sets of coherent subdocuments facilitates the information retrieval process. This paper presents a novel technique to automatically subdivide a textual document into consistent components based on a coherence quantification function. This function is based on stem or term chains linking document entities, such as sentences or paragraphs, based on the reoccurrences of stems or terms. Applying this function on a document results in a coherence graph of the document linking its entities. Spectral graph partitioning techniques are used to divide this coherence graph into a number of subdocuments. A novel technique is introduced to obtain the most suitable number of subdocuments. These subdocuments are an aggregation of (not necessarily adjacent) entities. Performance tests are conducted in test environments based on standardized datasets to prove the algorithm’s capabilities. The relevance of these techniques for information retrieval and text mining is discussed. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
Full Text from ScienceDirect