Pairwise-adaptive dissimilarity measure for document clustering
| Autor: | Joris D'hondt, Joris Vertommen, Joost Duflou, Paul-Armand Verhaegen, Dirk Cattrysse |
|---|---|
| Rok vydání: | 2010 |
| Předmět: |
Information Systems and Management
business.industry Single-linkage clustering Process (computing) Feature selection Pattern recognition Document clustering computer.software_genre Measure (mathematics) Computer Science Applications Theoretical Computer Science ComputingMethodologies_PATTERNRECOGNITION Artificial Intelligence Control and Systems Engineering Trigonometric functions Pairwise comparison Artificial intelligence Data mining business Cluster analysis computer Software Mathematics |
| Zdroj: | Information Sciences. 180:2341-2358 |
| ISSN: | 0020-0255 |
| DOI: | 10.1016/j.ins.2010.02.021 |
| Popis: | This paper introduces a novel pairwise-adaptive dissimilarity measure for large high dimensional document datasets that improves the unsupervised clustering quality and speed compared to the original cosine dissimilarity measure. This measure dynamically selects a number of important features of the compared pair of document vectors. Two approaches for selecting the number of features in the application of the measure are discussed. The proposed feature selection process makes this dissimilarity measure especially applicable in large, high dimensional document collections. Its performance is validated on several test sets originating from standardized datasets. The dissimilarity measure is compared to the well-known cosine dissimilarity measure using the average F-measures of the hierarchical agglomerative clustering result. This new dissimilarity measure results in an improved clustering result obtained with a lower required computational time. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
Full Text from ScienceDirect