A Similarity Measure for Text Classification and Clustering
| Autor: | Yung Shen Lin, Shie Jue Lee, Jung-Yi Jiang |
|---|---|
| Rok vydání: | 2014 |
| Předmět: |
Clustering high-dimensional data
Fuzzy clustering Computer science Correlation clustering Single-linkage clustering Similarity measure computer.software_genre Text mining Text processing Similarity (network science) Consensus clustering Entropy (information theory) Cluster analysis k-medians clustering business.industry Document classification Pattern recognition Document clustering Spectral clustering Computer Science Applications Data set Euclidean distance Data stream clustering Computational Theory and Mathematics Normalized compression distance Data mining Artificial intelligence business computer Information Systems |
| Zdroj: | IEEE Transactions on Knowledge and Data Engineering. 26:1575-1590 |
| ISSN: | 1041-4347 |
| DOI: | 10.1109/tkde.2013.19 |
| Popis: | Measuring the similarity between documents is an important operation in the text processing field. In this paper, a new similarity measure is proposed. To compute the similarity between two documents with respect to a feature, the proposed measure takes the following three cases into account: a) The feature appears in both documents, b) the feature appears in only one document, and c) the feature appears in none of the documents. For the first case, the similarity increases as the difference between the two involved feature values decreases. Furthermore, the contribution of the difference is normally scaled. For the second case, a fixed value is contributed to the similarity. For the last case, the feature has no contribution to the similarity. The proposed measure is extended to gauge the similarity between two sets of documents. The effectiveness of our measure is evaluated on several real-world data sets for text classification and clustering problems. The results show that the performance obtained by the proposed measure is better than that achieved by other measures. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|