An improved measuring similarity for short text snippets and its application in clustering search engine

Autor: Xi-Ping Jia, Hong Peng, Jiabing Wang, Zhao Li, Peng Peng
Rok vydání: 2008
Předmět:
Zdroj: 2008 International Conference on Machine Learning and Cybernetics.
Popis: Measuring the similarity of short text snippets plays an important role in information retrieval and natural language processing. Measuring the similarity for short text snippets, such as search queries, remains a challenging task. In this paper, we develop a new similarity measure, which can further improve the accuracy of semantic similarity for short text snippets, especially in the case of insufficient content, such as Web page snippets. Then we introduce our similarity measure combined with information entropy to the clustering search engine to automatically find the best clustering numbers. Meanwhile, we rank the clusters with our method and illustrate the results.
Databáze: OpenAIRE