Outlier detection using an ensemble of clustering algorithms
| Autor: | Shameem Ahmed, Ram Sarkar, Biswarup Ray, Soulib Ghosh, Mita Nasipuri |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
Fuzzy clustering
Source code Computer Networks and Communications Computer science media_common.quotation_subject computer.software_genre Fuzzy logic Field (computer science) ComputingMethodologies_PATTERNRECOGNITION Hardware and Architecture Outlier Media Technology Unsupervised learning Anomaly detection Data mining Cluster analysis computer Software media_common |
| Zdroj: | Multimedia Tools and Applications. 81:2681-2709 |
| ISSN: | 1573-7721 1380-7501 |
| Popis: | Outlier detection is an important research area in the field of machine learning and data science. The presence of outliers in a dataset limits its true usefulness in a real-life scenario. Due to the varied challenges, researchers strive to find a general method to be useful for different datasets. In this paper, we have proposed an outlier detection technique based on unsupervised learning using an ensemble of three clustering algorithms, namely K-means, K-means++ and Fuzzy C-means. We have proposed a unique way to deal with clustered outliers. Outcomes of the three aforementioned clustering algorithms are combined intelligently to accumulate all the complementary information. To combine the decisions of the hard and soft clustering algorithms, we have proposed a novel probability-based technique, which assigns a membership value to each data point in the case of a hard clustering algorithm. Three cluster validity indices are used as our evaluation metrics, which measure the goodness of a cluster. Significant improvement of cluster validity indices is observed after removing the outliers, which ensures the removal of outliers has resulted in stringent clusters. The method is evaluated on eight datasets, among which, three datasets are comparatively large. Source code of this work is available at: https://github.com/biswarup9/Outlier-Detection-Using-an-Ensemble-of-Clustering-Algorithms- . |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
Full text from SpringerLink