Generalized outlier detection with flexible kernel density estimates

Erich Schubert, Arthur Zimek, Hans Peter Kriegel

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

Abstract

We analyse the interplay of density estimation and outlier detection in density-based outlier detection. By clear and principled decoupling of both steps, we formulate a generalization of density-based outlier detection methods based on kernel density estimation. Embedded in a broader framework for outlier detection, the resulting method can be easily adapted to detect novel types of outliers: while common outlier detection methods are designed for detecting objects in sparse areas of the data set, our method can be modified to also detect unusual local concentrations or trends in the data set if desired. It allows for the integration of domain knowledge and specific requirements. We demonstrate the flexible applicability and scalability of the method on large real world data sets.

Original languageEnglish
Title of host publicationProceedings of the 2014 SIAM International Conference on Data Mining
EditorsMohammed Zaki, Zoran Obradovic, Pang Ning-Tan, Arindam Banerjee, Chandrika Kamath, Srinivasan Parthasarathy
PublisherSociety for Industrial and Applied Mathematics Publications
Publication date2014
Pages542-550
ISBN (Electronic)978-1-61197-344-0
DOIs
Publication statusPublished - 2014
Externally publishedYes
Event14th SIAM International Conference on Data Mining - Philadelphia, United States
Duration: 24. Apr 201426. Apr 2014

Conference

Conference14th SIAM International Conference on Data Mining
CountryUnited States
CityPhiladelphia
Period24/04/201426/04/2014
SponsorAmerican Statistical Association

Fingerprint

Scalability

Cite this

Schubert, E., Zimek, A., & Kriegel, H. P. (2014). Generalized outlier detection with flexible kernel density estimates. In M. Zaki, Z. Obradovic, P. Ning-Tan, A. Banerjee, C. Kamath, & S. Parthasarathy (Eds.), Proceedings of the 2014 SIAM International Conference on Data Mining (pp. 542-550). Society for Industrial and Applied Mathematics Publications. https://doi.org/10.1137/1.9781611973440.63
Schubert, Erich ; Zimek, Arthur ; Kriegel, Hans Peter. / Generalized outlier detection with flexible kernel density estimates. Proceedings of the 2014 SIAM International Conference on Data Mining. editor / Mohammed Zaki ; Zoran Obradovic ; Pang Ning-Tan ; Arindam Banerjee ; Chandrika Kamath ; Srinivasan Parthasarathy. Society for Industrial and Applied Mathematics Publications, 2014. pp. 542-550
@inproceedings{a8898a6925574664b0b93cd2293b9bca,
title = "Generalized outlier detection with flexible kernel density estimates",
abstract = "We analyse the interplay of density estimation and outlier detection in density-based outlier detection. By clear and principled decoupling of both steps, we formulate a generalization of density-based outlier detection methods based on kernel density estimation. Embedded in a broader framework for outlier detection, the resulting method can be easily adapted to detect novel types of outliers: while common outlier detection methods are designed for detecting objects in sparse areas of the data set, our method can be modified to also detect unusual local concentrations or trends in the data set if desired. It allows for the integration of domain knowledge and specific requirements. We demonstrate the flexible applicability and scalability of the method on large real world data sets.",
author = "Erich Schubert and Arthur Zimek and Kriegel, {Hans Peter}",
year = "2014",
doi = "10.1137/1.9781611973440.63",
language = "English",
pages = "542--550",
editor = "Mohammed Zaki and Zoran Obradovic and Pang Ning-Tan and Arindam Banerjee and Chandrika Kamath and Srinivasan Parthasarathy",
booktitle = "Proceedings of the 2014 SIAM International Conference on Data Mining",
publisher = "Society for Industrial and Applied Mathematics Publications",
address = "United States",

}

Schubert, E, Zimek, A & Kriegel, HP 2014, Generalized outlier detection with flexible kernel density estimates. in M Zaki, Z Obradovic, P Ning-Tan, A Banerjee, C Kamath & S Parthasarathy (eds), Proceedings of the 2014 SIAM International Conference on Data Mining. Society for Industrial and Applied Mathematics Publications, pp. 542-550, 14th SIAM International Conference on Data Mining, Philadelphia, United States, 24/04/2014. https://doi.org/10.1137/1.9781611973440.63

Generalized outlier detection with flexible kernel density estimates. / Schubert, Erich; Zimek, Arthur; Kriegel, Hans Peter.

Proceedings of the 2014 SIAM International Conference on Data Mining. ed. / Mohammed Zaki; Zoran Obradovic; Pang Ning-Tan; Arindam Banerjee; Chandrika Kamath; Srinivasan Parthasarathy. Society for Industrial and Applied Mathematics Publications, 2014. p. 542-550.

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

TY - GEN

T1 - Generalized outlier detection with flexible kernel density estimates

AU - Schubert, Erich

AU - Zimek, Arthur

AU - Kriegel, Hans Peter

PY - 2014

Y1 - 2014

N2 - We analyse the interplay of density estimation and outlier detection in density-based outlier detection. By clear and principled decoupling of both steps, we formulate a generalization of density-based outlier detection methods based on kernel density estimation. Embedded in a broader framework for outlier detection, the resulting method can be easily adapted to detect novel types of outliers: while common outlier detection methods are designed for detecting objects in sparse areas of the data set, our method can be modified to also detect unusual local concentrations or trends in the data set if desired. It allows for the integration of domain knowledge and specific requirements. We demonstrate the flexible applicability and scalability of the method on large real world data sets.

AB - We analyse the interplay of density estimation and outlier detection in density-based outlier detection. By clear and principled decoupling of both steps, we formulate a generalization of density-based outlier detection methods based on kernel density estimation. Embedded in a broader framework for outlier detection, the resulting method can be easily adapted to detect novel types of outliers: while common outlier detection methods are designed for detecting objects in sparse areas of the data set, our method can be modified to also detect unusual local concentrations or trends in the data set if desired. It allows for the integration of domain knowledge and specific requirements. We demonstrate the flexible applicability and scalability of the method on large real world data sets.

U2 - 10.1137/1.9781611973440.63

DO - 10.1137/1.9781611973440.63

M3 - Article in proceedings

AN - SCOPUS:84958543874

SP - 542

EP - 550

BT - Proceedings of the 2014 SIAM International Conference on Data Mining

A2 - Zaki, Mohammed

A2 - Obradovic, Zoran

A2 - Ning-Tan, Pang

A2 - Banerjee, Arindam

A2 - Kamath, Chandrika

A2 - Parthasarathy, Srinivasan

PB - Society for Industrial and Applied Mathematics Publications

ER -

Schubert E, Zimek A, Kriegel HP. Generalized outlier detection with flexible kernel density estimates. In Zaki M, Obradovic Z, Ning-Tan P, Banerjee A, Kamath C, Parthasarathy S, editors, Proceedings of the 2014 SIAM International Conference on Data Mining. Society for Industrial and Applied Mathematics Publications. 2014. p. 542-550 https://doi.org/10.1137/1.9781611973440.63