Details

Autor(en) / Beteiligte

• Herrmann, Moritz
• Pfisterer, Florian
• Scheipl, Fabian

Titel

A geometric framework for outlier detection in high‐dimensional data

Ist Teil von

• Wiley interdisciplinary reviews. Data mining and knowledge discovery, 2023-05, Vol.13 (3), p.e1491-n/a

Ort / Verlag

Hoboken, USA: Wiley Periodicals, Inc

Erscheinungsjahr

2023

Quelle

Alma/SFX Local Collection

Beschreibungen/Notizen

• Outlier or anomaly detection is an important task in data analysis. We discuss the problem from a geometrical perspective and provide a framework which exploits the metric structure of a data set. Our approach rests on the manifold assumption, that is, that the observed, nominally high‐dimensional data lie on a much lower dimensional manifold and that this intrinsic structure can be inferred with manifold learning methods. We show that exploiting this structure significantly improves the detection of outlying observations in high dimensional data. We also suggest a novel, mathematically precise and widely applicable distinction between distributional and structural outliers based on the geometry and topology of the data manifold that clarifies conceptual ambiguities prevalent throughout the literature. Our experiments focus on functional data as one class of structured high‐dimensional data, but the framework we propose is completely general and we include image and graph data applications. Our results show that the outlier structure of high‐dimensional and non‐tabular data can be detected and visualized using manifold learning methods and quantified using standard outlier scoring methods applied to the manifold embedding vectors. This article is categorized under: Technologies > Structure Discovery and Clustering Fundamental Concepts of Data and Knowledge > Data Concepts Technologies > Visualization A geometric framework exploiting the metric structure of a data set allows to (1) conceptualize outlier detection on a general level and (2) to conduct outlier detection in a principled and canonical way in very different high‐dimensional and/or non‐tabular data types such as functions (A.1), graphs (B.1), or images (C.1). The framework furthermore distinguishes structural (red) and distributional (blue) outliers, which can be detected, visualized, and quantified (A.2–C.2) with simple and well‐established manifold learning and outlier scoring methods such as MDS and LOF (not all graph and image observations can be plotted at once in B.1 and C.1.).