Skip to main content
SHARE
Publication

Motivating Complex Dependence Structures in Data Mining: A Case Study with Anomaly Detection in Climate...

by Shih-chieh Kao, Auroop R Ganguly, Karsten J Steinhaeuser
Publication Type
Conference Paper
Publication Date
Page Numbers
223 to 230
Conference Name
IEEE ICDM Workshop on Knowledge Discovery from Climate Data: Prediction, Extremes, and Impacts
Conference Location
Miami, Florida, United States of America
Conference Date

While data mining aims to identify hidden knowledge from massive and high dimensional datasets, the importance of dependence structure among time, space, and between different variables is less emphasized. Analogous to the use of probability density functions in modeling individual variables, it is now possible to characterize the complete dependence space mathematically through the application of copulas. By adopting copulas, the multivariate joint probability distribution can be constructed without constraint to specific types of marginal distributions. Some common assumptions, like normality and independence between variables, can also be relieved. This study provides fundamental introduction and illustration of dependence structure, aimed at the potential applicability of copulas in general data mining. The case study in hydro-climatic anomaly detection shows that the frequency of multivariate anomalies is affected by the dependence level between variables. The appropriate multivariate thresholds can be determined through a copula-based approach.