Provenance Capture Mining
ADIOS-PApril 26, 2013
The Adaptable IO System (ADIOS) was developed to provides a simple and flexible way to manage IO related tasks in large-scale and data-intensive scientific applications and it has been playing a central role in many real-world scientific applications, such as Gyrokinetic Toroidal Code (GTC), plasma fusion simulation code (XGC), combustion simulation code (S3D), etc. ADIOS-P is a project to extend its success. Based on the ADIOS framework, we take a further step toward supporting intelligence in data management. Our research goal is two fold; i) support provenance through ADIOS in collecting and indexing various metadata information generated during the data accessing and processing, and ii) provide a systematic way to exploit collected information for enhancing IO performance and tuning performance parameters in applications. In other words, ADIOS-P will provide not only data lineage or audit trails, but also provide a framework to perform various knowledge discovery and data mining processes to discover hidden knowledge in a collaborative multi-user environment or a large-scale simulations with multiple components.