Publication Type
Conference Paper
Book Title
2024 IEEE Conference on Artificial Intelligence (CAI)
Publication Date
Page Numbers
1436 to 1437
Publisher Location
New Jersey, United States of America
Conference Name
2024 IEEE Conference on Artificial Intelligence (CAI)
Conference Location
Singapore, Singapore
Conference Sponsor
IEEE
Conference Date
-
Abstract
A de-centralized, peer-to-peer AI metadata framework is demonstrated which can enable end-to-end metadata & lineage tracking for distributed Machine Learning pipelines spanning edge, High Performance Computing, and cloud environments. With a specific example of end-to-end microscopy algorithm and datasets, the proposed method shows how to enable reproducibility, audit trail, provenance of metadata artifacts. The emerging needs of automation in experimental sciences, ML-centric workflows, and FAIR metadata management across federated compute environments is addressed.
