Skip to main content
SHARE
Publication

WfCommons: Data Collection and Runtime Experiments using Multiple Workflow Systems

by Henri Casanova, Kyle Berney, Serge Chastel, Rafael Ferreira Da Silva
Publication Type
Conference Paper
Book Title
2023 IEEE 47th Annual Computers, Software, and Applications Conference (COMPSAC)
Publication Date
Page Numbers
1870 to 1875
Publisher Location
New Jersey, United States of America
Conference Name
1st IEEE International Workshop on Workflows in Distributed Environments
Conference Location
Torino, Italy
Conference Sponsor
IEEE
Conference Date
-

Scientific workflows have become ubiquitous across scientific fields, and their execution methods and systems continue to be the subject of research and development. Most experimental evaluations of these workflows rely on workflow instances, which can be either real-world or synthetic, to ensure relevance to current application domains or explore hypothetical/future scenarios. The WfCommons project addresses this need by providing data and tools to support such evaluations. In this paper, we present an overview of WfCommons and describe two recent developments. Firstly, we introduce a workflow execution "tracer" for Nextflow, which significantly enhances the set of real-world instances available in WfCommons. Secondly, we describe a workflow instance "translator" that enables the execution of any real-world or synthetic WfCommons workflow instance using Dask. Our contributions aim to provide researchers and practitioners with more comprehensive resources for evaluating scientific workflows.