Content area

Abstract

Realistic, relevant, and reproducible experiments often need input traces collected from real-world environments. We focus in this work on traces of workflows---common in datacenters, clouds, and HPC infrastructures. We show that the state-of-the-art in using workflow-traces raises important issues: (1) the use of realistic traces is infrequent, and (2) the use of realistic, {\it open-access} traces even more so. Alleviating these issues, we introduce the Workflow Trace Archive (WTA), an open-access archive of workflow traces from diverse computing infrastructures and tooling to parse, validate, and analyze traces. The WTA includes \({>}48\) million workflows captured from \({>}10\) computing infrastructures, representing a broad diversity of trace domains and characteristics. To emphasize the importance of trace diversity, we characterize the WTA contents and analyze in simulation the impact of trace diversity on experiment results. Our results indicate significant differences in characteristics, properties, and workflow structures between workload sources, domains, and fields.

Details

1009240
Title
The Workflow Trace Archive: Open-Access Data from Public and Private Computing Infrastructures -- Technical Report
Publication title
arXiv.org; Ithaca
Publication year
2019
Publication date
Jul 11, 2019
Section
Computer Science
Publisher
Cornell University Library, arXiv.org
Source
arXiv.org
Place of publication
Ithaca
Country of publication
United States
University/institution
Cornell University Library arXiv.org
e-ISSN
2331-8422
Source type
Working Paper
Language of publication
English
Document type
Working Paper
Publication history
 
 
Online publication date
2019-07-12
Milestone dates
2019-06-18 (Submission v1); 2019-07-11 (Submission v2)
Publication history
 
 
   First posting date
12 Jul 2019
ProQuest document ID
2243263728
Document URL
https://www.proquest.com/working-papers/workflow-trace-archive-open-access-data-public/docview/2243263728/se-2?accountid=208611
Full text outside of ProQuest
Copyright
© 2019. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Last updated
2019-07-13
Database
ProQuest One Academic