Abstract

The Fermilab HEPCloud Facility Project has as its goal to extend the current Fermilab facility interface to provide transparent access to disparate resources including commercial and community clouds, grid federations, and HPC centers. This facility enables experiments to perform the full spectrum of computing tasks, including data-intensive simulation and reconstruction. We have evaluated the use of the commercial cloud to provide elasticity to respond to peaks of demand without overprovisioning local resources. Full scale data-intensive workflows have been successfully completed on Amazon Web Services for two High Energy Physics Experiments, CMS and NOνA, at the scale of 58000 simultaneous cores. This paper describes the significant improvements that were made to the virtual machine provisioning system, code caching system, and data movement system to accomplish this work. The virtual image provisioning and contextualization service was extended to multiple AWS regions, and to support experiment-specific data configurations. A prototype Decision Engine was written to determine the optimal availability zone and instance type to run on, minimizing cost and job interruptions. We have deployed a scalable on-demand caching service to deliver code and database information to jobs running on the commercial cloud. It uses the frontiersquid server and CERN VM File System (CVMFS) clients on EC2 instances and utilizes various services provided by AWS to build the infrastructure (stack). We discuss the architecture and load testing benchmarks on the squid servers. We also describe various approaches that were evaluated to transport experimental data to and from the cloud, and the optimal solutions that were used for the bulk of the data transport. Finally, we summarize lessons learned from this scale test, and our future plans to expand and improve the Fermilab HEP Cloud Facility.

Details

Title
Virtual machine provisioning, code management, and data movement design for the Fermilab HEPCloud Facility
Author
Timm, S 1 ; Cooper, G 1 ; Fuess, S 1 ; Garzoglio, G 1 ; Holzman, B 1 ; Kennedy, R 1 ; Grassano, D 1 ; Tiradani, A 1 ; Krishnamurthy, R 2 ; Vinayagam, S 2 ; Raicu, I 2 ; H Wu 2 ; Ren, S 2 ; S-Y Noh 3 

 Scientific Computing Division, Fermilab, Batavia, IL 60563, USA 
 Computer Science Dept., Illinois Institute of Technology, Chicago, IL 60616 USA 
 Korea Institute of Science and Technology Information, Daejeon, Korea 
Publication year
2017
Publication date
Oct 2017
Publisher
IOP Publishing
ISSN
17426588
e-ISSN
17426596
Source type
Scholarly Journal
Language of publication
English
ProQuest document ID
2574546592
Copyright
© 2017. This work is published under http://creativecommons.org/licenses/by/3.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.