<span class="var-sub_title">Cloud Infrastructure Solutions To Run HPC Workloads</span> SC18 Proceedings

The International Conference for High Performance Computing, Networking, Storage, and Analysis

Cloud Infrastructure Solutions To Run HPC Workloads

Authors: Martial Michel (Data Machines Corporation), Michael Jennings (Los Alamos National Laboratory), Micheal Lowe (Indiana University), Robert Budden (Pittsburgh Supercomputing Center), Adam Simpson (Nvidia Corporation), Christian Kniep (Docker Inc), Blair Bethwaite (New Zealand eScience Infrastructure), Bob Killen (University of Michigan), Jay Kruemcke (SUSE)

Abstract: Virtualization and containers have grown to see more prominent use within the realm of HPC. Adoption of these tools has enabled IT Organizations to reduce costs while making it easier to manage large pools of compute, storage and networking resources. However, performance overheads, networking integrations, and system complexity pose daunting architectural challenges.

OpenStack, Docker, Charliecloud, Singularity, Kubernetes, and Mesos all pose their own set of unique benefits and challenges. This Birds of a Feather is aimed at architects, administrators, software engineers, and scientists interested in designing and deploying cloud infrastructure solutions to run HPC workloads.

Long Description: Cloud Computing represents one of the most significant shifts in IT, and the group of projects that comprise Open Infrastructure clouds is the new standard for putting cloud technologies and methodologies within reach. The level of interest in the application of OpenStack, Container and Container Orchestration technologies in the High-Performance and Research Computing space reflects the already strong representation of scientific cloud deployments amongst the research community.

As cloud computing has matured and adoption increased, the industry has turned to the challenges posed by application portability and service orchestration across multiple IaaS platforms and other cloud platforms. Containers and container-orchestration platforms provide solutions to these problems which are well suited to web-service developers and operators but often feel alien to HPC users and operators. However, several HPC-centric container solutions exist, and industry leaders such as Docker are now paying attention to HPC use-cases and needs.

The intent of this BoF is to provide the broader HPC community an overview of the challenges of supporting HPC workloads with OpenStack, Docker, and Kubernetes, among others, as well as introduce the best practices adopted by members of the scientific cloud community.

HPC-centric topics revolve around accounting and scheduling, including practical resource allocation approaches with the on-demand IaaS model. Through an open and thoughtful exchange, we intend to begin developing a shared understanding and vision of how open cloud computing solutions can best support existing and emerging uses in a range of research disciplines.

This meeting has already happened at SC16 and SC17 in the form of an OpenStack BoF. This year we intend to bring the conversation toward the Open Infrastructure and have invited container and container-orchestration panelists to the table.

The sponsors of this BoF represent a range of those leading the charge in the OpenStack, Container, and Container-Orchestration community. The group includes (with "PE" for "Panel Expertise"): Martial Michel (Data Machines Corp., co-chair OpenStack Scientific Special Interest Group, Docker mentor, PE: OpenStack, containers) Michael Jennings (Los Alamos National Lab, OpenStack user, PE: Charliecloud/container expert) Mike Lowe (Indiana University, PE: OpenStack) Robert Budden (Pittsburgh SuperComputing Center, PE: OpenStack) Christine Lovett (Docker Inc, PE: containers) Christian Kniep (Docker Inc, PE: containers) Blair Bethwaite (Monash, co-chair OpenStack Scientific Special Interest Group, PE: OpenStack) Bob Killen (University of Michigan, Cloud Native Computing Foundation Ambassador, PE: Kubernetes) Jay Kruemcke (Suse, PE: Kubernetes)

URL: https://etherpad.openstack.org/p/sc18CloudInfrastructures

Back to Birds of a Feather Archive Listing