The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Authors: Peter Messmer (Nvidia Corporation), John Stone (University of Illinois), Albert Reuther (Massachusetts Institute of Technology), Michael Ringenburg (Cray Inc), Sadaf Alam (Swiss National Supercomputing Centre)
Abstract: Interactive exploration of data, preparation and debugging of large-scale scientific simulations, in-situ visualization, and application steering are compelling scenarios for exploratory science, design optimizations, and signal processing. However, various technical, organizational and sociological challenges must be overcome to make these interactive workflows mainstream in HPC centers: What problem domains and workflows can benefit most from interactivity? How can we simplify the tool-chain? How can centers support highly interactive workflows? The goal of this BoF is bring together domain scientists, tool developers, and HPC center administrators to identify the scientific impact and technical challenges of highly interactive access to HPC resources.
Long Description: The goal of this BOF is to form a community of domain scientists, tool developers and HPC center operators interested in the interactive use of supercomputing systems, to gain a better understanding of the respective needs and requirements, share current progress and best practices, and to form a common vision for interactive HPC resources.
The traditional scientific computing workflow is driven by external constraints, like time available on a shared resource or physical distance between user and HPC center, rather than by the requirements of the scientist’s agenda. Intuition-forming exploratory parameter scans, “what-if” scenarios or investigations of extreme cases are at the core of the scientific discovery process. Preparation and debugging of of state-of-the-art simulations has become increasingly difficult as data sizes have grown. Even the most powerful on-site workstations are no longer capable of supporting interactivity during simulation preparation, and repeated data transfers between the scientist’s laboratory and remote supercomputing sites represents an increasing fraction of a scientist’s workday with conventional approaches. These computationally highly demanding scenarios could all benefit from supercomputing resources, but they are often hard to realize in the traditional, batch oriented workflow.
Similarly, data analytics and machine learning workflows frequently require interactive exploration of large data sets. These data-intensive workflows can greatly benefit from HPC resources, and supercomputing centers are increasingly looking to support these applications. However, enabling true real-time interactive exploration of large data sets requires a significant shift in the traditional usage model of supercomputing resources and the metrics that are used to evaluate their effective usage and the quality-of-service provided to the researcher using them.
Well attended BOF sessions at SC15 and SC17 (~100 participants) and a packed half-day workshop at ISC 2018 (~50 participants, 7 talks, 4 proceedings papers) have demonstrated the interest in a cross-disciplinary and cross-organizational venue to define the right policies, develop the necessary tools and provide the scientific and analytics framework to optimally benefit from interactive supercomputing. These sessions also demonstrated the breadth of opportunities and challenges faced by domain scientists, center managers and tool developers when using supercomputing systems interactively.
As one of the few events that attracts a global audience of members from all these groups, SC offers a unique venue to form such a community. The informal nature of a BOF is the ideal format to foster the cross-disciplinary exchange and to kick-start joint activities. Domain scientists will better understand the available tools and the perspective of center operators. Tool developers should get a broader perspective of the needs of domain scientists and challenges faced by center operators, whereas HPC center operators should get an understanding of the different use case and their impact on the scientific users.
We will to summarize the discussion and insight gained at this workshop into a report for the greater community and expand upon these activities at future SC conferences.
A request: it will be great if this BoF is not scheduled when the Top 500 BoF, or In-situ visualization BoFs are happening at SC18.
URL: https://sites.google.com/view/interative-hpc/
Back to Birds of a Feather Archive Listing