ResCuE-HPC: 1st Workshop on Reproducible, Customizable, and Portable Workflows for HPC
TimeSunday, November 11th9am - 9:10am
DescriptionExperiment reproducibility and artifact sharing is gradually becoming a norm for publications at HPC conferences. However, our recent experience to validate experimental results during artifact evaluation (AE) at PPoPP, CGO, PACT and SC also highlighted multiple problems. Ad-hoc experimental workflows, lack of common experimental methodology and tools, and the ever changing software/hardware stack all place a heavy burden on evaluators when installing, running and analyzing complex experiments. Worse still, even when artifacts (e.g. benchmarks, data sets, models) are shared, they are typically difficult to customize, port, reuse, and build upon. Establishing common experimental workflow frameworks with common formats/APIs and portable package managers can considerably simplify validation and reuse of experimental results.
This workshop will bring together HPC researchers and practitioners interested in developing common experimental methodologies, workflow frameworks and package managers for HPC (in particular, participants in CLUSTER competitions and the recent SC reproducibility initiative). We will discuss the best practices of sharing benchmarks, data sets, tools and experimental results in a customizable and reusable way. We will also cover state-of-the-art in frameworks for running workloads on ever changing systems, their current drawbacks and future improvements. Finally, we will compile a report on the current state-of-the art techniques and tools which we will share with artifact evaluation committees at SC and other top-tier conferences, and the ACM taskforce on reproducibility where we are founding members. We believe such a practical approach will help to improve reproducibility initiative and artifact exchange at SC while accelerating hardware/software co-design of efficient HPC systems.