Authors: Todd Gamblin (Lawrence Livermore National Laboratory), Adam Stewart (University of Illinois, Argonne National Laboratory), Johannes Albert-von der Gönna (Leibniz Supercomputing Centre), Marc Pérache (Atomic Energy and Alternative Energies Commission (CEA)), Matt Belhorn (Oak Ridge National Laboratory)
Abstract: Spack is a package manager for scientific software, with a rapidly growing open source community. Spack has over 300 contributors from academia, industry, and laboratories across the world. It is used to manage software releases for the U.S. Exascale Computing Project and the Summit supercomputer. At this BoF, Spack developers will give updates on the community and on new features such as environments and binary distribution. Sites using Spack will describe their experiences, and we will open the floor for discussion. All are invited to provide feedback, request features, and discuss future directions. Help us make installing HPC software simple!
Long Description: HPC software is becoming increasingly complex. The largest applications require over 100 dependency libraries, and they combine interpreted languages like Python with C, C++, and Fortran libraries. To achieve good performance, developers tune for multiple compilers, build options, and implementations of dependency libraries like MPI, BLAS, and LAPACK. The space of possible build configurations is combinatorial, and developers waste countless hours rebuilding software instead of producing new scientific results.
Spack (https://spack.io) is a package manager for scientific computing. It aims to reduce the complexity of building and installing HPC software on laptops, clusters, and the most powerful supercomputers in the world. There are over 2,800 package recipes in the Spack mainline repository, and Spack itself is downloaded from GitHub 400-500 times per day. Spack has a rapidly growing open source community from across the world, with over 290 contributors from academia, industry, and laboratories. End users install complex HPC applications; developers manage dependencies for themselves and for their team; and the largest supercomputing sites in the world use Spack to deploy software for thousands of users. Spack is also being used to manage software releases for the U.S. Exascale Computing Project.
The goals of this BOF are 1) to inform users about recent and upcoming developments, 2) to connect sites and developers using Spack to manage software, 3) to solicit feedback to guide future directions, and 4) to build the Spack community.
Since we originally presented Spack at SC15, the community has grown very rapidly, and new features are being added at a rapid pace. SC is a great venue to keep the user community informed about the most important new developments. At this BOF, we plan to highlight two features we have been working on for the past year. First, reproducible environments provide a way for developers to easily manage multiple stacks of software in a coherent way. They allow projects to specify component dependencies, build them, and easily reproduce the same build with the same versions, even on a different machine. Second, we are deploying cloud-hosted build infrastructure for Spack, and we will be providing binary packages for end users. We will also focus on how the Spack community is managed, and how attendees themselves can join and contribute. We will have a presentation from one of our core maintainers on how to join and contribute to open source projects like Spack.
This is our first Spack BOF submission, but we have held Spack tutorials at SC16 and SC17, with a third to be held at SC18. Past tutorials garnered around 40 attendees. We also presented short overviews of Spack as part of the “Getting Scientific Software Installed” BOF, held at SC13, SC14, and SC15. That BOF attracted 46, 78, and 158 attendees, so we believe there is strong interest in this topic area. Users and contributors have asked for a more interactive forum to build the community, make announcements, and solicit feedback. An SC BOF is the ideal venue for these interactions.
Back to Birds of a Feather Archive Listing