Machine Learning in HPC Environments

Scaling Deep Learning for Cancer with Advanced Workflow Storage Integration

Authors: Justin Wozniak (Argonne National Laboratory)

Abstract: Cancer Deep Learning Environment (CANDLE) benchmarks and workflows will combine the power of exascale computing with neural network-based machine learning to address a range of loosely connected problems in cancer research. This application area poses unique challenges to the exascale computing environment. Here, we identify one challenge in CANDLE workflows, namely, saving neural network model representations to persistent storage. In this paper, we provide background on this problem, describe our solution, the Model Cache, and present performance results from running the system on a test cluster, ANL/LCRC Blues, and the petascale supercomputer NERSC Cori. We also sketch next steps for this promising workflow storage solution.

Archive Materials


Back to Machine Learning in HPC Environments Archive Listing

Back to Full Workshop Archive Listing