Benchmarking Scientific Reconfigurable / FPGA Computing

Authors: Franck Cappello (Argonne National Laboratory), Taisuke Boku (University of Tsukuba), Martin Herbordt (Boston University), Naoya Maruyama (Lawrence Livermore National Laboratory), Andrew Putnam (Microsoft Corporation), Kentaro Sano (RIKEN), Jeffrey Vetter (Oak Ridge National Laboratory), Kazutomo Yoshii (Argonne National Laboratory), Xavier Martorell (Barcelona Supercomputing Center)

Abstract: In the past three years, FPGAs have gone from niche components to a central part of many data centers worldwide. But while many use cases match those of traditional accelerators (e.g., AWS), most of these millions of newly deployed FPGAs are in other configurations such as smart NICs and storage architectures. A central problem in advancing HPC with FPGAs is therefore developing new benchmarking methodologies. In this BoF, leading researchers from industry, government, and academia will showcase cutting-edge FPGAs applications related to HPC and propose evaluation metrics. Discussion will aim at creating a new benchmark. Audience participation is highly encouraged.

Long Description: The recent integration of FPGAs in the cloud and data centers has been remarkably successful, with millions of devices deployed. In fact, according to the top500 site, Microsoft’s Azure FPGA cluster has reached to exaflops computational power. But while many of the applications are recognizable as traditional HPC -- generic acceleration and real-time AI -- the architectures of the new FPGA-based systems is far from traditional. Two surprises are noteworthy. One is how effective OpenCL has been in freeing programmers from HDLs. The other is that, while FPGAs have found a place as common accelerators (AWS), far more are being used as smart NICs, bump-in-the-wire processors, and storage accelerators. In this brave new world, existing evaluation methods are no longer sufficient. Hence, while FPGA use is in line with future HPC needs such as AI and data analytics, those performance gains are still not tangible to the wider HPC community.

Now is the perfect time to discuss the transition path to incorporate the power of reconfigurable architectures into future heterogeneous HPC systems. First, we need to understand the characteristics of FPGAs or reconfigurable computing (RC) in the context of HPC. In this BOF, we will showcase cutting-edge performance results and explain how they are achieved (e.g., reconfigurability, data flow, in-memory computing, low power, support for low precision, support for direct communication). We will then discuss benchmarking techniques and methodologies to demonstrate practical performance gains to help design future RC heterogeneous HPC systems, taking into account the following points:

- Workloads (simulation, data, learning, edge/fog, cloud) - Components (compute accelerator, network accelerator, storage accelerator) - Programming models (OpenMP, OpenCL, OpenACC, HSL, or HDL) - Metrics, how to aggregate measurements - The benefit of reconfigurability

One of potential risks with benchmarking FPGAs or RC computers is that we tend to position them as accelerators that can replace CPUs or GPUs, which can potentially obscure real potential of FPGAs (e.g., network/storage accelerators, data flow processors). Rather, we need to position FPGAs as an essential component in heterogeneous systems that work together with other components to maximize the performance of the entire system. Thus it is also important to devise benchmark metrics to measure interoperability or synergic effects.

This BOF is a confluence of two sets of previous related events. The first is a series of three international workshops on FPGAs for HPC. During the last of these, in March 2018 (https://www.ccs.tsukuba.ac.jp/hpc-iwfh/ ), we organized a panel session about FPGA benchmarking, which led to this proposal. The other previous thread is the series of eight RC BOFs at SC that were held from 2010 to 2017. We have decided that, with the advent of widespread use of FPGAs in HPC, the RC BOF should become more targeted.

All session leaders have strong experience in HPC, reconfigurable computing, or both. We will select several speakers from session leaders and leading experts in academia, government, and industry.

Audience participation is a key success metric of this BOF, and half the time has been allocated for discussion.




Back to Birds of a Feather Archive Listing