<span class="var-sub_title">Chapel Aggregation Library (CAL)</span> SC18 Proceedings

The International Conference for High Performance Computing, Networking, Storage, and Analysis

PAW-ATM: Parallel Applications Workshop - Alternatives to MPI

Chapel Aggregation Library (CAL)

Abstract: Fine-grained communication is a fundamental principle of the Partitioned Global Address Space (PGAS), which serves to simplify creating and reasoning about programs in the distributed context. However, per-message overheads of communication rapidly accumulate in programs that generate a high volume of small messages, limiting the effective bandwidth and potentially increasing latency if the messages are generated at a much higher rate than the effective network bandwidth. One way to reduce such fine-grained communication is by coarsening the granularity by aggregating data, or by buffering the smaller communications together in a way that they can be dispatched in bulk. Once these communications are buffered, an additional optimization called coalescing can be performed to make processing of the data more efficient for the receiver by combining multiple units of data. The Chapel Aggregation Library (CAL) provides a straightforward approach to handling both aggregation and coalescing of data in Chapel and aims to be as generic and minimal as possible to maximize code reuse and minimize its increase in complexity on user applications. CAL provides a high-performance, distributed, and parallel-safe solution that is entirely written as a Chapel module. In addition to being easy to use, CAL has been shown to improve performance of some benchmarks by one to two orders of magnitude over naive implementations at 32 compute-nodes on a Cray XC50.

Archive Materials

Back to PAW-ATM: Parallel Applications Workshop - Alternatives to MPI Archive Listing

Back to Full Workshop Archive Listing