Implementing Efficient Data Compression and Encryption in a Persistent Key-Value Store for HPC

Authors: Jungwon Kim (Oak Ridge National Laboratory), Jeffrey S. Vetter (Oak Ridge National Laboratory)

Abstract: Recently, persistent data structures, like key-value stores (KVSs), which are stored in an HPC system's nonvolatile memory, provide an attractive solution for a number of emerging challenges like limited I/O performance. This paper investigates how to efficiently integrate data compression and encryption into persistent KVSs for HPC with the ultimate goal of hiding their costs and complexity in terms of performance and ease of use. We implement the proposed techniques on top of a distributed embedded KVS to evaluate the benefits and costs of incorporating these capabilities along different points in the dataflow path, illustrating differences in effective bandwidth, latency, and additional computational expense.

Best Poster Finalist (BP): yes

Poster: pdf
Poster summary: PDF
Reproducibility Description Appendix: PDF


Back to Poster Archive Listing