Designing High-Performance, Resilient, and Heterogeneity-Aware Key-Value Storage for Modern HPC Clusters

Student: Dipti Shankar (Ohio State University)
Advisor: Dhabaleswar K. Panda (Ohio State University), Xiaoyi Lu (Ohio State University)

Abstract: Distributed key-value stores are being increasingly used to accelerate Big Data workloads on modern HPC clusters. The advances in HPC technologies (e.g., RDMA, SSDs) has directed several efforts towards employing hybrid storage with RDMA, for designing high- performance key-value stores. With this as basis, in my research, I take a holistic approach to designing a high-performance key-value storage system for HPC clusters that can maximize end-to-end performance while ensuring data resilience, that encompasses: (1) RDMA-enabled networking, (2) high-speed NVMs, and, (3) heterogeneous compute capabilities, available on current HPC systems. Towards this, I introduce RDMA-aware designs to enable: (1) non-blocking API semantics for designing high-performance client-side read/write pipelines, (2) fast online erasure coding for memory-efficient resilience, and, (3) SIMD-aware server-side accelerations; to enable Big Data applications to optimally leverage hybrid key-value stores in HPC environments.

Summary: pdf
Thesis Canvas: pdf

Presentation: pdf



Back to Doctoral Showcase Archive Listing