Designing High-Performance, Resilient, and Heterogeneity-Aware Key-Value Storage for Modern HPC Clusters
TimeWednesday, November 14th8:30am - 5pm
DescriptionDistributed key-value stores are being increasingly used to accelerate Big Data workloads on modern HPC clusters. The advances in HPC technologies (e.g., RDMA, SSDs) has directed several efforts towards employing hybrid storage with RDMA, for designing high- performance key-value stores. With this as basis, in my research, I take a holistic approach to designing a high-performance key-value storage system for HPC clusters that can maximize end-to-end performance while ensuring data resilience, that encompasses: (1) RDMA-enabled networking, (2) high-speed NVMs, and, (3) heterogeneous compute capabilities, available on current HPC systems. Towards this, I introduce RDMA-aware designs to enable: (1) non-blocking API semantics for designing high-performance client-side read/write pipelines, (2) fast online erasure coding for memory-efficient resilience, and, (3) SIMD-aware server-side accelerations; to enable Big Data applications to optimally leverage hybrid key-value stores in HPC environments.