Enabling Efficient Data Infrastructure and Analytics on HPC Systems

Student: Huansong Fu (Florida State University)
Advisor: Weikuan Yu (Florida State University)

Abstract: We propose to leverage PGAS and one-sided communication for building data infrastructure and analytics frameworks on HPC systems. Specifically, we have developed SHMEMCache, a distributed in-memory key-value store and SHMEMGraph, a balanced graph processing framework. We have tackled unique challenges in data consistency, load balancing, etc. with novel design features. The experimental results show that SHMEMCache achieves significant performance improvements over other KV stores such as Memcached in terms of latency and throughput. In addition, SHMEMGraph achieves an average improvement of 35.5% in terms of job completion time compared to a state-of-the-art graph processing system called Gemini.

Summary: pdf
Thesis Canvas: pdf



Back to Doctoral Showcase Archive Listing