Presentation
SP-Cache: Load-Balanced, Redundancy-Free Cluster Caching with Selective Partition
SessionData and Storage
Event Type
Paper
TP
Clouds and Distributed Computing
File Systems
I/O
Storage
TimeTuesday, November 13th10:30am - 11am
LocationC146
DescriptionData-intensive clusters increasingly employ in-memory solutions to improve I/O performance. However, the routinely observed file popularity skew and load imbalance create hotspots, which significantly degrades the benefits of in-memory solutions. Common approaches to tame load imbalance include copying multiple replicas of hot files and creating parity chunks using storage codes. Yet, these techniques either suffer from high memory redundancy or incur non-trivial encoding/decoding overhead. In this paper, we propose a different approach to achieve load balancing without memory redundancy or encoding/decoding overhead. Our solution, termed SP-Cache, selectively partitions files based on their popularity and evenly caches those partitions across the cluster. We develop an efficient algorithm to determine the optimal number of partitions for hot files—too few partitions are incapable of mitigating hotspots, while too many are susceptible to stragglers. EC2 deployment and trace-driven simulations show that, compared with existing solutions, SP-Cache reduces the read latencies by up to 40%.
Download PDF
Archive
