<span class="var-sub_title">A Statistical Analysis of Compressed Climate Model Data</span> SC18 Proceedings

The International Conference for High Performance Computing, Networking, Storage, and Analysis

The 4th International Workshop on Data Reduction for Big Scientific Data (DRBSD-4)


A Statistical Analysis of Compressed Climate Model Data

Authors: Dorit Hammerling (National Center for Atmospheric Research)

Abstract: The data storage burden resulting from large climate model simulations continues to grow. While lossy data compression methods can alleviate this burden, they introduce the possibility that key climate variables could be altered to the point of affecting scientific conclusions. Therefore, developing a detailed understanding of how compressed model output differs from the original is important. Here, we evaluate the effects of two leading compression algorithms, SZ and ZFP, on daily surface temperature and precipitation rate data from a popular climate model. While both algorithms show promising fidelity with the original output, detectable artifacts are introduced even at relatively low error tolerances. This study highlights the need for evaluation methods that are sensitive to errors at different spatiotemporal scales and specific to the particular climate variable of interest, with the ultimate goal to improve lossy compression collaboratively with the algorithm development teams.

Archive Materials


Back to The 4th International Workshop on Data Reduction for Big Scientific Data (DRBSD-4) Archive Listing

Back to Full Workshop Archive Listing