Large Scale Computation of Quantiles Using MELISSA

Authors: Alejandro Ribes (EDF Research and Development), Théophile Terraz (French Institute for Research in Computer Science and Automation (INRIA)), Yvan Fournier (EDF Research and Development), Bertrand Iooss (EDF Research and Development), Bruno Raffin (French Institute for Research in Computer Science and Automation (INRIA))

Abstract: Quantiles being order statistics, the classical approach for their computation requires availability of the full sample before ranking it. This approach is not suitable at exascale. Large ensembles would need to gather a prohibitively large amount of data. We propose an iterative approach based on the stochastic quantile algorithm of Robbins-Monro. We rely on the Melissa framework, a file avoiding, adaptive, fault tolerant and elastic framework in order to compute in transit ubiquitous quantiles. Quantiles are updated on-the-fly as soon as the in transit parallel server receives results from one of the running simulations. We run 80,000 fluid dynamics parallel simulations of 6M hexahedra and 100 timespteps. They were executed on up to 4800 cores, avoiding 288 TB of file storage. We produce ubiquitous spatio-temporal maps of quantiles and inter-quantile based intervals.

Best Poster Finalist (BP): no

Poster: pdf
Poster summary: PDF


Back to Poster Archive Listing