PMIx: Enabling Workflow Orchestration

Authors: Ralph Castain (Intel Corporation), Joshua Hursey (IBM)

Abstract: The PMI Exascale (PMIx) community will be concluding its fourth year that included release of the PMIx v3 Standard. We will briefly review the last year's accomplishments, but the majority of the meeting will focus on how community members are using PMIx at the application level. Examples are expected to include population modeling and swarm intelligence; coordination of resource utilization between programming libraries; orchestration of dynamic workflows; and recovery from failures.

Please join the discussion as we explore the expanding uses of PMIx beyond the traditional MPI/OpenSHMEM programming models.

Long Description: PMI-Exascale (PMIx) provides an extended version of the PMI definition specifically designed to support clusters up to and including exascale sizes. The community's efforts include:

(a) augmenting the PMI-1 and PMI-2 APIs to eliminate current restrictions that impact scalability;

(b) extending the capability of applications to interact with the RM;

(c) providing an optional standalone "reference" library (including both client and server support) to ease adoption of the desired capabilities while removing licensing issues that exist in some current implementations.

(d) providing a companion PMIx Reference RTE (PRRTE) for development of PMIx-enabled applications and tools

Last year's BOF provided descriptions of the v2.0 features. This year, we will cover the v3.0 features, but also spend more time discussing the status of the reference implementation. Planned topics include:

* new support for system inventory collection and fabric endpoint assignment

* extended support for logging status information

* cross-library integration for OpenMP/MPI and other hybrid models

* enhanced debugger/tool support

* positioning of files for use by the application or another job step within the same allocated session;

* storage policies such as hot/warm/cold locations, burst buffer management, and persistence of files and/or shared memory regions across job steps within the same allocated session; and

Our objectives for the BOF remain to present the current state of the PMIx effort, describe its planned directions, and stimulate a discussion regarding desired features and other elements of the roadmap. Here are some of the highlights of what will be covered during the discussions:

- An overview of PMIx - State-of-the-Union of RM and programming model support - PMIx v3.0 status and plans - Where do we go from here?

We consider community interaction vital to the future of PMIx and the development of the project's roadmap. Be part of the discussion: come contribute to the roadmap and see how you can (and should!) join our efforts.

URL: https://pmix.org



Back to Birds of a Feather Archive Listing