<span class="var-sub_title">LUSTRE Community BOF: Lustre in HPC and Emerging Data Markets: Roadmap, Features and Challenges</span> SC18 Proceedings

The International Conference for High Performance Computing, Networking, Storage, and Analysis

LUSTRE Community BOF: Lustre in HPC and Emerging Data Markets: Roadmap, Features and Challenges


Authors: Dr. Sarp Oral (OpenSFS Inc, Oak Ridge National Laboratory)

Abstract: Lustre is the leading open source file system for HPC. Over 70% of the top 100 supercomputers use Lustre. It is a community developed technology with contributors from around the world. Lustre currently supports many HPC infrastructures beyond scientific research, sch as financial services, oil and gas and advanced manufacturing. Lustre clients are available for variety of technologies, such as x86, POWER, and ARM. At this BOF, Lustre developers, administrators, and solution providers will gather to discuss recent Lustre developments, challenges, and corresponding opportunities for the next 5 years of Lustre, including the role of Lustre in future exascale environments.

Long Description: Lustre is the leading open source file system for HPC. Over 70% of the top 100 supercomputers use Lustre file systems. Lustre today is a community developed file system with contributors from around the world. Lustre supports many HPC infrastructures beyond its traditional stronghold of scientific research including financial services, oil and gas, advanced manufacturing, and visual effects and Lustre clients are available for x86, POWER, and ARM. At this BOF, Lustre developers, administrators, and solution providers will gather to discuss recent developments, such as File Level Redundancy (FLR) and Data on Metadata (DoM) and new challenges and corresponding opportunities for the next 5 years of Lustre, including the role of Lustre in future Exascale environments.

Today Lustre is one of the most widely adopted technologies in HPC, from the Tianhe-2 system at the National Supercomputer Center in China, the Titan system at Oak Ridge National Laboratory, the Sequoia system at Lawrence Livermore National Laboratory, to the K computer at RIKEN Advanced Institute for Computer Science in Kobe, Japan, and more. Lustre is also widely used across mid and small-scale HPC systems with continued adoption attributable to the stability of the Lustre file system as it has matured.

Vital to the technology is the community that continues to drive Lustre forward. As Lustre development has evolved into a true open source model, the end users, developers, and solutions providers have come together through the worldwide OpenSFS and the EOFS communities. This community development model has resulted in significant new features, improved stability and broader adoption.

At the 2018 Lustre BOF we will focus on feature developments and discuss how they will shape the next 5 years of Lustre deployments. From financial services, oil and gas, advanced manufacturing, to basic sciences, Lustre is at the heart of many HPC infrastructures and must continue to evolve to support new use cases. We will explore these use cases and discuss the Lustre roadmap for meeting the requirements they present.

The session will include a town hall meeting open to participants. OpenSFS and EOFS will provide initial discussion topics focusing on a brief set of challenges, such as recent Lustre business acquisitions. The list of challenges will be compiled by the OpenSFS and EOFS by polling the community before the BOF.

The Lustre BOF has been held at previous SC events and has been well attended and enthusiastically received. SC presents a rare opportunity for the worldwide Lustre community to meet and discuss how to make Lustre more successful. Attendance is usually very good with 150 attendees. The Lustre BOF at SC17 had about 190 attendees.

The targeted audience includes all who are involved with commercial and/or community (open source) based Lustre deployments like administrators, system architects, developers, solution providers and end users. It includes all who have been contributing or reviewing new features or providing additional tools for management and control.

A written report summarizing the Lustre Community BOF will be developed and will be made available to the Lustre community.


URL: http://opensfs.org



Back to Birds of a Feather Archive Listing