An Opportunity for Lustre Based Computational Storage
Professor of Weather & Climate Computing
University of Reading
In the opening session, Dr. Bryan Lawrence shared his experiences from the ExcaliData project, and an implementation of Active Storage which could help to accelerate some types of HPC codes by delegating certain computation tasks to storage subsystems. Bryan reviewed initial implementations for S3 (Fuse) and Posix (DDN Infinia), and looks at the steps that might be needed to introduce into Lustre.
Student Cluster Challenge
As part of CIUK’s annual cluster challenge, teams of students were given three hours to analyze the performance of a Lustre system, and research the best strategies to optimize and document their findings. As part of the LUG UK event, the best six teams were given the opportunity to share their experiences. After a fast-paced series of five-minute lightning sessions, the LUG UK audience were able to vote for the best presentation. Which was your favourite? Check out the results at the end of this page!
Find out more information about CIUK Cluster Challenge.
Lustre on Azure: Optimizing Performance & Costs in the Cloud
After the break, Dr. Renier Dryer, CEO of CrunchYard, looked at the challenges of optimizing performance and costs of a Lustre cloud application running in Microsoft Azure. Using SAS Grid, an application often used in financial services industry, Renier compared the Lustre workloads supported by different types of Azure instance, when using the configurations recommended by SAS for secure and resilient service deployment on Lustre.
Lustre Community & Roadmap
Senior Director of Engineering
Peter Jones is the Senior Director of Engineering at Whamcloud, responsible for the roadmap and development strategy for Lustre. Peter shared the latest survey data for deployment of Lustre 2.15, and outlined the release schedule for Lustre 2.16 and LTS options, as well as key features expected in forthcoming releases.
Client-Side Data Compression with Lustre
Senior Lustre FS Engineer
Finally, Artem Blagodarenko, Senior Engineer from the Lustre Filesystem team at Whamcloud, presented details of Client-Side Data Compression capability in Lustre, expected to be widely available in Lustre 2.17. Artem looked at the metrics from the most recent test releases, including CPU loading, compression ratios for different datatypes, and previewed the tools and command line utilities which will support this new feature.
Student Cluster Challenge Results
After a fast-paced series of five minute sessions, the LUG UK audience were able to vote for the best presentation. Which was your favourite?
Congratulations to team ClusDur, the winners of the presentation section of this challenge – followed closely by Isambards, and in third place for presentations were Parallel Pioneers Parallel Pioneers. The results of this vote were credited towards their score for Challenge 6, out of a total of 8 separate challenges.
Congratulations also to Team Isambards from University of Bristol, who won the overall contest as CIUK 2023 Cluster Challenge Champions.
Thanks to Chris Coates, Head of Engineering and HPC at Logicalis for helping to arrange this session – for more information on the CIUK Cluster Challenge, and details of the other teams and challenges, see https://www.scd.stfc.ac.uk/Pages/CIUK-2023-Cluster-Challenge.aspx