25 years of HPC innovation – IBM Spectrum LSF 10

Even after 25 years, our high-performance computing (HPC) innovations show no signs of slowing. Following the launch of IBM Spectrum LSF Suites Version 10.2, and our acceptance of the 2017 HPCwire Editors’ Choice award for Best HPC Cluster Solution or Technology, we have released the next update to our core scheduler, IBM Spectrum LSF Version 10.1.0.4.

This update generally delivers the capabilities that are being deployed at Oak Ridge National Lab and Lawrence Livermore National Lab as part of the US Department of Energy’s CORAL program, including support for IBM POWER9-based systems and additional capabilities for managing job reservations.

“We are integrating LSF capabilities with facility awareness to influence scheduling decisions,” says Jim Rogers, Director of Computing and Facilities for the Oak Ridge National Laboratory’s National Center for Computational Sciences. “With the ability to exchange information, we expect increased operational effectiveness.”

In addition to over a dozen customer enhancement requests, this release extends the Spectrum LSF resource connector, which provides intelligent dynamic hybrid cloud capabilities to support Google Compute Engine.  The Spectrum LSF resource connector dynamically provision resources across a variety of cloud providers in response to workloads and return them when no longer required.  The computing cluster grows and shrinks intelligently in response to demand.  Additionally, Spectrum LSF now leverages Elasticsearch (included with IBM Spectrum LSF Explorer) as a backend data store for job and energy accounting.

Lastly, a preview of an integration between IBM Spectrum LSF and IBM Spectrum Scale is included that provides end-to-end data provenance, allowing a user to look at files and trace them back through multiple jobs. The job output files are automatically tagged with file creation metadata including job details, container info, input files and more. This is important in many industries where repeatability of results is required.  For a closer look at the end-to-end data provenance preview, refer to the following technical article.

Find out more about IBM Spectrum LSF and high-performance computing solutions in this detailed hands-on video series.