High Performance Computing (HPC) Parallel Programming Load Balancing

Technology Readiness Level: 6

*As assessed with the Human Brain Project's TRL Guide

Technology Description

DLB aims at optimizing the performance of hybrid applications without a previous analysis or modifying the code. Thus, DLB will improve the load balance of the outer level of parallelism by redistributing the computational resources at the inner level of parallelism. This readjustment of resources, performed done dynamically at runtime, allows DLB to react to different sources of imbalance: algorithm, data, hardware architecture and resource availability, among others.

DLB library is devoted to speed up hybrid parallel applications and maximize the utilization of computational resources since 2009

The efficient utilization of computational resources is crucial for HPC applications and systems

Competitive Advantages

A dynamic library transparent to the user:

  • No need to analyze nor modify the application
  • Transversal to the different layers of the HPC software stack
  • Compatible with MPI, OpenMP and OmpSs

Maximizes the utilization of computational resources:

  • Obtaining an efficiency close to 100%

Improves the load balance of hybrid applications resulting in:

  • 2x times speed up with DLB simulating the particle transport in the human respiratory system up to 16k cores using Alya
  • 5x times speed up of a relational databases process using DLB with Cybeletech
WITHOUT DLB
WITH DLB

Applications and Market Potential

HPC applications: aimed at obtaining the maximum efficiency from computational resources

System administrators: specially those of many-core systems, to maximize the utilization of the system

Parallel programming models and resource managers: to coordinate with other layers of the software stack

Hybrid applications: to address load imbalances at the outer level of parallelism

Interesting Facts

  • Current stable version 2.0.2 released on October 2018 
  • Used in several EU projects (TEXT, Mont-blanc3, HPCEurope3, Human Brain Project, EOCOEII)
  • Used by spin-off ELEM Biotech
  • Collaboration with CybeleTech company