Scheduling of malleable HPC applications using a Particle Swarm optimised greedy algorithm

banner

This project is a prototype of a simulator supporting the scheduling of malleable jobs along with the shutting down of idle servers for energy saving. Similar works have tackled seperately both the problematics of using malleable jobs to improve the efficiency schedulers on one side, and the data centers energy efficiency on the other side. In our approach we combine both goals to find a solution that optimises both objectives. The paper of this implementation can be found here

Background

This project started as academic research project for the course of Optimisation for Computer Science by Dr. Pascal BOUVRY and Dr. Grégoire DANOY from the University of Luxembourg. It was supervised by Dr. Georges Da Costa from the University of Toulouse. It was made by three MICS students: Nesryne Mejri, Briag Dupont and Lucas Romao.

General goals of this project are:

  • Using a metaheuristic approach to find a solution that optimises the multi-objective cost function.
  • Providing an environment that supports the scheduling of malleable jobs.
  • Using the scientific approach to analyse the obtained results.

Features

The project implements a greedy algorithm that schedules jobs in a FIFO fashion. It also makes use of the Particle Swarm Optimisation to tune the scheduler and let it make decisions of whether to reconfigure a job or to shut down an idle server.

Other features:

  • A realistic job generator based on Google data centers distribution from here.
  • Drawing of the scheduling results as Gantt charts.
  • Allowing job reconfigurations from a lower to a higher number of servers.
  • Automatic documentation generation.
  • Dependency management using Poetry

How to install

  • Clone this project and enter the directory.
  • Execute the setup script by using ./setup.sh

Use

Use the command ./launcher.sh start --help. and select which operation to run
To generate documentation use ./launcher.sh gendoc

Changing parameters.

To produces visuals and/or change the parameters of either the swarm training or the benchmarking experiments, modify the config.yml file

License

This project is distributed under the MIT License