My PhD thesis : « Multi-players Bandit Algorithms for Internet of Things Networks »

This repository contains the LaTeX code of my PhD thesis, written by Lilian Besson, entitled "Multi-players Bandit Algorithms for Internet of Things Networks".

Calendar

I started to think about the outline in January 2019, I started to play with the template end of January, and to seriously write middle of February 2019.
I wrote intensely in March and May 2019.
I finished the introduction and conclusion in June 2019.
I sent the thesis to the reviewers on the 12th of July 2019.
I received the first feedback on the 28th of August, and the second feedback on the 12th of September 2019.
I defended on the 20th of November 2019, and this repository was open-sourced on the 18th of November, just because I was too impatient to wait.

Where & when?

My defense happened the 20th of November 2019, at 10h30, in CentraleSupélec (campus de Rennes). Room UBL. Direction to come?
It went very well, I am now proud to hold a Doctorate of Philosophy (PhD), in Telecommunications!

See https://perso.crans.org/besson/phd/defense/ this page to get more details for coming to my defense!

Advisors, reviewers, jury

Thanks to my advisors: Christophe Moy and Émilie Kaufmann.
Thanks to the reviewers: Vianney Perchet and Nathalie Mitton.
Thanks to the dear member of my jury reviewers: Patrick Maillé, Raphaël Féraud and Richard Combes.

About this thesis

🇬🇧 In English

Title

Multi-players Bandit Algorithms for Internet of Things Networks.

Summary

In this PhD thesis, we study wireless networks and reconfigurable end-devices that can access Cognitive Radio networks, in unlicensed bands and without central control. We focus on Internet of Things networks (IoT), with the objective of extending the devices' battery life, by equipping them with low-cost but efficient machine learning algorithms, in order to let them automatically improve the efficiency of their wireless communications. We propose different models of IoT networks, and we show empirically on both numerical simulations and real-world validation the possible gain of our methods, that use Reinforcement Learning. The different network access problems are modeled as Multi-Armed Bandits (MAB), but we found that analyzing the realistic models was intractable, because proving the convergence of many IoT devices playing a collaborative game, without communication nor coordination is hard, when they all follow random active pattern. The rest of this manuscript thus studies two restricted models, first multi-players bandits in stationary problems, then non-stationary single-player bandits. We also detail another contribution, SMPyBandits, our open-source Python library for numerical MAB simulations, that covers all the studied models and more.

Keywords

Internet of Things (IoT), Cognitive Radio, Learning Theory, Collision Mitigation Sequential Learning, Reinforcement Learning, Multi-Armed Bandits (MAB), Decentralized Learning, Multi-Player Multi-Armed Bandits, Change Point Detection, Non-Stationary Multi-Armed Bandits.

🇫🇷 En français

Titre

Algorithmes de Bandits Multi-Joueurs pour les Réseaux de l'Internet des Objets.

Résumé

Dans cette thèse de doctorat, nous étudions les réseaux sans fil et les appareils reconfigurables qui peuvent accéder à des réseaux de type radio intelligente, dans des bandes non licenciées et sans supervision centrale. Nous considérons des réseaux actuels ou futurs de l'Internet des Objets (IoT), avec l'objectif d'augmenter la durée de vie de la batterie des appareils, en les équipant d'algorithmes d'apprentissage machine peu coûteux mais efficaces, qui leur permettent d'améliorer automatiquement l'efficacité de leurs communications sans fil. Nous proposons deux modèles de réseaux IoT, et nous montrons empiriquement, par des simulations numériques et une validation expérimentale réaliste, le gain que peuvent apporter nos méthodes, qui se reposent sur l'apprentissage par renforcement. Les différents problèmes d'accès au réseau sont modélisés avec des Bandits Multi-Bras (MAB), mais leur analyse est difficile à réaliser, car il est délicat de prouver la convergence d'un grand nombre d'appareils jouant à un jeu collaboratif sans communication ni aucune coordination, lorsque les appareils suivent tous un modèle d'activation aléatoire. Le reste de ce manuscrit étudie donc deux modèles restreints, d'abord des bandits multi-joueurs dans des problèmes stationnaires, puis des bandits mono-joueur non stationnaires. Nous détaillons également une autre contribution, la bibliothèque Python open-source SMPyBandits, qui permet des simulations numériques de problèmes MAB, qui couvre les modèles étudiés et d'autres.

Mot clés

Internet des Objets (IoT), Radio Intelligente, Théorie de l'apprentissage, Apprentissage séquentiel de l'atténuation des collisions, Apprentissage par renforcement, Bandits multi-bras (MAB), Apprentissage décentralisé, Bandits multi-bras multi-joueurs, Détection des points de changement, Bandits multi-bras non stationnaires.

How to compile the PDF?

You will need a complete TeX distribution, like TeX Live, and PDFLaTeX, as well as GNU make, git, and latexmk,
First, git clone this repository,

$ git clone https://github.com/Naereen/phd-thesis/
$ cd phd-thesis/

Then, simply run make pdf (or latexmk -pdflatex=pdflatex -pdf PhD_thesis__Lilian_Besson.tex)

$ make pdf

It should produce the file PhD_thesis__Lilian_Besson.pdf (PDF/X for online version).
It was only tested on a GNU/Linux machine, under XUbuntu 18.04.

How to compile to get a printable version?

In PhD_thesis__Lilian_Besson.tex, add the print option to the \documentclass{0-Misc/PhDThesisPSnPDF} line.

How to compile to get a PDF/A archivable version?

😭 I didn't succeed in finding the correct options to the pdfx LaTeX package (not like Jill-Jênn Vie) to compile my thesis to a PDF/A archivable PDF.
But I managed to fix my PDF and make it a PDF/A archivable document by using the online service hosted online by Facile.Cines.fr.