/masters-thesis

Master's thesis - Assessment of cognitive load in extreme environment

Primary LanguageJupyter NotebookCreative Commons Attribution Share Alike 4.0 InternationalCC-BY-SA-4.0

Assessment of cognitive load in extreme environment

License: MIT

This is my master's thesis that I wrote during my final year of studying the biomedical engineering program at the Faculty of Biomedical Engineering, CTU in Prague. The work was made possible thanks to the Hydronaut project under the auspices of the European Space Agency (ESA). Unfortunately, the whole work is written in the Czech language. I may translate it into English in the future.

Abstract

The thesis focuses on the assessment of cognitive load in extreme environments, which is critical for the success and safety of individuals and teams performing demanding and essential tasks. Traditional monitoring methods using questionnaires or behavioral analysis may be impractical or even impossible in extreme conditions. For this reason, there is a growing interest in using peripheral biosignals for real-time cognitive load assessment. Specifically, the thesis examines the impact of extreme environments, such as an analog space mission, on the manifestations of cognitive load in electrical cardiac, respiratory, and electrodermal activity. To assess the cognitive load, a new multimodal approach is introduced based on the creation of physiological features in the form of multivariate spatiotemporal causal patterns, allowing for a unique encoding of specific cognitive states. A capsular neural network is designed for synergic uniform integration of the physiological features to capture spatiotemporal causal relations by exploiting autoencoder compression capability. The proposed solution is tested on popular publicly available benchmark datasets, including data from an analog space mission.

Experimental part of the work

This thesis benefits from the second space analogue mission (mission DIANA), which simulated a lunar landing and was carried out as part of the Hydronaut project in the summer of 2022. The individual compartments of the mission had the following roles: the control tower was the station on Earth, the MotherShip orbited the lunar orbit, and the landing module (Lander) was on the surface of the Moon. The individual compartments can be seen in the Figure 1.

Figure 1 - Detail, location and individual mission compartments (map source: Mapy.cz)

A six-member crew was selected for the DIANA mission. Three individuals on the floating platform (MotherShip) and another three for the underwater station (Lander). Given that the underwater habitat involved a long-term saturation dive, the Lander crew was made up of professionally trained divers. The mission primarily served to examine the influence of personality characteristics and external factors on team dynamics during a long-term stay in an ICE environment. The whole experimental part is described in detail in the thesis itself.

Figure 2 - H03 DeepLab and its schematic (source: Hydronaut Project a.s.)

Novel method for cognitive load assessment using multivariate spatiotemporal causal patterns

A multimodal approach to assess cognitive load using peripheral biosignals, specifically using electrical cardiac, respiratory and electrodermal activity, was developed as part of the thesis work. The Copula-Granger approach with Lasso (ℓ1) regularization was chosen to capture time-causal relationships in the used biosignals, which combines the concept of Granger causality with the theory of copulas. However, it cannot be assumed that this captures all of the complex dynamics present in biosignals. This problem is compensated for by using Gramian Angular Fields. These patterns capture a certain kind of temporal and spatial correlation within physiological signals.

Figure 3 - Scheme of physiological features creation

For the purposes of classification tasks (cognitive load assessment), a capsular neural network architecture based on the solution presented by Mazzia et al. (2021), Efficient-CapsNet, was proposed. To synergistically unify the generated physiological features, autoencoder compression into a single latent space is used to capture spatiotemporal causal relations across biosignals.

Figure 4 - Schematic of the proposed capsular neural network model based on the Efficient-CapsNet architecture. Each convolutional layer in the proposed solution is additionally followed by BatchNormalization, MaxPool2D and Dropout layers. (scheme adapted from Wang et al. (2021))

Results

The proposed solution (cognitive load recognition and classification) in this work was compared to selected existing approaches that utilize machine learning for cognitive load assessment. The comparison can be seen in the table below.

Author Dataset Window Test accuracy Year
Feradov et al. CLAS 30s 84% 2020
Ahmad et al. WESAD 1s 72% 2021
Garg et al. WESAD 10s 84% 2021
Rabbani et al. WESAD 10s 87% 2022
Zhu et al. CLAS 30s 86% 2022
Quadrini et al. WESAD 60s 91% 2022
Chatterjee et al. WESAD 30s 94% 2022
Dair et al. WESAD 10s 69% 2023
Rashid et al. WESAD 30s 93% 2023
Galarraga et al. WESAD 20s 90% 2023
Henry et al. WESAD 6s 81% 2023
Yong et al. CLAS 5s 75% 2023
Singh et al. WESAD 5s 91% 2023
Author of the work WESAD 5s 97% 2023
Author of the work CLAS 5s 92% 2023
Author of the work WESAD 1s 95% 2023
Author of the work WESAD 1s 92% 2023

Acknowledgements

I would like to thank the supervisor of my master thesis, Mgr. Ksenia Sedová, Ph.D. for help, advice, and professional management of this work. I would also like to thank Ing. et Ing. Jan Hejda, Ph.D., for all-around help, a lot of valuable and inspiring advice, suggestions adn recommendations. Last but not least, I thank my family and all the friends who supported me in creating this work.

Contributors

  • Marek Sokol - Author
  • Mgr. Ksenia Sedova, Ph.D. - Supervisor
  • Ing. et Ing. Jan Hejda, Ph.D. - Consultant