/ASD-Classification

Classifying whether or not the person with a particular fixation pattern on an image has autism or not.

Primary LanguageJupyter Notebook

Improved Visual Attention Classification for Autism Spectrum Disorder through Time-Dependent Representations.

task image

People with autism spectrum disorder (ASD) tend to attend differently to visual stimuli when compared to neurotypical individuals. These differences in visual attention can be used for the automatic screening of ASD. We classify a given sequence of fixations on a particular image belonging to either a person not having or having ASD. A deep learning framework is proposed which extracts visual feature embeddings around each point of fixation and processes them sequentially using recurrent neural networks (LSTM) [1] or using transformers [2]. We also propose to exploit temporal information of each fixation such as duration using time-masking and time-event joint embedding techniques [3].

With duration-sensitive visual features for each fixation, we achieve an improvement of 11.46% in the F1 score compared to the baseline recurrent neural network without duration encoded into it. We also experiment with transformers and demonstrate their potential for the classification of visual attention.

All experiments are performed on the Saliency4ASD [4] dataset which contains 300 images with around 24 fixations per image where half of them are from people with ASD and the other half from people without ASD. 10% of the data is used for validation and the remaining 90% is used for training. The dataset is available at link.

There are two notebooks in the repo, which can be run on colab to train LSTM and Transformer networks respectively. The notebooks are self-explanatory and can be run without any changes. The data is downloaded using wget library in the notebook itself.
There is also a folder named "Data Preprocessing" which contain notebooks and scripts to preprocess the following datasets:
  1. MIT1003 [5]
  2. CAT2000 [6]
  3. SALICON [7]
  4. VIU [8]

References

  1. Attention-based Autism Spectrum Disorder Screening with Privileged Modality [Chen et al 2019, ICCV]

  2. Attention is all you need [Vaswani et al 2017, NIPS]

  3. Time-Dependent Representation for Neural Event Sequence Prediction [Li et al 2017, ArXiv]

  4. Saliency4ASD: Challenge, dataset and tools for visual attention modeling for autism spectrum disorder [Gutierrez 2021, Signal Process. Image Commun.]

  5. Learning to Predict where Humans Look [Judd et al 2009, ICCV]

  6. CAT2000: A Large Scale Fixation Dataset for Boosting Saliency Research [Itti et al 2015, CVPR]

  7. SALICON: Saliency in Context [Jiang et al 2015, CVPR]

  8. What Do Saliency Models Predict? [Koehler et al 2014, JoV]