/SER-datasets

A collection of datasets for the purpose of emotion recognition/detection in speech.

Primary LanguageHTMLMIT LicenseMIT

Spoken Emotion Recognition Datasets: A collection of datasets (count=43) for the purpose of emotion recognition/detection in speech. The table is chronologically ordered and includes a description of the content of each dataset along with the emotions included. The table can be browsed, sorted and searched under https://superkogito.github.io/SER-datasets/

Dataset Year Content Emotions Format Size Language Paper Access License
MESD 2022 864 audio files of single-word emotional utterances with Mexican cultural shaping. 6 emotions provides single-word utterances for anger, disgust, fear, happiness, neutral, and sadness. Audio 0,097 GB Spanish (Mexican) The Mexican Emotional Speech Database (MESD): elaboration and assessment based on machine learning Open CC BY 4.0
SyntAct 2022 Synthesized database of three basic emotions and neutral expression based on rule-based manipulation for a diphone synthesizer which we release to the public 997 utterances including 6 emotions: angry, bored, happy, neutral, sad and scared Audio 941 MB German SyntAct: A Synthesized Database of Basic Emotions Open CC BY-SA 4.0
MLEnd 2021 ~32700 audio recordings files produced by 154 speakers. Each audio recording corresponds to one English numeral (from "zero" to "billion") Intonations: neutral, bored, excited and question Audio 2.27 GB -- -- Open Unknown
ASVP-ESD 2021 ~13285 audio files collected from movies, tv shows and youtube containing speech and non-speech. 12 different natural emotions (boredom, neutral, happiness, sadness, anger, fear, surprise, disgust, excitement, pleasure, pain, disappointment) with 2 levels of intensity. Audio 2 GB Chinese, English, French, Russian and others -- Open Unknown
ESD 2021 29 hours, 3500 sentences, by 10 native English speakers and 10 native Chinese speakers. 5 emotions: angry, happy, neutral, sad, and surprise. Audio, Text 2.4 GB (zip) Chinese, English Seen And Unseen Emotional Style Transfer For Voice Conversion With A New Emotional Speech Dataset Open Academic License
MuSe-CAR 2021 40 hours, 6,000+ recordings of 25,000+ sentences by 70+ English speakers (see db link for details). continuous emotion dimensions characterized using valence, arousal, and trustworthiness. Audio, Video, Text 15 GB English The Multimodal Sentiment Analysis in Car Reviews (MuSe-CaR) Dataset: Collection, Insights and Improvements Restricted Academic License & Commercial License
MSP-Podcast corpus 2020 100 hours by over 100 speakers (see db link for details). This corpus is annotated with emotional labels using attribute-based descriptors (activation, dominance and valence) and categorical labels (anger, happiness, sadness, disgust, surprised, fear, contempt, neutral and other). Audio -- -- The MSP-Conversation Corpus Restricted Academic License & Commercial License
emotiontts open db 2020 Recordings and their associated transcriptions by a diverse group of speakers. 4 emotions: general, joy, anger, and sadness. Audio, Text -- Korean -- Partially open CC BY-NC-SA 4.0
URDU-Dataset 2020 400 utterances by 38 speakers (27 male and 11 female). 4 emotions: angry, happy, neutral, and sad. Audio 0.072 GB Urdu Cross Lingual Speech Emotion Recognition: Urdu vs. Western Languages Open --
BAVED 2020 1935 recording by 61 speakers (45 male and 16 female). 3 levels of emotion. Audio 0.195 GB Arabic -- Open --
VIVAE 2020 non-speech, 1085 audio file by 12 speakers. non-speech 6 emotions: achievement, anger, fear, pain, pleasure, and surprise with 3 emotional intensities (low, moderate, strong, peak). Audio -- -- -- Restricted CC BY-NC-SA 4.0
SEWA 2019 more than 2000 minutes of audio-visual data of 398 people (201 male and 197 female) coming from 6 cultures. emotions are characterized using valence and arousal. Audio, Video -- Chinese, English, German, Greek, Hungarian and Serbian SEWA DB: A Rich Database for Audio-Visual Emotion and Sentiment Research in the Wild Restricted SEWA EULA
MELD 2019 1400 dialogues and 14000 utterances from Friends TV series by multiple speakers. 7 emotions: Anger, disgust, sadness, joy, neutral, surprise and fear. MELD also has sentiment (positive, negative and neutral) annotation for each utterance. Audio, Video, Text 10.1 GB English MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations Open MELD: GPL-3.0 License
ShEMO 2019 3000 semi-natural utterances, equivalent to 3 hours and 25 minutes of speech data from online radio plays by 87 native-Persian speakers. 6 emotions: anger, fear, happiness, sadness, neutral and surprise. Audio 0.101 GB Persian ShEMO: a large-scale validated database for Persian speech emotion detection Open --
DEMoS 2019 9365 emotional and 332 neutral samples produced by 68 native speakers (23 females, 45 males). 7/6 emotions: anger, sadness, happiness, fear, surprise, disgust, and the secondary emotion guilt. Audio -- Italian DEMoS: An Italian emotional speech corpus. Elicitation methods, machine learning, and perception Restricted EULA: End User License Agreement
AESDD 2018 around 500 utterances by a diverse group of actors (over 5 actors) siumlating various emotions. 5 emotions: anger, disgust, fear, happiness, and sadness. Audio 0.392 GB Greek Speech Emotion Recognition for Performance Interaction Open --
Emov-DB 2018 Recordings for 4 speakers- 2 males and 2 females. The emotional styles are neutral, sleepiness, anger, disgust and amused. Audio 5.88 GB English The emotional voices database: Towards controlling the emotion dimension in voice generation systems Open --
RAVDESS 2018 7356 recordings by 24 actors. 7 emotions: calm, happy, sad, angry, fearful, surprise, and disgust Audio, Video 24.8 GB English The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American English Open CC BY-NC-SA 4.0
JL corpus 2018 2400 recording of 240 sentences by 4 actors (2 males and 2 females). 5 primary emotions: angry, sad, neutral, happy, excited. 5 secondary emotions: anxious, apologetic, pensive, worried, enthusiastic. Audio -- English An Open Source Emotional Speech Corpus for Human Robot Interaction Applications Open CC0 1.0
CaFE 2018 6 different sentences by 12 speakers (6 fmelaes + 6 males). 7 emotions: happy, sad, angry, fearful, surprise, disgust and neutral. Each emotion is acted in 2 different intensities. Audio 2 GB French (Canadian) -- Open CC BY-NC-SA 4.0
EmoFilm 2018 1115 audio instances sentences extracted from various films. 5 emotions: anger, contempt, happiness, fear, and sadness. Audio -- English, Italian & Spanish Categorical vs Dimensional Perception of Italian Emotional Speech Restricted EULA: End User License Agreement
ANAD 2018 1384 recording by multiple speakers. 3 emotions: angry, happy, surprised. Audio 2 GB Arabic Arabic Natural Audio Dataset Open CC BY-NC-SA 4.0
EmoSynth 2018 144 audio file labelled by 40 listeners. Emotion (no speech) defined in regard of valence and arousal. Audio 0.1034 GB -- The Perceived Emotion of Isolated Synthetic Audio: The EmoSynth Dataset and Results Open CC BY 4.0
CMU-MOSEI 2018 65 hours of annotated video from more than 1000 speakers and 250 topics. 6 Emotion (happiness, sadness, anger,fear, disgust, surprise) + Likert scale. Audio, Video -- English Multi-attention Recurrent Network for Human Communication Comprehension Open CMU-MOSEI License
VERBO 2018 14 different phrases by 12 speakers (6 female + 6 male) for a total of 1167 recordings. 7 emotions: Happiness, Disgust, Fear, Neutral, Anger, Surprise, Sadness Audio -- Portuguese VERBO: Voice Emotion Recognition dataBase in Portuguese Language Restricted Available for research purposes only
CMU-MOSI 2017 2199 opinion utterances with annotated sentiment. Sentiment annotated between very negative to very positive in seven Likert steps. Audio, Video -- English Multi-attention Recurrent Network for Human Communication Comprehension Open CMU-MOSI License
MSP-IMPROV 2017 20 sentences by 12 actors. 4 emotions: angry, sad, happy, neutral, other, without agreement Audio, Video -- English MSP-IMPROV: An Acted Corpus of Dyadic Interactions to Study Emotion Perception Restricted Academic License & Commercial License
CREMA-D 2017 7442 clip of 12 sentences spoken by 91 actors (48 males and 43 females). 6 emotions: angry, disgusted, fearful, happy, neutral, and sad Audio, Video -- English CREMA-D: Crowd-sourced Emotional Multimodal Actors Dataset Open Open Database License & Database Content License
Example emotion videos used in investigation of emotion perception in schizophrenia 2017 6 videos:Two example videos from each emotion category (angry, happy and neutral) by one female speaker. 3 emotions: angry, happy and neutral. Audio, Video 0.063 GB English -- Open Permitted Non-commercial Re-use with Acknowledgment
EMOVO 2014 6 actors who played 14 sentences. 6 emotions: disgust, fear, anger, joy, surprise, sadness. Audio 0.355 GB Italian EMOVO Corpus: an Italian Emotional Speech Database Open --
RECOLA 2013 3.8 hours of recordings by 46 participants. negative and positive sentiment (valence and arousal). Audio, Video -- -- Introducing the RECOLA Multimodal Corpus of Remote Collaborative and Affective Interactions Restricted Academic License & Commercial License
GEMEP corpus 2012 Videos10 actors portraying 10 states. 12 emotions: amusement, anxiety, cold anger (irritation), despair, hot anger (rage), fear (panic), interest, joy (elation), pleasure(sensory), pride, relief, and sadness. Plus, 5 additional emotions: admiration, contempt, disgust, surprise, and tenderness. Audio, Video -- French Introducing the Geneva Multimodal Expression Corpus for Experimental Research on Emotion Perception Restricted --
OGVC 2012 9114 spontaneous utterances and 2656 acted utterances by 4 professional actors (two male and two female). 9 emotional states: fear, surprise, sadness, disgust, anger, anticipation, joy, acceptance and the neutral state. Audio -- Japanese Naturalistic emotional speech collectionparadigm with online game and its psychological and acoustical assessment Restricted --
LEGO corpus 2012 347 dialogs with 9,083 system-user exchanges. Emotions classified as garbage, non-angry, slightly angry and very angry. Audio 1.1 GB -- A Parameterized and Annotated Spoken Dialog Corpus of the CMU Let’s Go Bus Information System Open License available with the data. Free of charges for research purposes only.
SEMAINE 2012 95 dyadic conversations from 21 subjects. Each subject converses with another playing one of four characters with emotions. 5 FeelTrace annotations: activation, valence, dominance, power, intensity Audio, Video, Text 104 GB English The SEMAINE Database: Annotated Multimodal Records of Emotionally Colored Conversations between a Person and a Limited Agent Restricted Academic EULA
SAVEE 2011 480 British English utterances by 4 males actors. 7 emotions: anger, disgust, fear, happiness, sadness, surprise and neutral. Audio, Video -- English (British) Multimodal Emotion Recognition Restricted Free of charges for research purposes only.
TESS 2010 2800 recording by 2 actresses. 7 emotions: anger, disgust, fear, happiness, pleasant surprise, sadness, and neutral. Audio -- English BEHAVIOURAL FINDINGS FROM THE TORONTO EMOTIONAL SPEECH SET Open CC BY-NC-ND 4.0
EEKK 2007 26 text passage read by 10 speakers. 4 main emotions: joy, sadness, anger and neutral. -- 0.352 GB Estonian Estonian Emotional Speech Corpus Open CC-BY license
IEMOCAP 2007 12 hours of audiovisual data by 10 actors. 5 emotions: happiness, anger, sadness, frustration and neutral. -- -- English IEMOCAP: Interactive emotional dyadic motion capture database Restricted IEMOCAP license
Keio-ESD 2006 A set of human speech with vocal emotion spoken by a Japanese male speaker. 47 emotions including angry, joyful, disgusting, downgrading, funny, worried, gentle, relief, indignation, shameful, etc. Audio -- Japanese EMOTIONAL SPEECH SYNTHESIS USING SUBSPACE CONSTRAINTS IN PROSODY Restricted Available for research purposes only.
EMO-DB 2005 800 recording spoken by 10 actors (5 males and 5 females). 7 emotions: anger, neutral, fear, boredom, happiness, sadness, disgust. Audio -- German A Database of German Emotional Speech Open --
eNTERFACE05 2005 Videos by 42 subjects, coming from 14 different nationalities. 6 emotions: anger, fear, surprise, happiness, sadness and disgust. Audio, Video 0.8 GB German -- Open Free of charges for research purposes only.
DES 2002 4 speakers (2 males and 2 females). 5 emotions: neutral, surprise, happiness, sadness and anger -- -- Danish Documentation of the Danish Emotional Speech Database -- --
  • Swain, Monorama & Routray, Aurobinda & Kabisatpathy, Prithviraj, Databases, features and classifiers for speech emotion recognition: a review, International Journal of Speech Technology, paper
  • Dimitrios Ververidis and Constantine Kotropoulos, A State of the Art Review on Emotional Speech Databases, Artificial Intelligence & Information Analysis Laboratory, Department of Informatics Aristotle, University of Thessaloniki, paper
  • A. Pramod Reddy and V. Vijayarajan, Extraction of Emotions from Speech-A Survey, VIT University, International Journal of Applied Engineering Research, paper
  • Emotional Speech Databases, document
  • Expressive Synthetic Speech, website
  • Towards a standard set of acoustic features for the processing of emotion in speech, Technical university Munich, document

Contribution

  • All contributions are welcome! If you know a dataset that belongs here (see criteria) but is not listed, please feel free to add it. For more information on Contributing, please refer to CONTRIBUTING.md.

  • If you notice a typo or a mistake, please report this as an issue and help us improve the quality of this list.

Disclaimer

  • The mainter and the contributors try their best to keep this list up-to-date, and to only include working links (using automated verification with the help of the urlchecker-action). However, we cannot guarantee that all listed links are up-to-date. Read more in DISCLAIMER.md.