cmu-mosei

Created by AmirAli Bagher Zadeh, Paul Pu Liang, Soujanya Poria, Erik Cambria, Louis-Philippe Morency

version

1.2.4

license

CC-BY-NC-4.0

usage

research

languages

English

format

wav

channel

1

sampling rate

16000

bit depth

16

duration

4 days 19:40:44.218500001

files

3293, duration distribution: 6.0 s cmu-mosei-1.2.4-file-duration-distribution 1160.2 s

segments

23259, duration distribution: 0.1 s cmu-mosei-1.2.4-segment-duration-distribution 109.4 s

Description

Multimodal Opinion Sentiment and Emotion Intensity Sentiment and emotion annotated multimodal data automatically collected from YouTube. The dataset contains more than 23,500 sentence utterance videos from more than 1000 online YouTube speakers. The dataset is gender balanced. All the sentences utterance are randomly chosen from various topics and monologue videos. The videos are transcribed and properly punctuated. All videos are stated to have a creative commons license that allows for personal unrestricted use. The annotations have a different less strict license and can be used also for commercial applications. Reference: http://dx.doi.org/10.18653/v1/P18-1208

Example

96337-Audio-WAV/84176.wav

../_images/cmu-mosei-1.2.4-player-waveform.png

Tables

Click on a row to toggle a preview.

ID

Type

Columns

dev

filewise

file

cM1Zuji24dI-Audio-WAV/188343.wav

4HAC00EAdno-Audio-WAV/VAXhC2U9-2A.wav

KPeos6f4HuQ-Audio-WAV/AxNy9TeTLq8.wav

mmg_eTDHjkk-Audio-WAV/EmmuWoCUgXs.wav

HObh42PhOfw-Audio-WAV/icbUzboLcDQ.wav

300 rows x 0 columns

emotion

segmented

happiness, sadness, anger, fear, disgust, surprise

file

start

end

happiness

sadness

anger

fear

disgust

surprise

9zBj8VkRBpE-Audio-WAV/--qXJuDtHPw.wav

0 days 00:00:23.199000

0 days 00:00:30.325000

0.6666667

0.0

0.0

0.0

0.0

0.0

qFsGMA75-oQ-Audio-WAV/-3g5yACwYnA.wav

0 days 00:01:22.753000

0 days 00:01:40.555000

0.6666667

0.6666667

0.0

0.6666667

0.0

0.0

qFsGMA75-oQ-Audio-WAV/-3g5yACwYnA.wav

0 days 00:01:59.919000

0 days 00:02:05.299000

0.0

0.0

0.0

0.0

0.0

0.0

qFsGMA75-oQ-Audio-WAV/-3g5yACwYnA.wav

0 days 00:00:04.840000

0 days 00:00:14.052000

0.6666667

0.6666667

0.0

0.33333334

0.0

0.0

qFsGMA75-oQ-Audio-WAV/-3g5yACwYnA.wav

0 days 00:00:13.211000

0 days 00:00:27.521000

0.33333334

0.33333334

0.0

0.0

0.0

0.0

23259 rows x 6 columns

emotion.presence

segmented

happiness, sadness, anger, fear, disgust, surprise

file

start

end

happiness

sadness

anger

fear

disgust

surprise

9zBj8VkRBpE-Audio-WAV/--qXJuDtHPw.wav

0 days 00:00:23.199000

0 days 00:00:30.325000

True

False

False

False

False

False

qFsGMA75-oQ-Audio-WAV/-3g5yACwYnA.wav

0 days 00:01:22.753000

0 days 00:01:40.555000

True

True

False

True

False

False

qFsGMA75-oQ-Audio-WAV/-3g5yACwYnA.wav

0 days 00:01:59.919000

0 days 00:02:05.299000

False

False

False

False

False

False

qFsGMA75-oQ-Audio-WAV/-3g5yACwYnA.wav

0 days 00:00:04.840000

0 days 00:00:14.052000

True

True

False

True

False

False

qFsGMA75-oQ-Audio-WAV/-3g5yACwYnA.wav

0 days 00:00:13.211000

0 days 00:00:27.521000

True

True

False

False

False

False

23259 rows x 6 columns

sentiment

segmented

sentiment, sentiment.binarized, sentiment.binary, sentiment.binary.old

file

start

end

sentiment

sentiment.binarized

sentiment.binary

sentiment.binary.old

9zBj8VkRBpE-Audio-WAV/--qXJuDtHPw.wav

0 days 00:00:23.199000

0 days 00:00:30.325000

1.0

weakly positive

positive

non-negative

qFsGMA75-oQ-Audio-WAV/-3g5yACwYnA.wav

0 days 00:01:22.753000

0 days 00:01:40.555000

1.0

weakly positive

positive

non-negative

qFsGMA75-oQ-Audio-WAV/-3g5yACwYnA.wav

0 days 00:01:59.919000

0 days 00:02:05.299000

0.6666667

weakly positive

positive

non-negative

qFsGMA75-oQ-Audio-WAV/-3g5yACwYnA.wav

0 days 00:00:04.840000

0 days 00:00:14.052000

0.0

neutral

non-negative

qFsGMA75-oQ-Audio-WAV/-3g5yACwYnA.wav

0 days 00:00:13.211000

0 days 00:00:27.521000

0.0

neutral

non-negative

23259 rows x 4 columns

test

filewise

file

8wdxczYf1jI-Audio-WAV/286943.wav

QEG_hkJsaYc-Audio-WAV/126872.wav

100232-Audio-WAV/qgC8_emxSIU.wav

JATMzuV6sUE-Audio-WAV/kld9r0iFkWM.wav

-egA8-b7-3M-Audio-WAV/rC29Qub0U7A.wav

676 rows x 0 columns

train

filewise

file

XzVapdEr_GY-Audio-WAV/hh04W3xXa5s.wav

GdFP_p4eQX0-Audio-WAV/GdFP_p4eQX0.wav

87MsiC3E2-w-Audio-WAV/4iG0ffmnCOw.wav

75892-Audio-WAV/81406.wav

4HAC00EAdno-Audio-WAV/qyJiDgtj6YE.wav

2249 rows x 0 columns

transcription

segmented

transcription

file

start

end

transcription

9zBj8VkRBpE-Audio-WAV/--qXJuDtHPw.wav

0 days 00:00:23.199000

0 days 00:00:30.325000

i see that a writer is somebody who has an incredible command of mechanics of the english language

qFsGMA75-oQ-Audio-WAV/-3g5yACwYnA.wav

0 days 00:01:22.753000

0 days 00:01:40.555000

key is part of the people that we use to solve those issues whether it's stretch or outdoor resis...

qFsGMA75-oQ-Audio-WAV/-3g5yACwYnA.wav

0 days 00:01:59.919000

0 days 00:02:05.299000

that we do they've been able to find solutions or at least bring some answers to the table

qFsGMA75-oQ-Audio-WAV/-3g5yACwYnA.wav

0 days 00:00:04.840000

0 days 00:00:14.052000

key polymer brings a technical aspect to our operation that we don't have internally

qFsGMA75-oQ-Audio-WAV/-3g5yACwYnA.wav

0 days 00:00:13.211000

0 days 00:00:27.521000

we're a huge user of adhesives for our operation called flocking and we don't have the technical ...

23259 rows x 1 column

Schemes

ID

Dtype

Min

Max

Labels

emotion.intensity

float

0

3

emotion.presence

bool

sentiment

float

-3

3

sentiment.binarized

str

highly negative, highly positive, negative, neutral, positive, weakly negative, weakly positive

sentiment.binary

str

negative, positive

sentiment.binary.old

str

negative, non-negative

transcription

str