Robustness background noise

Overall scores

w2v2-L

hubert-L

wavlm

data2vec

Overall Score

37.5% passed tests (9 passed / 15 failed).

41.7% passed tests (10 passed / 14 failed).

62.5% passed tests (15 passed / 9 failed).

37.5% passed tests (9 passed / 15 failed).

Change Ccc Babble Noise

Threshold: -0.05

Data

Change CCC Babble Noise

w2v2-L

hubert-L

wavlm

data2vec

iemocap-2.3.0-emotion.dimensions.test.gold_standard

0.00

-0.01

-0.01

-0.00

msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard

-0.02

-0.01

-0.01

-0.01

mean

-0.01

-0.01

-0.01

-0.01

Change Ccc Coughing

Threshold: -0.05

Data

Change CCC Coughing

w2v2-L

hubert-L

wavlm

data2vec

iemocap-2.3.0-emotion.dimensions.test.gold_standard

-0.04

-0.02

0.04

-0.02

msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard

-0.08

-0.05

-0.06

-0.04

mean

-0.06

-0.04

-0.01

-0.03

Change Ccc Environmental Noise

Threshold: -0.05

Data

Change CCC Environmental Noise

w2v2-L

hubert-L

wavlm

data2vec

iemocap-2.3.0-emotion.dimensions.test.gold_standard

-0.01

-0.02

0.00

-0.01

msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard

-0.01

-0.02

-0.00

-0.01

mean

-0.01

-0.02

0.00

-0.01

Change Ccc Music

Threshold: -0.05

Data

Change CCC Music

w2v2-L

hubert-L

wavlm

data2vec

iemocap-2.3.0-emotion.dimensions.test.gold_standard

-0.00

-0.02

-0.02

-0.00

msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard

0.00

-0.01

-0.02

-0.01

mean

0.00

-0.01

-0.02

-0.01

Change Ccc Sneezing

Threshold: -0.05

Data

Change CCC Sneezing

w2v2-L

hubert-L

wavlm

data2vec

iemocap-2.3.0-emotion.dimensions.test.gold_standard

-0.05

-0.04

0.03

-0.05

msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard

-0.09

-0.05

-0.08

-0.05

mean

-0.07

-0.04

-0.03

-0.05

Change Ccc White Noise

Threshold: -0.05

Data

Change CCC White Noise

w2v2-L

hubert-L

wavlm

data2vec

iemocap-2.3.0-emotion.dimensions.test.gold_standard

-0.07

-0.05

0.05

-0.02

msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard

-0.05

-0.05

0.01

-0.05

mean

-0.06

-0.05

0.03

-0.04

Percentage Unchanged Predictions Babble Noise

Threshold: 0.9

Data

Percentage Unchanged Predictions Babble Noise

w2v2-L

hubert-L

wavlm

data2vec

iemocap-2.3.0-emotion.dimensions.test.gold_standard

0.95

0.94

0.94

0.88

msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard

0.89

0.92

0.92

0.87

mean

0.92

0.93

0.93

0.88

Percentage Unchanged Predictions Coughing

Threshold: 0.9

Data

Percentage Unchanged Predictions Coughing

w2v2-L

hubert-L

wavlm

data2vec

iemocap-2.3.0-emotion.dimensions.test.gold_standard

0.63

0.66

0.67

0.66

msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard

0.62

0.67

0.51

0.61

mean

0.62

0.67

0.59

0.64

Percentage Unchanged Predictions Environmental Noise

Threshold: 0.9

Data

Percentage Unchanged Predictions Environmental Noise

w2v2-L

hubert-L

wavlm

data2vec

iemocap-2.3.0-emotion.dimensions.test.gold_standard

0.85

0.84

0.93

0.84

msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard

0.79

0.87

0.94

0.83

mean

0.82

0.85

0.94

0.83

Percentage Unchanged Predictions Music

Threshold: 0.9

Data

Percentage Unchanged Predictions Music

w2v2-L

hubert-L

wavlm

data2vec

iemocap-2.3.0-emotion.dimensions.test.gold_standard

0.93

0.88

0.93

0.88

msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard

0.87

0.88

0.87

0.89

mean

0.90

0.88

0.90

0.89

Percentage Unchanged Predictions Sneezing

Threshold: 0.9

Data

Percentage Unchanged Predictions Sneezing

w2v2-L

hubert-L

wavlm

data2vec

iemocap-2.3.0-emotion.dimensions.test.gold_standard

0.51

0.51

0.55

0.51

msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard

0.55

0.61

0.35

0.52

mean

0.53

0.56

0.45

0.52

Percentage Unchanged Predictions White Noise

Threshold: 0.9

Data

Percentage Unchanged Predictions White Noise

w2v2-L

hubert-L

wavlm

data2vec

iemocap-2.3.0-emotion.dimensions.test.gold_standard

0.27

0.62

0.62

0.58

msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard

0.20

0.54

0.88

0.44

mean

0.24

0.58

0.75

0.51

Visualization Babble Noise

Difference of predictions for clean audio and audio with added babble noise. The allowed prediction difference \(\delta < 0.05\) is highlighted in green in the upper plot. The lower plot shows the distributions of the two predictions.

w2v2-L

hubert-L

wavlm

data2vec

../../../_images/visualization-babble-noise_iemocap-2.3.0-emotion.dimensions.test.gold_standard4.png
../../../_images/visualization-babble-noise_iemocap-2.3.0-emotion.dimensions.test.gold_standard8.png
../../../_images/visualization-babble-noise_iemocap-2.3.0-emotion.dimensions.test.gold_standard9.png
../../../_images/visualization-babble-noise_iemocap-2.3.0-emotion.dimensions.test.gold_standard10.png
../../../_images/visualization-babble-noise_msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard4.png
../../../_images/visualization-babble-noise_msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard8.png
../../../_images/visualization-babble-noise_msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard9.png
../../../_images/visualization-babble-noise_msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard10.png
../../../_images/visualization-coughing_iemocap-2.3.0-emotion.dimensions.test.gold_standard4.png
../../../_images/visualization-coughing_iemocap-2.3.0-emotion.dimensions.test.gold_standard8.png
../../../_images/visualization-coughing_iemocap-2.3.0-emotion.dimensions.test.gold_standard9.png
../../../_images/visualization-coughing_iemocap-2.3.0-emotion.dimensions.test.gold_standard10.png
../../../_images/visualization-coughing_msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard4.png
../../../_images/visualization-coughing_msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard8.png
../../../_images/visualization-coughing_msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard9.png
../../../_images/visualization-coughing_msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard10.png
../../../_images/visualization-environmental-noise_iemocap-2.3.0-emotion.dimensions.test.gold_standard4.png
../../../_images/visualization-environmental-noise_iemocap-2.3.0-emotion.dimensions.test.gold_standard8.png
../../../_images/visualization-environmental-noise_iemocap-2.3.0-emotion.dimensions.test.gold_standard9.png
../../../_images/visualization-environmental-noise_iemocap-2.3.0-emotion.dimensions.test.gold_standard10.png
../../../_images/visualization-environmental-noise_msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard4.png
../../../_images/visualization-environmental-noise_msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard8.png
../../../_images/visualization-environmental-noise_msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard9.png
../../../_images/visualization-environmental-noise_msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard10.png
../../../_images/visualization-music_iemocap-2.3.0-emotion.dimensions.test.gold_standard4.png
../../../_images/visualization-music_iemocap-2.3.0-emotion.dimensions.test.gold_standard8.png
../../../_images/visualization-music_iemocap-2.3.0-emotion.dimensions.test.gold_standard9.png
../../../_images/visualization-music_iemocap-2.3.0-emotion.dimensions.test.gold_standard10.png
../../../_images/visualization-music_msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard4.png
../../../_images/visualization-music_msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard8.png
../../../_images/visualization-music_msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard9.png
../../../_images/visualization-music_msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard10.png
../../../_images/visualization-sneezing_iemocap-2.3.0-emotion.dimensions.test.gold_standard4.png
../../../_images/visualization-sneezing_iemocap-2.3.0-emotion.dimensions.test.gold_standard8.png
../../../_images/visualization-sneezing_iemocap-2.3.0-emotion.dimensions.test.gold_standard9.png
../../../_images/visualization-sneezing_iemocap-2.3.0-emotion.dimensions.test.gold_standard10.png
../../../_images/visualization-sneezing_msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard4.png
../../../_images/visualization-sneezing_msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard8.png
../../../_images/visualization-sneezing_msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard9.png
../../../_images/visualization-sneezing_msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard10.png
../../../_images/visualization-white-noise_iemocap-2.3.0-emotion.dimensions.test.gold_standard4.png
../../../_images/visualization-white-noise_iemocap-2.3.0-emotion.dimensions.test.gold_standard8.png
../../../_images/visualization-white-noise_iemocap-2.3.0-emotion.dimensions.test.gold_standard9.png
../../../_images/visualization-white-noise_iemocap-2.3.0-emotion.dimensions.test.gold_standard10.png
../../../_images/visualization-white-noise_msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard4.png
../../../_images/visualization-white-noise_msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard8.png
../../../_images/visualization-white-noise_msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard9.png
../../../_images/visualization-white-noise_msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard10.png