Robustness background noise

Overall scores

CNN14-cat

w2v2-b-cat

hubert-b-cat

axlstm-cat

Overall Score

56.7% passed tests (34 passed / 26 failed).

38.3% passed tests (23 passed / 37 failed).

30.0% passed tests (18 passed / 42 failed).

35.0% passed tests (21 passed / 39 failed).

Change Uar Babble Noise

Threshold: -0.05

Data

Change UAR Babble Noise

CNN14-cat

w2v2-b-cat

hubert-b-cat

axlstm-cat

crema-d-1.2.0-emotion.categories.test.gold_standard

-0.01

-0.00

-0.01

0.04

emovo-1.2.1-emotion.test

0.04

0.05

-0.00

0.03

iemocap-2.3.0-emotion.categories.test.gold_standard

-0.01

-0.03

-0.04

-0.05

meld-1.3.1-emotion.categories.test.gold_standard

-0.01

-0.00

-0.01

-0.02

msppodcast-2.6.0-emotion.categories.test-1.gold_standard

-0.04

-0.05

-0.04

-0.08

mean

-0.01

-0.01

-0.02

-0.02

Change Uar Coughing

Threshold: -0.05

Data

Change UAR Coughing

CNN14-cat

w2v2-b-cat

hubert-b-cat

axlstm-cat

crema-d-1.2.0-emotion.categories.test.gold_standard

0.04

0.05

0.09

0.03

emovo-1.2.1-emotion.test

0.01

-0.11

-0.05

-0.01

iemocap-2.3.0-emotion.categories.test.gold_standard

0.01

-0.09

-0.06

-0.03

meld-1.3.1-emotion.categories.test.gold_standard

-0.00

-0.03

-0.05

-0.03

msppodcast-2.6.0-emotion.categories.test-1.gold_standard

-0.06

-0.15

-0.14

-0.04

mean

0.00

-0.07

-0.04

-0.02

Change Uar Environmental Noise

Threshold: -0.05

Data

Change UAR Environmental Noise

CNN14-cat

w2v2-b-cat

hubert-b-cat

axlstm-cat

crema-d-1.2.0-emotion.categories.test.gold_standard

0.00

-0.02

-0.05

-0.04

emovo-1.2.1-emotion.test

0.04

0.04

-0.01

-0.03

iemocap-2.3.0-emotion.categories.test.gold_standard

-0.00

-0.02

-0.05

-0.04

meld-1.3.1-emotion.categories.test.gold_standard

-0.00

-0.01

-0.02

-0.01

msppodcast-2.6.0-emotion.categories.test-1.gold_standard

-0.05

-0.02

-0.05

-0.05

mean

-0.00

-0.01

-0.04

-0.03

Change Uar Music

Threshold: -0.05

Data

Change UAR Music

CNN14-cat

w2v2-b-cat

hubert-b-cat

axlstm-cat

crema-d-1.2.0-emotion.categories.test.gold_standard

0.01

-0.02

-0.04

0.02

emovo-1.2.1-emotion.test

0.01

0.04

-0.02

0.01

iemocap-2.3.0-emotion.categories.test.gold_standard

-0.01

-0.02

-0.05

-0.03

meld-1.3.1-emotion.categories.test.gold_standard

-0.00

-0.00

-0.01

-0.02

msppodcast-2.6.0-emotion.categories.test-1.gold_standard

-0.05

-0.02

-0.05

-0.05

mean

-0.01

-0.00

-0.03

-0.01

Change Uar Sneezing

Threshold: -0.05

Data

Change UAR Sneezing

CNN14-cat

w2v2-b-cat

hubert-b-cat

axlstm-cat

crema-d-1.2.0-emotion.categories.test.gold_standard

0.02

0.02

0.04

-0.01

emovo-1.2.1-emotion.test

-0.01

-0.06

-0.04

0.00

iemocap-2.3.0-emotion.categories.test.gold_standard

0.01

-0.10

-0.07

-0.04

meld-1.3.1-emotion.categories.test.gold_standard

0.01

-0.04

-0.05

-0.03

msppodcast-2.6.0-emotion.categories.test-1.gold_standard

-0.03

-0.16

-0.13

-0.05

mean

0.00

-0.07

-0.05

-0.03

Change Uar White Noise

Threshold: -0.05

Data

Change UAR White Noise

CNN14-cat

w2v2-b-cat

hubert-b-cat

axlstm-cat

crema-d-1.2.0-emotion.categories.test.gold_standard

0.04

-0.06

-0.13

-0.11

emovo-1.2.1-emotion.test

-0.00

-0.01

-0.12

-0.08

iemocap-2.3.0-emotion.categories.test.gold_standard

0.00

-0.03

-0.13

-0.09

meld-1.3.1-emotion.categories.test.gold_standard

-0.02

-0.01

-0.03

-0.02

msppodcast-2.6.0-emotion.categories.test-1.gold_standard

-0.12

-0.04

-0.17

-0.11

mean

-0.02

-0.03

-0.12

-0.08

Percentage Unchanged Predictions Babble Noise

Threshold: 0.9

Data

Percentage Unchanged Predictions Babble Noise

CNN14-cat

w2v2-b-cat

hubert-b-cat

axlstm-cat

crema-d-1.2.0-emotion.categories.test.gold_standard

0.94

0.80

0.94

0.67

emovo-1.2.1-emotion.test

0.85

0.73

0.81

0.60

iemocap-2.3.0-emotion.categories.test.gold_standard

0.93

0.85

0.89

0.74

meld-1.3.1-emotion.categories.test.gold_standard

0.90

0.90

0.91

0.88

msppodcast-2.6.0-emotion.categories.test-1.gold_standard

0.72

0.85

0.85

0.67

mean

0.87

0.83

0.88

0.71

Percentage Unchanged Predictions Coughing

Threshold: 0.9

Data

Percentage Unchanged Predictions Coughing

CNN14-cat

w2v2-b-cat

hubert-b-cat

axlstm-cat

crema-d-1.2.0-emotion.categories.test.gold_standard

0.91

0.68

0.75

0.68

emovo-1.2.1-emotion.test

0.79

0.50

0.48

0.62

iemocap-2.3.0-emotion.categories.test.gold_standard

0.84

0.63

0.63

0.70

meld-1.3.1-emotion.categories.test.gold_standard

0.81

0.80

0.74

0.80

msppodcast-2.6.0-emotion.categories.test-1.gold_standard

0.69

0.63

0.57

0.74

mean

0.81

0.65

0.63

0.71

Percentage Unchanged Predictions Environmental Noise

Threshold: 0.9

Data

Percentage Unchanged Predictions Environmental Noise

CNN14-cat

w2v2-b-cat

hubert-b-cat

axlstm-cat

crema-d-1.2.0-emotion.categories.test.gold_standard

0.93

0.83

0.92

0.76

emovo-1.2.1-emotion.test

0.82

0.79

0.76

0.72

iemocap-2.3.0-emotion.categories.test.gold_standard

0.92

0.84

0.87

0.78

meld-1.3.1-emotion.categories.test.gold_standard

0.88

0.89

0.87

0.86

msppodcast-2.6.0-emotion.categories.test-1.gold_standard

0.71

0.85

0.84

0.74

mean

0.85

0.84

0.85

0.77

Percentage Unchanged Predictions Music

Threshold: 0.9

Data

Percentage Unchanged Predictions Music

CNN14-cat

w2v2-b-cat

hubert-b-cat

axlstm-cat

crema-d-1.2.0-emotion.categories.test.gold_standard

0.95

0.85

0.94

0.79

emovo-1.2.1-emotion.test

0.82

0.79

0.80

0.70

iemocap-2.3.0-emotion.categories.test.gold_standard

0.93

0.88

0.89

0.77

meld-1.3.1-emotion.categories.test.gold_standard

0.90

0.91

0.90

0.87

msppodcast-2.6.0-emotion.categories.test-1.gold_standard

0.72

0.85

0.83

0.73

mean

0.86

0.86

0.87

0.77

Percentage Unchanged Predictions Sneezing

Threshold: 0.9

Data

Percentage Unchanged Predictions Sneezing

CNN14-cat

w2v2-b-cat

hubert-b-cat

axlstm-cat

crema-d-1.2.0-emotion.categories.test.gold_standard

0.92

0.66

0.83

0.67

emovo-1.2.1-emotion.test

0.83

0.48

0.49

0.58

iemocap-2.3.0-emotion.categories.test.gold_standard

0.86

0.64

0.71

0.66

meld-1.3.1-emotion.categories.test.gold_standard

0.83

0.78

0.76

0.78

msppodcast-2.6.0-emotion.categories.test-1.gold_standard

0.75

0.62

0.58

0.74

mean

0.84

0.64

0.67

0.69

Percentage Unchanged Predictions White Noise

Threshold: 0.9

Data

Percentage Unchanged Predictions White Noise

CNN14-cat

w2v2-b-cat

hubert-b-cat

axlstm-cat

crema-d-1.2.0-emotion.categories.test.gold_standard

0.85

0.77

0.83

0.68

emovo-1.2.1-emotion.test

0.68

0.72

0.51

0.53

iemocap-2.3.0-emotion.categories.test.gold_standard

0.84

0.76

0.71

0.68

meld-1.3.1-emotion.categories.test.gold_standard

0.75

0.78

0.67

0.73

msppodcast-2.6.0-emotion.categories.test-1.gold_standard

0.46

0.72

0.60

0.59

mean

0.72

0.75

0.66

0.64

Visualization Babble Noise

Confusion Matrix showing the shift from the predictions of the clean audio to the predictions of the audio with added babble noise.

CNN14-cat

w2v2-b-cat

hubert-b-cat

axlstm-cat

../../../_images/visualization-babble-noise_crema-d-1.2.0-emotion.categories.test.gold_standard5.png
../../../_images/visualization-babble-noise_crema-d-1.2.0-emotion.categories.test.gold_standard.png
../../../_images/visualization-babble-noise_crema-d-1.2.0-emotion.categories.test.gold_standard6.png
../../../_images/visualization-babble-noise_crema-d-1.2.0-emotion.categories.test.gold_standard7.png
../../../_images/visualization-babble-noise_emovo-1.2.1-emotion.test5.png
../../../_images/visualization-babble-noise_emovo-1.2.1-emotion.test.png
../../../_images/visualization-babble-noise_emovo-1.2.1-emotion.test6.png
../../../_images/visualization-babble-noise_emovo-1.2.1-emotion.test7.png
../../../_images/visualization-babble-noise_iemocap-2.3.0-emotion.categories.test.gold_standard5.png
../../../_images/visualization-babble-noise_iemocap-2.3.0-emotion.categories.test.gold_standard.png
../../../_images/visualization-babble-noise_iemocap-2.3.0-emotion.categories.test.gold_standard6.png
../../../_images/visualization-babble-noise_iemocap-2.3.0-emotion.categories.test.gold_standard7.png
../../../_images/visualization-babble-noise_meld-1.3.1-emotion.categories.test.gold_standard5.png
../../../_images/visualization-babble-noise_meld-1.3.1-emotion.categories.test.gold_standard.png
../../../_images/visualization-babble-noise_meld-1.3.1-emotion.categories.test.gold_standard6.png
../../../_images/visualization-babble-noise_meld-1.3.1-emotion.categories.test.gold_standard7.png
../../../_images/visualization-babble-noise_msppodcast-2.6.0-emotion.categories.test-1.gold_standard5.png
../../../_images/visualization-babble-noise_msppodcast-2.6.0-emotion.categories.test-1.gold_standard.png
../../../_images/visualization-babble-noise_msppodcast-2.6.0-emotion.categories.test-1.gold_standard6.png
../../../_images/visualization-babble-noise_msppodcast-2.6.0-emotion.categories.test-1.gold_standard7.png
../../../_images/visualization-coughing_crema-d-1.2.0-emotion.categories.test.gold_standard5.png
../../../_images/visualization-coughing_crema-d-1.2.0-emotion.categories.test.gold_standard.png
../../../_images/visualization-coughing_crema-d-1.2.0-emotion.categories.test.gold_standard6.png
../../../_images/visualization-coughing_crema-d-1.2.0-emotion.categories.test.gold_standard7.png
../../../_images/visualization-coughing_emovo-1.2.1-emotion.test5.png
../../../_images/visualization-coughing_emovo-1.2.1-emotion.test.png
../../../_images/visualization-coughing_emovo-1.2.1-emotion.test6.png
../../../_images/visualization-coughing_emovo-1.2.1-emotion.test7.png
../../../_images/visualization-coughing_iemocap-2.3.0-emotion.categories.test.gold_standard5.png
../../../_images/visualization-coughing_iemocap-2.3.0-emotion.categories.test.gold_standard.png
../../../_images/visualization-coughing_iemocap-2.3.0-emotion.categories.test.gold_standard6.png
../../../_images/visualization-coughing_iemocap-2.3.0-emotion.categories.test.gold_standard7.png
../../../_images/visualization-coughing_meld-1.3.1-emotion.categories.test.gold_standard5.png
../../../_images/visualization-coughing_meld-1.3.1-emotion.categories.test.gold_standard.png
../../../_images/visualization-coughing_meld-1.3.1-emotion.categories.test.gold_standard6.png
../../../_images/visualization-coughing_meld-1.3.1-emotion.categories.test.gold_standard7.png
../../../_images/visualization-coughing_msppodcast-2.6.0-emotion.categories.test-1.gold_standard5.png
../../../_images/visualization-coughing_msppodcast-2.6.0-emotion.categories.test-1.gold_standard.png
../../../_images/visualization-coughing_msppodcast-2.6.0-emotion.categories.test-1.gold_standard6.png
../../../_images/visualization-coughing_msppodcast-2.6.0-emotion.categories.test-1.gold_standard7.png
../../../_images/visualization-environmental-noise_crema-d-1.2.0-emotion.categories.test.gold_standard5.png
../../../_images/visualization-environmental-noise_crema-d-1.2.0-emotion.categories.test.gold_standard.png
../../../_images/visualization-environmental-noise_crema-d-1.2.0-emotion.categories.test.gold_standard6.png
../../../_images/visualization-environmental-noise_crema-d-1.2.0-emotion.categories.test.gold_standard7.png
../../../_images/visualization-environmental-noise_emovo-1.2.1-emotion.test5.png
../../../_images/visualization-environmental-noise_emovo-1.2.1-emotion.test.png
../../../_images/visualization-environmental-noise_emovo-1.2.1-emotion.test6.png
../../../_images/visualization-environmental-noise_emovo-1.2.1-emotion.test7.png
../../../_images/visualization-environmental-noise_iemocap-2.3.0-emotion.categories.test.gold_standard5.png
../../../_images/visualization-environmental-noise_iemocap-2.3.0-emotion.categories.test.gold_standard.png
../../../_images/visualization-environmental-noise_iemocap-2.3.0-emotion.categories.test.gold_standard6.png
../../../_images/visualization-environmental-noise_iemocap-2.3.0-emotion.categories.test.gold_standard7.png
../../../_images/visualization-environmental-noise_meld-1.3.1-emotion.categories.test.gold_standard5.png
../../../_images/visualization-environmental-noise_meld-1.3.1-emotion.categories.test.gold_standard.png
../../../_images/visualization-environmental-noise_meld-1.3.1-emotion.categories.test.gold_standard6.png
../../../_images/visualization-environmental-noise_meld-1.3.1-emotion.categories.test.gold_standard7.png
../../../_images/visualization-environmental-noise_msppodcast-2.6.0-emotion.categories.test-1.gold_standard5.png
../../../_images/visualization-environmental-noise_msppodcast-2.6.0-emotion.categories.test-1.gold_standard.png
../../../_images/visualization-environmental-noise_msppodcast-2.6.0-emotion.categories.test-1.gold_standard6.png
../../../_images/visualization-environmental-noise_msppodcast-2.6.0-emotion.categories.test-1.gold_standard7.png
../../../_images/visualization-music_crema-d-1.2.0-emotion.categories.test.gold_standard5.png
../../../_images/visualization-music_crema-d-1.2.0-emotion.categories.test.gold_standard.png
../../../_images/visualization-music_crema-d-1.2.0-emotion.categories.test.gold_standard6.png
../../../_images/visualization-music_crema-d-1.2.0-emotion.categories.test.gold_standard7.png
../../../_images/visualization-music_emovo-1.2.1-emotion.test5.png
../../../_images/visualization-music_emovo-1.2.1-emotion.test.png
../../../_images/visualization-music_emovo-1.2.1-emotion.test6.png
../../../_images/visualization-music_emovo-1.2.1-emotion.test7.png
../../../_images/visualization-music_iemocap-2.3.0-emotion.categories.test.gold_standard5.png
../../../_images/visualization-music_iemocap-2.3.0-emotion.categories.test.gold_standard.png
../../../_images/visualization-music_iemocap-2.3.0-emotion.categories.test.gold_standard6.png
../../../_images/visualization-music_iemocap-2.3.0-emotion.categories.test.gold_standard7.png
../../../_images/visualization-music_meld-1.3.1-emotion.categories.test.gold_standard5.png
../../../_images/visualization-music_meld-1.3.1-emotion.categories.test.gold_standard.png
../../../_images/visualization-music_meld-1.3.1-emotion.categories.test.gold_standard6.png
../../../_images/visualization-music_meld-1.3.1-emotion.categories.test.gold_standard7.png
../../../_images/visualization-music_msppodcast-2.6.0-emotion.categories.test-1.gold_standard5.png
../../../_images/visualization-music_msppodcast-2.6.0-emotion.categories.test-1.gold_standard.png
../../../_images/visualization-music_msppodcast-2.6.0-emotion.categories.test-1.gold_standard6.png
../../../_images/visualization-music_msppodcast-2.6.0-emotion.categories.test-1.gold_standard7.png
../../../_images/visualization-sneezing_crema-d-1.2.0-emotion.categories.test.gold_standard5.png
../../../_images/visualization-sneezing_crema-d-1.2.0-emotion.categories.test.gold_standard.png
../../../_images/visualization-sneezing_crema-d-1.2.0-emotion.categories.test.gold_standard6.png
../../../_images/visualization-sneezing_crema-d-1.2.0-emotion.categories.test.gold_standard7.png
../../../_images/visualization-sneezing_emovo-1.2.1-emotion.test5.png
../../../_images/visualization-sneezing_emovo-1.2.1-emotion.test.png
../../../_images/visualization-sneezing_emovo-1.2.1-emotion.test6.png
../../../_images/visualization-sneezing_emovo-1.2.1-emotion.test7.png
../../../_images/visualization-sneezing_iemocap-2.3.0-emotion.categories.test.gold_standard5.png
../../../_images/visualization-sneezing_iemocap-2.3.0-emotion.categories.test.gold_standard.png
../../../_images/visualization-sneezing_iemocap-2.3.0-emotion.categories.test.gold_standard6.png
../../../_images/visualization-sneezing_iemocap-2.3.0-emotion.categories.test.gold_standard7.png
../../../_images/visualization-sneezing_meld-1.3.1-emotion.categories.test.gold_standard5.png
../../../_images/visualization-sneezing_meld-1.3.1-emotion.categories.test.gold_standard.png
../../../_images/visualization-sneezing_meld-1.3.1-emotion.categories.test.gold_standard6.png
../../../_images/visualization-sneezing_meld-1.3.1-emotion.categories.test.gold_standard7.png
../../../_images/visualization-sneezing_msppodcast-2.6.0-emotion.categories.test-1.gold_standard5.png
../../../_images/visualization-sneezing_msppodcast-2.6.0-emotion.categories.test-1.gold_standard.png
../../../_images/visualization-sneezing_msppodcast-2.6.0-emotion.categories.test-1.gold_standard6.png
../../../_images/visualization-sneezing_msppodcast-2.6.0-emotion.categories.test-1.gold_standard7.png
../../../_images/visualization-white-noise_crema-d-1.2.0-emotion.categories.test.gold_standard5.png
../../../_images/visualization-white-noise_crema-d-1.2.0-emotion.categories.test.gold_standard.png
../../../_images/visualization-white-noise_crema-d-1.2.0-emotion.categories.test.gold_standard6.png
../../../_images/visualization-white-noise_crema-d-1.2.0-emotion.categories.test.gold_standard7.png
../../../_images/visualization-white-noise_emovo-1.2.1-emotion.test5.png
../../../_images/visualization-white-noise_emovo-1.2.1-emotion.test.png
../../../_images/visualization-white-noise_emovo-1.2.1-emotion.test6.png
../../../_images/visualization-white-noise_emovo-1.2.1-emotion.test7.png
../../../_images/visualization-white-noise_iemocap-2.3.0-emotion.categories.test.gold_standard5.png
../../../_images/visualization-white-noise_iemocap-2.3.0-emotion.categories.test.gold_standard.png
../../../_images/visualization-white-noise_iemocap-2.3.0-emotion.categories.test.gold_standard6.png
../../../_images/visualization-white-noise_iemocap-2.3.0-emotion.categories.test.gold_standard7.png
../../../_images/visualization-white-noise_meld-1.3.1-emotion.categories.test.gold_standard5.png
../../../_images/visualization-white-noise_meld-1.3.1-emotion.categories.test.gold_standard.png
../../../_images/visualization-white-noise_meld-1.3.1-emotion.categories.test.gold_standard6.png
../../../_images/visualization-white-noise_meld-1.3.1-emotion.categories.test.gold_standard7.png
../../../_images/visualization-white-noise_msppodcast-2.6.0-emotion.categories.test-1.gold_standard5.png
../../../_images/visualization-white-noise_msppodcast-2.6.0-emotion.categories.test-1.gold_standard.png
../../../_images/visualization-white-noise_msppodcast-2.6.0-emotion.categories.test-1.gold_standard6.png
../../../_images/visualization-white-noise_msppodcast-2.6.0-emotion.categories.test-1.gold_standard7.png