Robustness background noise

Overall scores

	w2v2-b-cat	w2v2-L-cat	w2v2-L-robust-cat	w2v2-L-vox-cat	w2v2-L-xls-r-cat
Overall Score	38.3% passed tests (23 passed / 37 failed).	33.3% passed tests (20 passed / 40 failed).	40.0% passed tests (24 passed / 36 failed).	43.3% passed tests (26 passed / 34 failed).	55.0% passed tests (33 passed / 27 failed).

Change Uar Babble Noise

Threshold: -0.05

Data	Change UAR Babble Noise
	w2v2-b-cat	w2v2-L-cat	w2v2-L-robust-cat	w2v2-L-vox-cat	w2v2-L-xls-r-cat
crema-d-1.2.0-emotion.categories.test.gold_standard	-0.00	0.01	0.02	-0.00	0.03
emovo-1.2.1-emotion.test	0.05	0.04	0.05	0.02	0.02
iemocap-2.3.0-emotion.categories.test.gold_standard	-0.03	-0.03	-0.01	-0.04	-0.02
meld-1.3.1-emotion.categories.test.gold_standard	-0.00	-0.00	-0.01	-0.00	-0.00
msppodcast-2.6.0-emotion.categories.test-1.gold_standard	-0.05	-0.03	-0.05	-0.06	-0.02
mean	-0.01	-0.00	0.00	-0.02	0.00

Change Uar Coughing

Threshold: -0.05

Data	Change UAR Coughing
	w2v2-b-cat	w2v2-L-cat	w2v2-L-robust-cat	w2v2-L-vox-cat	w2v2-L-xls-r-cat
crema-d-1.2.0-emotion.categories.test.gold_standard	0.05	-0.02	-0.04	0.05	0.04
emovo-1.2.1-emotion.test	-0.11	-0.09	-0.10	-0.03	-0.04
iemocap-2.3.0-emotion.categories.test.gold_standard	-0.09	-0.07	-0.09	-0.10	-0.06
meld-1.3.1-emotion.categories.test.gold_standard	-0.03	-0.05	-0.05	-0.05	-0.02
msppodcast-2.6.0-emotion.categories.test-1.gold_standard	-0.15	-0.13	-0.13	-0.17	-0.10
mean	-0.07	-0.07	-0.08	-0.06	-0.04

Change Uar Environmental Noise

Threshold: -0.05

Data	Change UAR Environmental Noise
	w2v2-b-cat	w2v2-L-cat	w2v2-L-robust-cat	w2v2-L-vox-cat	w2v2-L-xls-r-cat
crema-d-1.2.0-emotion.categories.test.gold_standard	-0.02	-0.00	-0.01	-0.01	0.01
emovo-1.2.1-emotion.test	0.04	0.03	0.02	-0.02	0.02
iemocap-2.3.0-emotion.categories.test.gold_standard	-0.02	-0.02	-0.02	-0.03	-0.01
meld-1.3.1-emotion.categories.test.gold_standard	-0.01	-0.01	-0.01	-0.00	0.00
msppodcast-2.6.0-emotion.categories.test-1.gold_standard	-0.02	-0.03	-0.03	-0.04	-0.02
mean	-0.01	-0.01	-0.01	-0.02	-0.00

Change Uar Music

Threshold: -0.05

Data	Change UAR Music
	w2v2-b-cat	w2v2-L-cat	w2v2-L-robust-cat	w2v2-L-vox-cat	w2v2-L-xls-r-cat
crema-d-1.2.0-emotion.categories.test.gold_standard	-0.02	-0.00	-0.00	-0.02	-0.00
emovo-1.2.1-emotion.test	0.04	0.04	0.04	-0.05	0.01
iemocap-2.3.0-emotion.categories.test.gold_standard	-0.02	-0.02	-0.01	-0.03	-0.00
meld-1.3.1-emotion.categories.test.gold_standard	-0.00	0.00	-0.01	-0.00	0.01
msppodcast-2.6.0-emotion.categories.test-1.gold_standard	-0.02	-0.03	-0.03	-0.05	-0.01
mean	-0.00	-0.00	-0.00	-0.03	0.00

Change Uar Sneezing

Threshold: -0.05

Data	Change UAR Sneezing
	w2v2-b-cat	w2v2-L-cat	w2v2-L-robust-cat	w2v2-L-vox-cat	w2v2-L-xls-r-cat
crema-d-1.2.0-emotion.categories.test.gold_standard	0.02	-0.03	-0.02	0.02	0.06
emovo-1.2.1-emotion.test	-0.06	-0.09	-0.12	0.01	-0.00
iemocap-2.3.0-emotion.categories.test.gold_standard	-0.10	-0.08	-0.08	-0.08	-0.04
meld-1.3.1-emotion.categories.test.gold_standard	-0.04	-0.05	-0.03	-0.03	-0.03
msppodcast-2.6.0-emotion.categories.test-1.gold_standard	-0.16	-0.16	-0.10	-0.16	-0.10
mean	-0.07	-0.08	-0.07	-0.05	-0.02

Change Uar White Noise

Threshold: -0.05

Data	Change UAR White Noise
	w2v2-b-cat	w2v2-L-cat	w2v2-L-robust-cat	w2v2-L-vox-cat	w2v2-L-xls-r-cat
crema-d-1.2.0-emotion.categories.test.gold_standard	-0.06	-0.07	-0.08	-0.07	-0.03
emovo-1.2.1-emotion.test	-0.01	0.06	-0.07	-0.07	0.02
iemocap-2.3.0-emotion.categories.test.gold_standard	-0.03	-0.03	-0.02	-0.07	-0.03
meld-1.3.1-emotion.categories.test.gold_standard	-0.01	-0.01	0.00	-0.01	0.01
msppodcast-2.6.0-emotion.categories.test-1.gold_standard	-0.04	-0.05	-0.04	-0.05	-0.02
mean	-0.03	-0.02	-0.04	-0.05	-0.01

Percentage Unchanged Predictions Babble Noise

Threshold: 0.9

Data	Percentage Unchanged Predictions Babble Noise
	w2v2-b-cat	w2v2-L-cat	w2v2-L-robust-cat	w2v2-L-vox-cat	w2v2-L-xls-r-cat
crema-d-1.2.0-emotion.categories.test.gold_standard	0.80	0.77	0.78	0.95	0.91
emovo-1.2.1-emotion.test	0.73	0.79	0.78	0.73	0.80
iemocap-2.3.0-emotion.categories.test.gold_standard	0.85	0.85	0.86	0.87	0.89
meld-1.3.1-emotion.categories.test.gold_standard	0.90	0.89	0.91	0.93	0.93
msppodcast-2.6.0-emotion.categories.test-1.gold_standard	0.85	0.85	0.86	0.82	0.80
mean	0.83	0.83	0.84	0.86	0.87

Percentage Unchanged Predictions Coughing

Threshold: 0.9

Data	Percentage Unchanged Predictions Coughing
	w2v2-b-cat	w2v2-L-cat	w2v2-L-robust-cat	w2v2-L-vox-cat	w2v2-L-xls-r-cat
crema-d-1.2.0-emotion.categories.test.gold_standard	0.68	0.63	0.57	0.79	0.82
emovo-1.2.1-emotion.test	0.50	0.57	0.58	0.41	0.56
iemocap-2.3.0-emotion.categories.test.gold_standard	0.63	0.67	0.69	0.63	0.66
meld-1.3.1-emotion.categories.test.gold_standard	0.80	0.79	0.80	0.78	0.78
msppodcast-2.6.0-emotion.categories.test-1.gold_standard	0.63	0.68	0.73	0.54	0.53
mean	0.65	0.67	0.67	0.63	0.67

Percentage Unchanged Predictions Environmental Noise

Threshold: 0.9

Data	Percentage Unchanged Predictions Environmental Noise
	w2v2-b-cat	w2v2-L-cat	w2v2-L-robust-cat	w2v2-L-vox-cat	w2v2-L-xls-r-cat
crema-d-1.2.0-emotion.categories.test.gold_standard	0.83	0.83	0.84	0.96	0.95
emovo-1.2.1-emotion.test	0.79	0.80	0.85	0.78	0.84
iemocap-2.3.0-emotion.categories.test.gold_standard	0.84	0.85	0.89	0.89	0.89
meld-1.3.1-emotion.categories.test.gold_standard	0.89	0.89	0.91	0.92	0.90
msppodcast-2.6.0-emotion.categories.test-1.gold_standard	0.85	0.86	0.89	0.84	0.84
mean	0.84	0.85	0.88	0.88	0.88

Percentage Unchanged Predictions Music

Threshold: 0.9

Data	Percentage Unchanged Predictions Music
	w2v2-b-cat	w2v2-L-cat	w2v2-L-robust-cat	w2v2-L-vox-cat	w2v2-L-xls-r-cat
crema-d-1.2.0-emotion.categories.test.gold_standard	0.85	0.85	0.84	0.97	0.96
emovo-1.2.1-emotion.test	0.79	0.79	0.84	0.75	0.83
iemocap-2.3.0-emotion.categories.test.gold_standard	0.88	0.87	0.91	0.89	0.90
meld-1.3.1-emotion.categories.test.gold_standard	0.91	0.90	0.92	0.94	0.92
msppodcast-2.6.0-emotion.categories.test-1.gold_standard	0.85	0.86	0.89	0.83	0.84
mean	0.86	0.85	0.88	0.88	0.89

Percentage Unchanged Predictions Sneezing

Threshold: 0.9

Data	Percentage Unchanged Predictions Sneezing
	w2v2-b-cat	w2v2-L-cat	w2v2-L-robust-cat	w2v2-L-vox-cat	w2v2-L-xls-r-cat
crema-d-1.2.0-emotion.categories.test.gold_standard	0.66	0.58	0.57	0.91	0.81
emovo-1.2.1-emotion.test	0.48	0.55	0.51	0.49	0.54
iemocap-2.3.0-emotion.categories.test.gold_standard	0.64	0.67	0.72	0.75	0.65
meld-1.3.1-emotion.categories.test.gold_standard	0.78	0.75	0.80	0.82	0.77
msppodcast-2.6.0-emotion.categories.test-1.gold_standard	0.62	0.62	0.76	0.58	0.50
mean	0.64	0.63	0.67	0.71	0.65

Percentage Unchanged Predictions White Noise

Threshold: 0.9

Data	Percentage Unchanged Predictions White Noise
	w2v2-b-cat	w2v2-L-cat	w2v2-L-robust-cat	w2v2-L-vox-cat	w2v2-L-xls-r-cat
crema-d-1.2.0-emotion.categories.test.gold_standard	0.77	0.73	0.69	0.91	0.91
emovo-1.2.1-emotion.test	0.72	0.76	0.82	0.67	0.79
iemocap-2.3.0-emotion.categories.test.gold_standard	0.76	0.79	0.80	0.79	0.86
meld-1.3.1-emotion.categories.test.gold_standard	0.78	0.83	0.82	0.79	0.82
msppodcast-2.6.0-emotion.categories.test-1.gold_standard	0.72	0.78	0.82	0.71	0.79
mean	0.75	0.78	0.79	0.77	0.83

Visualization Babble Noise

Confusion Matrix showing the shift from the predictions of the clean audio to the predictions of the audio with added babble noise.

w2v2-b-cat	w2v2-L-cat	w2v2-L-robust-cat	w2v2-L-vox-cat	w2v2-L-xls-r-cat