Robustness background noise¶
CNN14 |
w2v2-b |
hubert-b |
axlstm |
|
|---|---|---|---|---|
Overall Score |
41.7% passed tests (10 passed / 14 failed). |
29.2% passed tests (7 passed / 17 failed). |
33.3% passed tests (8 passed / 16 failed). |
45.8% passed tests (11 passed / 13 failed). |
Change Ccc Babble Noise¶
Data |
Change CCC Babble Noise |
|||
|---|---|---|---|---|
CNN14 |
w2v2-b |
hubert-b |
axlstm |
|
iemocap-2.3.0-emotion.dimensions.test.gold_standard |
-0.01 |
-0.02 |
-0.03 |
-0.02 |
msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard |
-0.05 |
-0.00 |
-0.03 |
-0.03 |
mean |
-0.03 |
-0.01 |
-0.03 |
-0.03 |
Change Ccc Coughing¶
Data |
Change CCC Coughing |
|||
|---|---|---|---|---|
CNN14 |
w2v2-b |
hubert-b |
axlstm |
|
iemocap-2.3.0-emotion.dimensions.test.gold_standard |
0.01 |
-0.12 |
-0.09 |
-0.01 |
msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard |
0.01 |
-0.13 |
-0.13 |
-0.01 |
mean |
0.01 |
-0.12 |
-0.11 |
-0.01 |
Change Ccc Environmental Noise¶
Data |
Change CCC Environmental Noise |
|||
|---|---|---|---|---|
CNN14 |
w2v2-b |
hubert-b |
axlstm |
|
iemocap-2.3.0-emotion.dimensions.test.gold_standard |
-0.01 |
-0.02 |
-0.03 |
-0.02 |
msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard |
-0.02 |
-0.01 |
-0.02 |
-0.03 |
mean |
-0.01 |
-0.01 |
-0.03 |
-0.03 |
Change Ccc Music¶
Data |
Change CCC Music |
|||
|---|---|---|---|---|
CNN14 |
w2v2-b |
hubert-b |
axlstm |
|
iemocap-2.3.0-emotion.dimensions.test.gold_standard |
-0.03 |
-0.01 |
-0.01 |
-0.02 |
msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard |
-0.04 |
-0.00 |
-0.02 |
-0.04 |
mean |
-0.04 |
-0.01 |
-0.01 |
-0.03 |
Change Ccc Sneezing¶
Data |
Change CCC Sneezing |
|||
|---|---|---|---|---|
CNN14 |
w2v2-b |
hubert-b |
axlstm |
|
iemocap-2.3.0-emotion.dimensions.test.gold_standard |
0.01 |
-0.09 |
-0.06 |
-0.01 |
msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard |
0.03 |
-0.09 |
-0.08 |
-0.01 |
mean |
0.02 |
-0.09 |
-0.07 |
-0.01 |
Change Ccc White Noise¶
Data |
Change CCC White Noise |
|||
|---|---|---|---|---|
CNN14 |
w2v2-b |
hubert-b |
axlstm |
|
iemocap-2.3.0-emotion.dimensions.test.gold_standard |
-0.09 |
-0.03 |
-0.03 |
0.04 |
msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard |
-0.01 |
-0.06 |
-0.03 |
-0.09 |
mean |
-0.05 |
-0.04 |
-0.03 |
-0.02 |
Percentage Unchanged Predictions Babble Noise¶
Data |
Percentage Unchanged Predictions Babble Noise |
|||
|---|---|---|---|---|
CNN14 |
w2v2-b |
hubert-b |
axlstm |
|
iemocap-2.3.0-emotion.dimensions.test.gold_standard |
0.67 |
0.81 |
0.68 |
0.51 |
msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard |
0.40 |
0.67 |
0.68 |
0.47 |
mean |
0.54 |
0.74 |
0.68 |
0.49 |
Percentage Unchanged Predictions Coughing¶
Data |
Percentage Unchanged Predictions Coughing |
|||
|---|---|---|---|---|
CNN14 |
w2v2-b |
hubert-b |
axlstm |
|
iemocap-2.3.0-emotion.dimensions.test.gold_standard |
0.65 |
0.36 |
0.36 |
0.53 |
msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard |
0.52 |
0.26 |
0.30 |
0.59 |
mean |
0.58 |
0.31 |
0.33 |
0.56 |
Percentage Unchanged Predictions Environmental Noise¶
Data |
Percentage Unchanged Predictions Environmental Noise |
|||
|---|---|---|---|---|
CNN14 |
w2v2-b |
hubert-b |
axlstm |
|
iemocap-2.3.0-emotion.dimensions.test.gold_standard |
0.56 |
0.77 |
0.69 |
0.53 |
msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard |
0.46 |
0.71 |
0.71 |
0.53 |
mean |
0.51 |
0.74 |
0.70 |
0.53 |
Percentage Unchanged Predictions Music¶
Data |
Percentage Unchanged Predictions Music |
|||
|---|---|---|---|---|
CNN14 |
w2v2-b |
hubert-b |
axlstm |
|
iemocap-2.3.0-emotion.dimensions.test.gold_standard |
0.59 |
0.82 |
0.75 |
0.53 |
msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard |
0.40 |
0.74 |
0.75 |
0.47 |
mean |
0.49 |
0.78 |
0.75 |
0.50 |
Percentage Unchanged Predictions Sneezing¶
Data |
Percentage Unchanged Predictions Sneezing |
|||
|---|---|---|---|---|
CNN14 |
w2v2-b |
hubert-b |
axlstm |
|
iemocap-2.3.0-emotion.dimensions.test.gold_standard |
0.63 |
0.47 |
0.47 |
0.47 |
msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard |
0.55 |
0.26 |
0.35 |
0.55 |
mean |
0.59 |
0.36 |
0.41 |
0.51 |
Percentage Unchanged Predictions White Noise¶
Data |
Percentage Unchanged Predictions White Noise |
|||
|---|---|---|---|---|
CNN14 |
w2v2-b |
hubert-b |
axlstm |
|
iemocap-2.3.0-emotion.dimensions.test.gold_standard |
0.31 |
0.53 |
0.53 |
0.22 |
msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard |
0.23 |
0.45 |
0.53 |
0.30 |
mean |
0.27 |
0.49 |
0.53 |
0.26 |
Visualization Babble Noise¶
Difference of predictions for clean audio and audio with added babble noise. The allowed prediction difference \(\delta < 0.05\) is highlighted in green in the upper plot. The lower plot shows the distributions of the two predictions.
CNN14 |
w2v2-b |
hubert-b |
axlstm |
|---|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|