Robustness background noise¶
w2v2-b |
w2v2-L |
w2v2-L-robust |
w2v2-L-xls-r |
w2v2-L-vox |
|
|---|---|---|---|---|---|
Overall Score |
50.0% passed tests (12 passed / 12 failed). |
50.0% passed tests (12 passed / 12 failed). |
75.0% passed tests (18 passed / 6 failed). |
45.8% passed tests (11 passed / 13 failed). |
54.2% passed tests (13 passed / 11 failed). |
Change Ccc Babble Noise¶
Data |
Change CCC Babble Noise |
||||
|---|---|---|---|---|---|
w2v2-b |
w2v2-L |
w2v2-L-robust |
w2v2-L-xls-r |
w2v2-L-vox |
|
iemocap-2.3.0-emotion.dimensions.test.gold_standard |
-0.01 |
-0.00 |
-0.01 |
-0.00 |
-0.01 |
msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard |
0.00 |
-0.01 |
0.00 |
-0.02 |
-0.00 |
mean |
-0.01 |
-0.01 |
-0.01 |
-0.01 |
-0.01 |
Change Ccc Coughing¶
Data |
Change CCC Coughing |
||||
|---|---|---|---|---|---|
w2v2-b |
w2v2-L |
w2v2-L-robust |
w2v2-L-xls-r |
w2v2-L-vox |
|
iemocap-2.3.0-emotion.dimensions.test.gold_standard |
-0.01 |
-0.01 |
-0.02 |
-0.02 |
-0.01 |
msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard |
-0.04 |
-0.06 |
-0.03 |
-0.06 |
-0.04 |
mean |
-0.03 |
-0.03 |
-0.03 |
-0.04 |
-0.03 |
Change Ccc Environmental Noise¶
Data |
Change CCC Environmental Noise |
||||
|---|---|---|---|---|---|
w2v2-b |
w2v2-L |
w2v2-L-robust |
w2v2-L-xls-r |
w2v2-L-vox |
|
iemocap-2.3.0-emotion.dimensions.test.gold_standard |
-0.02 |
-0.01 |
-0.01 |
-0.01 |
-0.02 |
msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard |
-0.01 |
-0.01 |
0.01 |
-0.01 |
-0.01 |
mean |
-0.01 |
-0.01 |
0.00 |
-0.01 |
-0.01 |
Change Ccc Music¶
Data |
Change CCC Music |
||||
|---|---|---|---|---|---|
w2v2-b |
w2v2-L |
w2v2-L-robust |
w2v2-L-xls-r |
w2v2-L-vox |
|
iemocap-2.3.0-emotion.dimensions.test.gold_standard |
-0.02 |
-0.01 |
-0.01 |
-0.01 |
-0.01 |
msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard |
-0.00 |
0.00 |
0.01 |
-0.00 |
-0.00 |
mean |
-0.01 |
-0.01 |
0.00 |
-0.01 |
-0.01 |
Change Ccc Sneezing¶
Data |
Change CCC Sneezing |
||||
|---|---|---|---|---|---|
w2v2-b |
w2v2-L |
w2v2-L-robust |
w2v2-L-xls-r |
w2v2-L-vox |
|
iemocap-2.3.0-emotion.dimensions.test.gold_standard |
-0.01 |
-0.02 |
-0.04 |
-0.02 |
-0.01 |
msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard |
-0.03 |
-0.08 |
-0.04 |
-0.05 |
-0.04 |
mean |
-0.02 |
-0.05 |
-0.04 |
-0.04 |
-0.03 |
Change Ccc White Noise¶
Data |
Change CCC White Noise |
||||
|---|---|---|---|---|---|
w2v2-b |
w2v2-L |
w2v2-L-robust |
w2v2-L-xls-r |
w2v2-L-vox |
|
iemocap-2.3.0-emotion.dimensions.test.gold_standard |
-0.07 |
-0.09 |
-0.03 |
-0.08 |
-0.10 |
msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard |
-0.08 |
-0.09 |
-0.00 |
-0.08 |
-0.09 |
mean |
-0.08 |
-0.09 |
-0.01 |
-0.08 |
-0.10 |
Percentage Unchanged Predictions Babble Noise¶
Data |
Percentage Unchanged Predictions Babble Noise |
||||
|---|---|---|---|---|---|
w2v2-b |
w2v2-L |
w2v2-L-robust |
w2v2-L-xls-r |
w2v2-L-vox |
|
iemocap-2.3.0-emotion.dimensions.test.gold_standard |
0.92 |
0.97 |
0.96 |
0.98 |
0.95 |
msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard |
0.94 |
0.95 |
0.97 |
0.95 |
0.95 |
mean |
0.93 |
0.96 |
0.96 |
0.96 |
0.95 |
Percentage Unchanged Predictions Coughing¶
Data |
Percentage Unchanged Predictions Coughing |
||||
|---|---|---|---|---|---|
w2v2-b |
w2v2-L |
w2v2-L-robust |
w2v2-L-xls-r |
w2v2-L-vox |
|
iemocap-2.3.0-emotion.dimensions.test.gold_standard |
0.71 |
0.77 |
0.83 |
0.76 |
0.79 |
msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard |
0.77 |
0.81 |
0.91 |
0.82 |
0.81 |
mean |
0.74 |
0.79 |
0.87 |
0.79 |
0.80 |
Percentage Unchanged Predictions Environmental Noise¶
Data |
Percentage Unchanged Predictions Environmental Noise |
||||
|---|---|---|---|---|---|
w2v2-b |
w2v2-L |
w2v2-L-robust |
w2v2-L-xls-r |
w2v2-L-vox |
|
iemocap-2.3.0-emotion.dimensions.test.gold_standard |
0.81 |
0.86 |
0.93 |
0.81 |
0.81 |
msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard |
0.81 |
0.83 |
0.90 |
0.87 |
0.88 |
mean |
0.81 |
0.84 |
0.92 |
0.84 |
0.84 |
Percentage Unchanged Predictions Music¶
Data |
Percentage Unchanged Predictions Music |
||||
|---|---|---|---|---|---|
w2v2-b |
w2v2-L |
w2v2-L-robust |
w2v2-L-xls-r |
w2v2-L-vox |
|
iemocap-2.3.0-emotion.dimensions.test.gold_standard |
0.87 |
0.94 |
0.96 |
0.90 |
0.89 |
msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard |
0.86 |
0.91 |
0.95 |
0.93 |
0.94 |
mean |
0.86 |
0.93 |
0.95 |
0.92 |
0.92 |
Percentage Unchanged Predictions Sneezing¶
Data |
Percentage Unchanged Predictions Sneezing |
||||
|---|---|---|---|---|---|
w2v2-b |
w2v2-L |
w2v2-L-robust |
w2v2-L-xls-r |
w2v2-L-vox |
|
iemocap-2.3.0-emotion.dimensions.test.gold_standard |
0.70 |
0.69 |
0.76 |
0.79 |
0.75 |
msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard |
0.79 |
0.75 |
0.90 |
0.88 |
0.84 |
mean |
0.74 |
0.72 |
0.83 |
0.83 |
0.79 |
Percentage Unchanged Predictions White Noise¶
Data |
Percentage Unchanged Predictions White Noise |
||||
|---|---|---|---|---|---|
w2v2-b |
w2v2-L |
w2v2-L-robust |
w2v2-L-xls-r |
w2v2-L-vox |
|
iemocap-2.3.0-emotion.dimensions.test.gold_standard |
0.26 |
0.14 |
0.70 |
0.16 |
0.10 |
msppodcast-2.6.1-emotion.dimensions.test-1.gold_standard |
0.23 |
0.12 |
0.43 |
0.23 |
0.28 |
mean |
0.24 |
0.13 |
0.56 |
0.20 |
0.19 |
Visualization Babble Noise¶
Difference of predictions for clean audio and audio with added babble noise. The allowed prediction difference \(\delta < 0.05\) is highlighted in green in the upper plot. The lower plot shows the distributions of the two predictions.
w2v2-b |
w2v2-L |
w2v2-L-robust |
w2v2-L-xls-r |
w2v2-L-vox |
|---|---|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|