Correctness consistency¶
w2v2-L |
hubert-L |
wavlm |
data2vec |
|
|---|---|---|---|---|
Overall Score |
59.6% passed tests (28 passed / 19 failed). |
36.2% passed tests (17 passed / 30 failed). |
55.3% passed tests (26 passed / 21 failed). |
29.8% passed tests (14 passed / 33 failed). |
Samples In Expected High Range¶
Proportion of samples whose predictions fall into the expected value range of >= 0.55
Data |
happiness |
|||
|---|---|---|---|---|
w2v2-L |
hubert-L |
wavlm |
data2vec |
|
crema-d-1.2.0-emotion.categories.test.gold_standard |
0.10 |
0.45 |
0.41 |
0.40 |
danish-emotional-speech-1.1.1-emotion.test |
0.04 |
0.63 |
0.37 |
0.52 |
emodb-1.2.0-emotion.categories.test.gold_standard |
0.19 |
0.11 |
0.74 |
0.22 |
emovo-1.2.1-emotion.test |
0.05 |
0.33 |
0.26 |
0.30 |
iemocap-2.3.0-emotion.categories.test.gold_standard |
0.40 |
0.58 |
0.43 |
0.58 |
meld-1.3.1-emotion.categories.test.gold_standard |
0.68 |
0.83 |
0.81 |
0.75 |
polish-emotional-speech-1.1.1-emotion.categories.test.gold_standard |
0.42 |
0.62 |
0.75 |
0.57 |
ravdess-1.1.2-emotion.speech.test |
0.00 |
0.00 |
0.19 |
0.00 |
mean |
0.23 |
0.44 |
0.49 |
0.42 |
Samples In Expected Low Range¶
Proportion of samples whose predictions fall into the expected value range of <= 0.45
Data |
anger |
disgust |
fear |
frustration |
sadness |
|||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
w2v2-L |
hubert-L |
wavlm |
data2vec |
w2v2-L |
hubert-L |
wavlm |
data2vec |
w2v2-L |
hubert-L |
wavlm |
data2vec |
w2v2-L |
hubert-L |
wavlm |
data2vec |
w2v2-L |
hubert-L |
wavlm |
data2vec |
|
crema-d-1.2.0-emotion.categories.test.gold_standard |
0.99 |
0.63 |
0.85 |
0.84 |
1.00 |
0.52 |
0.89 |
0.55 |
0.99 |
0.60 |
0.87 |
0.45 |
1.00 |
0.72 |
0.97 |
0.51 |
||||
danish-emotional-speech-1.1.1-emotion.test |
0.98 |
0.15 |
0.56 |
0.17 |
0.98 |
0.21 |
0.94 |
0.08 |
||||||||||||
emodb-1.2.0-emotion.categories.test.gold_standard |
1.00 |
0.65 |
0.64 |
0.67 |
0.96 |
0.58 |
0.50 |
0.38 |
0.88 |
0.48 |
0.42 |
0.30 |
1.00 |
0.93 |
1.00 |
0.33 |
||||
emovo-1.2.1-emotion.test |
1.00 |
0.79 |
0.79 |
0.67 |
0.94 |
0.50 |
0.61 |
0.30 |
0.95 |
0.54 |
0.51 |
0.25 |
1.00 |
0.51 |
0.88 |
0.26 |
||||
iemocap-2.3.0-emotion.categories.test.gold_standard |
0.72 |
0.64 |
0.77 |
0.71 |
0.65 |
0.53 |
0.71 |
0.65 |
0.58 |
0.58 |
0.75 |
0.57 |
0.86 |
0.73 |
0.92 |
0.52 |
||||
meld-1.3.1-emotion.categories.test.gold_standard |
0.37 |
0.35 |
0.45 |
0.44 |
0.22 |
0.39 |
0.37 |
0.37 |
0.18 |
0.26 |
0.26 |
0.26 |
0.35 |
0.42 |
0.51 |
0.37 |
||||
polish-emotional-speech-1.1.1-emotion.categories.test.gold_standard |
0.85 |
0.25 |
0.78 |
0.38 |
0.55 |
0.20 |
0.65 |
0.12 |
0.55 |
0.28 |
1.00 |
0.00 |
||||||||
ravdess-1.1.2-emotion.speech.test |
1.00 |
1.00 |
0.94 |
1.00 |
1.00 |
0.94 |
1.00 |
1.00 |
0.97 |
0.88 |
0.75 |
0.97 |
1.00 |
0.84 |
0.88 |
0.97 |
||||
mean |
0.86 |
0.56 |
0.72 |
0.61 |
0.82 |
0.59 |
0.67 |
0.52 |
0.74 |
0.50 |
0.60 |
0.43 |
0.58 |
0.58 |
0.75 |
0.57 |
0.84 |
0.58 |
0.89 |
0.38 |
Samples In Expected Neutral Range¶
Proportion of samples whose predictions fall into the expected value range of [0.3, 0.6]
Data |
boredom |
neutral |
||||||
|---|---|---|---|---|---|---|---|---|
w2v2-L |
hubert-L |
wavlm |
data2vec |
w2v2-L |
hubert-L |
wavlm |
data2vec |
|
crema-d-1.2.0-emotion.categories.test.gold_standard |
0.78 |
0.95 |
0.88 |
0.98 |
||||
danish-emotional-speech-1.1.1-emotion.test |
0.81 |
0.98 |
0.98 |
1.00 |
||||
emodb-1.2.0-emotion.categories.test.gold_standard |
1.00 |
1.00 |
0.97 |
1.00 |
1.00 |
1.00 |
1.00 |
1.00 |
emovo-1.2.1-emotion.test |
0.92 |
0.98 |
1.00 |
1.00 |
||||
iemocap-2.3.0-emotion.categories.test.gold_standard |
0.87 |
0.92 |
0.88 |
0.94 |
||||
meld-1.3.1-emotion.categories.test.gold_standard |
0.72 |
0.78 |
0.79 |
0.74 |
||||
polish-emotional-speech-1.1.1-emotion.categories.test.gold_standard |
1.00 |
1.00 |
0.92 |
1.00 |
1.00 |
1.00 |
1.00 |
1.00 |
ravdess-1.1.2-emotion.speech.test |
0.69 |
1.00 |
1.00 |
1.00 |
||||
mean |
1.00 |
1.00 |
0.95 |
1.00 |
0.85 |
0.95 |
0.94 |
0.96 |
Visualization¶
Distribution of dimensional model predictions for samples with different categorical emotions. The expected range of model predictions is highlighted by the green brackground.
w2v2-L |
hubert-L |
wavlm |
data2vec |
|---|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|