Fairness linguistic sentiment¶
CNN14-cat |
w2v2-b-cat |
hubert-b-cat |
axlstm-cat |
|
|---|---|---|---|---|
Overall Score |
97.9% passed tests (94 passed / 2 failed). |
90.6% passed tests (87 passed / 9 failed). |
90.6% passed tests (87 passed / 9 failed). |
100.0% passed tests (96 passed / 0 failed). |
Class Proportion Shift Difference Negative Sentiment¶
Shift in class proportions for negative sentiment for specific language - Average of the shift in class proportions for negative sentiment for all languages. The full expression leading to the test score is displayed in parentheses.
Data |
anger |
happiness |
neutral |
sadness |
||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
CNN14-cat |
w2v2-b-cat |
hubert-b-cat |
axlstm-cat |
CNN14-cat |
w2v2-b-cat |
hubert-b-cat |
axlstm-cat |
CNN14-cat |
w2v2-b-cat |
hubert-b-cat |
axlstm-cat |
CNN14-cat |
w2v2-b-cat |
hubert-b-cat |
axlstm-cat |
|
checklist-synth-1.0.0-words-in-context-de |
0.00 (0.00 - -0.00) |
-0.01 (-0.00 - 0.01) |
-0.01 (0.00 - 0.01) |
0.00 (0.00 - -0.00) |
0.02 (0.01 - -0.01) |
0.02 (-0.00 - -0.03) |
0.01 (0.00 - -0.01) |
0.00 (0.00 - -0.00) |
-0.01 (-0.00 - 0.01) |
-0.01 (-0.02 - -0.01) |
0.03 (0.00 - -0.03) |
-0.03 (-0.01 - 0.02) |
-0.00 (-0.00 - 0.00) |
-0.00 (0.03 - 0.03) |
-0.03 (-0.00 - 0.03) |
0.02 (0.01 - -0.01) |
checklist-synth-1.0.0-words-in-context-en |
0.00 (0.00 - -0.00) |
0.01 (0.01 - 0.01) |
-0.01 (0.00 - 0.01) |
-0.00 (-0.01 - -0.00) |
0.02 (0.01 - -0.01) |
-0.08 (-0.10 - -0.03) |
-0.04 (-0.04 - -0.01) |
-0.01 (-0.01 - -0.00) |
-0.02 (-0.01 - 0.01) |
-0.05 (-0.06 - -0.01) |
-0.09 (-0.13 - -0.03) |
0.00 (0.02 - 0.02) |
-0.00 (0.00 - 0.00) |
0.12 (0.15 - 0.03) |
0.14 (0.17 - 0.03) |
0.01 (-0.00 - -0.01) |
checklist-synth-1.0.0-words-in-context-es |
0.01 (0.00 - -0.00) |
-0.01 (-0.00 - 0.01) |
-0.01 (0.00 - 0.01) |
0.00 (-0.00 - -0.00) |
-0.00 (-0.01 - -0.01) |
0.03 (0.00 - -0.03) |
0.01 (0.00 - -0.01) |
0.00 (0.00 - -0.00) |
-0.00 (0.01 - 0.01) |
0.01 (-0.00 - -0.01) |
0.03 (0.00 - -0.03) |
0.01 (0.03 - 0.02) |
-0.00 (-0.00 - 0.00) |
-0.03 (0.00 - 0.03) |
-0.03 (-0.00 - 0.03) |
-0.02 (-0.03 - -0.01) |
checklist-synth-1.0.0-words-in-context-fr |
-0.00 (-0.00 - -0.00) |
-0.00 (0.01 - 0.01) |
-0.01 (0.00 - 0.01) |
0.01 (0.00 - -0.00) |
-0.03 (-0.04 - -0.01) |
0.02 (-0.01 - -0.03) |
0.01 (0.00 - -0.01) |
-0.01 (-0.01 - -0.00) |
0.04 (0.05 - 0.01) |
-0.00 (-0.01 - -0.01) |
-0.01 (-0.04 - -0.03) |
0.00 (0.02 - 0.02) |
-0.00 (0.00 - 0.00) |
-0.02 (0.01 - 0.03) |
0.01 (0.04 - 0.03) |
-0.00 (-0.01 - -0.01) |
checklist-synth-1.0.0-words-in-context-it |
0.00 (0.00 - -0.00) |
0.03 (0.04 - 0.01) |
0.04 (0.05 - 0.01) |
-0.00 (-0.01 - -0.00) |
-0.01 (-0.02 - -0.01) |
-0.02 (-0.05 - -0.03) |
0.00 (-0.00 - -0.01) |
0.00 (-0.00 - -0.00) |
0.01 (0.02 - 0.01) |
-0.00 (-0.01 - -0.01) |
-0.00 (-0.04 - -0.03) |
0.00 (0.02 - 0.02) |
-0.00 (0.00 - 0.00) |
-0.01 (0.02 - 0.03) |
-0.04 (-0.01 - 0.03) |
0.00 (-0.01 - -0.01) |
checklist-synth-1.0.0-words-in-context-ja |
-0.02 (-0.02 - -0.00) |
-0.01 (-0.00 - 0.01) |
-0.00 (0.00 - 0.01) |
-0.01 (-0.02 - -0.00) |
0.00 (-0.01 - -0.01) |
-0.01 (-0.03 - -0.03) |
0.02 (0.01 - -0.01) |
0.00 (-0.00 - -0.00) |
0.01 (0.02 - 0.01) |
0.04 (0.03 - -0.01) |
0.01 (-0.03 - -0.03) |
0.02 (0.03 - 0.02) |
0.01 (0.01 - 0.00) |
-0.02 (0.01 - 0.03) |
-0.02 (0.01 - 0.03) |
-0.01 (-0.02 - -0.01) |
checklist-synth-1.0.0-words-in-context-pt |
0.01 (0.01 - -0.00) |
-0.00 (0.01 - 0.01) |
-0.01 (-0.00 - 0.01) |
0.00 (-0.00 - -0.00) |
0.04 (0.03 - -0.01) |
0.03 (0.01 - -0.03) |
0.01 (0.00 - -0.01) |
0.02 (0.01 - -0.00) |
-0.05 (-0.04 - 0.01) |
-0.01 (-0.02 - -0.01) |
-0.00 (-0.03 - -0.03) |
-0.01 (0.00 - 0.02) |
-0.00 (0.00 - 0.00) |
-0.02 (0.01 - 0.03) |
0.00 (0.03 - 0.03) |
-0.01 (-0.02 - -0.01) |
checklist-synth-1.0.0-words-in-context-zh |
-0.01 (-0.01 - -0.00) |
-0.00 (0.00 - 0.01) |
0.00 (0.01 - 0.01) |
0.00 (-0.00 - -0.00) |
-0.02 (-0.03 - -0.01) |
0.00 (-0.02 - -0.03) |
-0.01 (-0.01 - -0.01) |
-0.01 (-0.02 - -0.00) |
0.03 (0.04 - 0.01) |
0.03 (0.02 - -0.01) |
0.04 (0.00 - -0.03) |
0.00 (0.02 - 0.02) |
0.00 (0.00 - 0.00) |
-0.03 (-0.00 - 0.03) |
-0.03 (0.00 - 0.03) |
0.01 (0.00 - -0.01) |
mean |
-0.00 |
0.00 |
-0.00 |
0.00 |
0.00 |
-0.00 |
0.00 |
-0.00 |
0.00 |
0.00 |
0.00 |
-0.00 |
0.00 |
-0.00 |
0.00 |
-0.00 |
Class Proportion Shift Difference Neutral Sentiment¶
Shift in class proportions for neutral sentiment for specific language - Average of the shift in class proportions for neutral sentiment for all languages. The full expression leading to the test score is displayed in parentheses.
Data |
anger |
happiness |
neutral |
sadness |
||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
CNN14-cat |
w2v2-b-cat |
hubert-b-cat |
axlstm-cat |
CNN14-cat |
w2v2-b-cat |
hubert-b-cat |
axlstm-cat |
CNN14-cat |
w2v2-b-cat |
hubert-b-cat |
axlstm-cat |
CNN14-cat |
w2v2-b-cat |
hubert-b-cat |
axlstm-cat |
|
checklist-synth-1.0.0-words-in-context-de |
-0.01 (0.00 - 0.01) |
0.01 (-0.00 - -0.01) |
-0.00 (0.00 - 0.00) |
-0.01 (0.00 - 0.01) |
0.01 (-0.01 - -0.02) |
-0.00 (-0.01 - -0.00) |
0.00 (0.00 - -0.00) |
-0.01 (-0.00 - 0.01) |
-0.01 (0.00 - 0.01) |
-0.01 (0.03 - 0.04) |
-0.03 (0.01 - 0.04) |
0.03 (0.01 - -0.03) |
0.01 (0.01 - 0.00) |
0.01 (-0.02 - -0.02) |
0.03 (-0.01 - -0.04) |
-0.01 (-0.00 - 0.01) |
checklist-synth-1.0.0-words-in-context-en |
-0.01 (0.00 - 0.01) |
-0.00 (-0.01 - -0.01) |
-0.00 (0.00 - 0.00) |
-0.01 (0.00 - 0.01) |
-0.01 (-0.03 - -0.02) |
-0.13 (-0.13 - -0.00) |
-0.04 (-0.04 - -0.00) |
0.03 (0.03 - 0.01) |
0.02 (0.03 - 0.01) |
0.19 (0.23 - 0.04) |
0.15 (0.20 - 0.04) |
-0.03 (-0.05 - -0.03) |
-0.00 (0.00 - 0.00) |
-0.06 (-0.08 - -0.02) |
-0.11 (-0.15 - -0.04) |
0.01 (0.02 - 0.01) |
checklist-synth-1.0.0-words-in-context-es |
-0.02 (-0.01 - 0.01) |
0.01 (-0.00 - -0.01) |
-0.00 (0.00 - 0.00) |
-0.01 (0.00 - 0.01) |
-0.02 (-0.04 - -0.02) |
0.00 (-0.00 - -0.00) |
0.00 (0.00 - -0.00) |
-0.01 (0.00 - 0.01) |
0.04 (0.05 - 0.01) |
-0.01 (0.03 - 0.04) |
-0.04 (0.00 - 0.04) |
-0.06 (-0.09 - -0.03) |
-0.00 (-0.00 - 0.00) |
0.00 (-0.02 - -0.02) |
0.04 (-0.00 - -0.04) |
0.07 (0.08 - 0.01) |
checklist-synth-1.0.0-words-in-context-fr |
-0.02 (-0.01 - 0.01) |
-0.04 (-0.05 - -0.01) |
-0.00 (-0.00 - 0.00) |
-0.01 (-0.00 - 0.01) |
0.06 (0.04 - -0.02) |
0.02 (0.01 - -0.00) |
0.00 (0.00 - -0.00) |
0.01 (0.02 - 0.01) |
-0.03 (-0.02 - 0.01) |
0.02 (0.06 - 0.04) |
-0.01 (0.03 - 0.04) |
0.04 (0.01 - -0.03) |
-0.00 (0.00 - 0.00) |
0.00 (-0.02 - -0.02) |
0.01 (-0.03 - -0.04) |
-0.04 (-0.03 - 0.01) |
checklist-synth-1.0.0-words-in-context-it |
-0.01 (-0.00 - 0.01) |
-0.04 (-0.04 - -0.01) |
-0.05 (-0.05 - 0.00) |
-0.01 (-0.00 - 0.01) |
0.05 (0.03 - -0.02) |
0.07 (0.07 - -0.00) |
0.01 (0.01 - -0.00) |
-0.00 (0.00 - 0.01) |
-0.03 (-0.02 - 0.01) |
-0.04 (-0.01 - 0.04) |
-0.04 (-0.00 - 0.04) |
0.05 (0.02 - -0.03) |
-0.00 (0.00 - 0.00) |
0.01 (-0.02 - -0.02) |
0.08 (0.04 - -0.04) |
-0.03 (-0.02 - 0.01) |
checklist-synth-1.0.0-words-in-context-ja |
0.09 (0.10 - 0.01) |
0.11 (0.10 - -0.01) |
0.10 (0.10 - 0.00) |
0.04 (0.05 - 0.01) |
-0.09 (-0.12 - -0.02) |
-0.05 (-0.06 - -0.00) |
-0.06 (-0.06 - -0.00) |
-0.04 (-0.03 - 0.01) |
0.01 (0.02 - 0.01) |
-0.07 (-0.04 - 0.04) |
-0.07 (-0.03 - 0.04) |
0.04 (0.01 - -0.03) |
-0.00 (0.00 - 0.00) |
0.02 (-0.01 - -0.02) |
0.02 (-0.01 - -0.04) |
-0.04 (-0.03 - 0.01) |
checklist-synth-1.0.0-words-in-context-pt |
-0.04 (-0.03 - 0.01) |
-0.00 (-0.01 - -0.01) |
0.01 (0.01 - 0.00) |
-0.00 (0.01 - 0.01) |
-0.02 (-0.04 - -0.02) |
0.00 (-0.00 - -0.00) |
0.00 (-0.00 - -0.00) |
-0.00 (0.01 - 0.01) |
0.07 (0.07 - 0.01) |
-0.00 (0.04 - 0.04) |
0.09 (0.14 - 0.04) |
-0.03 (-0.05 - -0.03) |
-0.00 (0.00 - 0.00) |
-0.00 (-0.02 - -0.02) |
-0.10 (-0.14 - -0.04) |
0.03 (0.04 - 0.01) |
checklist-synth-1.0.0-words-in-context-zh |
0.04 (0.05 - 0.01) |
-0.03 (-0.04 - -0.01) |
-0.06 (-0.06 - 0.00) |
0.01 (0.02 - 0.01) |
0.02 (0.00 - -0.02) |
0.08 (0.08 - -0.00) |
0.07 (0.07 - -0.00) |
0.03 (0.03 - 0.01) |
-0.06 (-0.05 - 0.01) |
-0.08 (-0.04 - 0.04) |
-0.05 (-0.01 - 0.04) |
-0.04 (-0.06 - -0.03) |
-0.00 (-0.00 - 0.00) |
0.03 (0.00 - -0.02) |
0.03 (-0.00 - -0.04) |
0.00 (0.01 - 0.01) |
mean |
0.00 |
0.00 |
0.00 |
0.00 |
0.00 |
-0.00 |
-0.00 |
0.00 |
0.00 |
0.00 |
-0.00 |
-0.00 |
0.00 |
0.00 |
0.00 |
-0.00 |
Class Proportion Shift Difference Positive Sentiment¶
Shift in class proportions for positive sentiment for specific language - Average of the shift in class proportions for positive sentiment for all languages. The full expression leading to the test score is displayed in parentheses.
Data |
anger |
happiness |
neutral |
sadness |
||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
CNN14-cat |
w2v2-b-cat |
hubert-b-cat |
axlstm-cat |
CNN14-cat |
w2v2-b-cat |
hubert-b-cat |
axlstm-cat |
CNN14-cat |
w2v2-b-cat |
hubert-b-cat |
axlstm-cat |
CNN14-cat |
w2v2-b-cat |
hubert-b-cat |
axlstm-cat |
|
checklist-synth-1.0.0-words-in-context-de |
0.00 (0.00 - -0.00) |
0.01 (0.00 - -0.01) |
0.01 (0.00 - -0.01) |
-0.00 (0.00 - 0.00) |
-0.02 (-0.00 - 0.02) |
-0.02 (0.01 - 0.03) |
-0.01 (0.00 - 0.01) |
0.00 (0.00 - 0.00) |
0.02 (0.00 - -0.01) |
0.02 (0.01 - -0.00) |
-0.02 (-0.01 - 0.02) |
0.02 (0.01 - -0.01) |
0.00 (-0.00 - -0.00) |
-0.00 (-0.02 - -0.02) |
0.02 (0.01 - -0.02) |
-0.02 (-0.01 - 0.01) |
checklist-synth-1.0.0-words-in-context-en |
0.00 (0.00 - -0.00) |
-0.00 (-0.01 - -0.01) |
0.01 (0.00 - -0.01) |
0.01 (0.01 - 0.00) |
-0.01 (0.01 - 0.02) |
0.13 (0.16 - 0.03) |
0.06 (0.06 - 0.01) |
-0.00 (-0.00 - 0.00) |
0.01 (-0.01 - -0.01) |
-0.03 (-0.03 - -0.00) |
0.03 (0.05 - 0.02) |
0.01 (-0.00 - -0.01) |
0.00 (0.00 - -0.00) |
-0.10 (-0.12 - -0.02) |
-0.10 (-0.12 - -0.02) |
-0.01 (-0.00 - 0.01) |
checklist-synth-1.0.0-words-in-context-es |
0.00 (-0.00 - -0.00) |
0.01 (0.00 - -0.01) |
0.01 (0.00 - -0.01) |
-0.00 (0.00 - 0.00) |
0.01 (0.03 - 0.02) |
-0.03 (-0.00 - 0.03) |
-0.01 (0.00 - 0.01) |
-0.00 (-0.00 - 0.00) |
-0.01 (-0.03 - -0.01) |
-0.00 (-0.01 - -0.00) |
-0.02 (-0.00 - 0.02) |
0.01 (0.01 - -0.01) |
0.00 (0.00 - -0.00) |
0.03 (0.01 - -0.02) |
0.02 (0.00 - -0.02) |
-0.01 (-0.01 - 0.01) |
checklist-synth-1.0.0-words-in-context-fr |
0.01 (0.01 - -0.00) |
0.02 (0.01 - -0.01) |
0.01 (-0.00 - -0.01) |
-0.00 (-0.00 - 0.00) |
0.01 (0.03 - 0.02) |
-0.03 (0.00 - 0.03) |
-0.01 (0.00 - 0.01) |
0.00 (0.01 - 0.00) |
-0.03 (-0.04 - -0.01) |
-0.01 (-0.01 - -0.00) |
0.02 (0.04 - 0.02) |
-0.02 (-0.03 - -0.01) |
0.00 (0.00 - -0.00) |
0.02 (-0.00 - -0.02) |
-0.02 (-0.03 - -0.02) |
0.02 (0.02 - 0.01) |
checklist-synth-1.0.0-words-in-context-it |
0.00 (-0.00 - -0.00) |
-0.02 (-0.02 - -0.01) |
-0.03 (-0.03 - -0.01) |
0.01 (0.01 - 0.00) |
-0.01 (0.01 - 0.02) |
-0.01 (0.02 - 0.03) |
-0.01 (0.00 - 0.01) |
0.00 (0.00 - 0.00) |
0.00 (-0.01 - -0.01) |
0.02 (0.02 - -0.00) |
0.02 (0.04 - 0.02) |
-0.02 (-0.03 - -0.01) |
0.00 (0.00 - -0.00) |
0.00 (-0.02 - -0.02) |
0.01 (-0.01 - -0.02) |
0.01 (0.02 - 0.01) |
checklist-synth-1.0.0-words-in-context-ja |
-0.01 (-0.02 - -0.00) |
-0.03 (-0.04 - -0.01) |
-0.04 (-0.04 - -0.01) |
-0.00 (-0.00 - 0.00) |
0.03 (0.05 - 0.02) |
0.03 (0.06 - 0.03) |
0.00 (0.01 - 0.01) |
0.01 (0.01 - 0.00) |
-0.01 (-0.03 - -0.01) |
-0.01 (-0.02 - -0.00) |
0.02 (0.04 - 0.02) |
-0.03 (-0.04 - -0.01) |
-0.01 (-0.01 - -0.00) |
0.01 (-0.01 - -0.02) |
0.01 (-0.00 - -0.02) |
0.02 (0.03 - 0.01) |
checklist-synth-1.0.0-words-in-context-pt |
0.00 (-0.00 - -0.00) |
0.00 (-0.00 - -0.01) |
0.01 (-0.00 - -0.01) |
-0.00 (-0.00 - 0.00) |
-0.03 (-0.01 - 0.02) |
-0.04 (-0.01 - 0.03) |
-0.01 (-0.00 - 0.01) |
-0.02 (-0.02 - 0.00) |
0.03 (0.01 - -0.01) |
0.01 (0.01 - -0.00) |
-0.03 (-0.02 - 0.02) |
0.02 (0.02 - -0.01) |
0.00 (0.00 - -0.00) |
0.02 (0.00 - -0.02) |
0.03 (0.02 - -0.02) |
-0.00 (0.00 - 0.01) |
checklist-synth-1.0.0-words-in-context-zh |
-0.01 (-0.01 - -0.00) |
0.02 (0.01 - -0.01) |
0.02 (0.02 - -0.01) |
-0.00 (-0.00 - 0.00) |
0.01 (0.03 - 0.02) |
-0.04 (-0.01 - 0.03) |
-0.02 (-0.01 - 0.01) |
0.00 (0.00 - 0.00) |
-0.00 (-0.02 - -0.01) |
0.00 (-0.00 - -0.00) |
-0.02 (-0.00 - 0.02) |
0.01 (0.01 - -0.01) |
0.00 (-0.00 - -0.00) |
0.02 (-0.00 - -0.02) |
0.02 (-0.00 - -0.02) |
-0.01 (-0.01 - 0.01) |
mean |
-0.00 |
0.00 |
-0.00 |
0.00 |
-0.00 |
-0.00 |
-0.00 |
-0.00 |
0.00 |
0.00 |
-0.00 |
-0.00 |
-0.00 |
0.00 |
-0.00 |
0.00 |
Visualization¶
CNN14-cat |
w2v2-b-cat |
hubert-b-cat |
axlstm-cat |
|---|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|