Fairness accent

Overall scores

w2v2-L

hubert-L

wavlm

data2vec

Overall Score

93.5% passed tests (87 passed / 6 failed).

100.0% passed tests (93 passed / 0 failed).

96.0% passed tests (119 passed / 5 failed).

96.8% passed tests (120 passed / 4 failed).

Mean Value

Threshold: 0.075

Data

Mean Value

w2v2-L

hubert-L

wavlm

data2vec

speech-accent-archive-2.2.0-amharic

-0.04

-0.04

-0.07

-0.06

speech-accent-archive-2.2.0-arabic

0.01

0.01

0.00

-0.01

speech-accent-archive-2.2.0-bengali

0.01

-0.01

0.01

0.00

speech-accent-archive-2.2.0-bulgarian

-0.04

-0.02

-0.05

-0.03

speech-accent-archive-2.2.0-cantonese

0.00

0.01

-0.05

0.00

speech-accent-archive-2.2.0-dutch

0.00

0.01

0.00

0.02

speech-accent-archive-2.2.0-english

0.02

0.03

-0.00

0.04

speech-accent-archive-2.2.0-farsi

0.00

0.00

0.01

0.01

speech-accent-archive-2.2.0-french

-0.04

-0.03

-0.02

-0.04

speech-accent-archive-2.2.0-german

0.03

0.04

0.06

0.06

speech-accent-archive-2.2.0-greek

0.03

0.03

0.04

0.04

speech-accent-archive-2.2.0-hindi

0.01

0.02

0.01

0.01

speech-accent-archive-2.2.0-italian

0.01

-0.01

0.02

-0.01

speech-accent-archive-2.2.0-japanese

0.01

0.01

0.02

0.01

speech-accent-archive-2.2.0-korean

-0.01

-0.01

-0.01

-0.03

speech-accent-archive-2.2.0-macedonian

0.06

0.05

0.10

0.05

speech-accent-archive-2.2.0-mandarin

0.05

0.04

0.03

0.05

speech-accent-archive-2.2.0-nepali

-0.02

-0.02

-0.01

-0.02

speech-accent-archive-2.2.0-polish

-0.00

0.01

0.02

0.02

speech-accent-archive-2.2.0-portuguese

-0.00

0.00

0.00

0.01

speech-accent-archive-2.2.0-romanian

-0.02

-0.03

-0.02

-0.03

speech-accent-archive-2.2.0-russian

0.02

0.00

0.02

0.04

speech-accent-archive-2.2.0-serbian

-0.04

-0.03

-0.02

-0.03

speech-accent-archive-2.2.0-spanish

-0.03

-0.02

-0.01

-0.03

speech-accent-archive-2.2.0-swedish

-0.01

-0.01

-0.01

0.00

speech-accent-archive-2.2.0-tagalog

0.02

0.01

-0.00

0.03

speech-accent-archive-2.2.0-thai

-0.00

-0.01

0.00

-0.02

speech-accent-archive-2.2.0-turkish

0.03

0.02

0.04

0.02

speech-accent-archive-2.2.0-ukrainian

0.00

-0.02

-0.01

-0.01

speech-accent-archive-2.2.0-urdu

0.01

0.02

0.02

0.03

speech-accent-archive-2.2.0-vietnamese

-0.04

-0.04

-0.05

-0.06

mean

0.00

0.00

0.00

0.00

Relative Difference Per Bin

Bins with too few samples are skipped.

Threshold: 0.225

Data

(-inf, 0.25]

(0.25, 0.5]

(0.5, 0.75]

(0.75, inf]

w2v2-L

hubert-L

wavlm

data2vec

w2v2-L

hubert-L

wavlm

data2vec

w2v2-L

hubert-L

wavlm

data2vec

w2v2-L

hubert-L

wavlm

data2vec

speech-accent-archive-2.2.0-amharic

-0.00

-0.01

0.19

0.15

0.16

0.26

-0.19

-0.14

-0.16

-0.25

speech-accent-archive-2.2.0-arabic

-0.01

-0.01

-0.01

-0.03

-0.00

0.08

0.01

0.03

0.02

-0.07

speech-accent-archive-2.2.0-bengali

-0.01

-0.00

-0.05

-0.00

0.04

0.02

0.05

0.00

-0.03

-0.02

speech-accent-archive-2.2.0-bulgarian

-0.01

-0.02

0.24

0.12

0.16

0.17

-0.24

-0.12

-0.15

-0.15

speech-accent-archive-2.2.0-cantonese

0.01

-0.02

0.01

-0.09

0.13

0.03

-0.01

0.09

-0.14

-0.01

speech-accent-archive-2.2.0-dutch

-0.01

-0.00

-0.04

0.04

0.01

-0.01

0.04

-0.03

0.01

-0.00

speech-accent-archive-2.2.0-english

0.00

-0.00

-0.17

-0.17

-0.08

-0.20

0.17

0.17

0.08

0.21

speech-accent-archive-2.2.0-farsi

0.00

-0.02

0.01

0.03

0.02

0.03

-0.01

-0.03

-0.02

-0.01

speech-accent-archive-2.2.0-french

-0.00

0.03

0.18

0.11

0.11

0.15

-0.18

-0.11

-0.11

-0.17

speech-accent-archive-2.2.0-german

-0.01

-0.02

-0.19

-0.21

-0.26

-0.22

0.19

0.21

0.27

0.24

speech-accent-archive-2.2.0-greek

-0.01

-0.02

-0.17

-0.10

-0.14

-0.13

0.17

0.10

0.15

0.15

speech-accent-archive-2.2.0-hindi

-0.01

0.02

-0.04

-0.03

-0.00

-0.05

0.04

0.03

0.01

0.04

speech-accent-archive-2.2.0-italian

-0.01

0.01

-0.08

0.12

-0.07

0.04

0.08

-0.12

0.08

-0.07

speech-accent-archive-2.2.0-japanese

0.05

-0.02

-0.08

-0.03

-0.15

-0.08

0.08

0.04

0.10

0.10

speech-accent-archive-2.2.0-korean

-0.01

0.04

0.08

0.03

0.05

0.04

-0.08

-0.02

-0.04

-0.07

speech-accent-archive-2.2.0-macedonian

-0.01

-0.00

-0.26

-0.17

-0.50

-0.18

0.26

0.17

0.51

0.18

speech-accent-archive-2.2.0-mandarin

-0.01

-0.02

-0.23

-0.18

-0.03

-0.22

0.23

0.18

0.04

0.24

speech-accent-archive-2.2.0-nepali

-0.00

0.00

0.11

0.09

0.11

0.08

-0.11

-0.09

-0.11

-0.09

speech-accent-archive-2.2.0-polish

0.00

-0.02

-0.01

-0.03

-0.09

-0.02

0.01

0.04

0.09

0.04

speech-accent-archive-2.2.0-portuguese

-0.01

-0.02

0.01

0.03

0.06

-0.06

-0.01

-0.03

-0.05

0.08

speech-accent-archive-2.2.0-romanian

0.01

-0.02

0.20

0.14

0.09

0.15

-0.20

-0.13

-0.10

-0.14

speech-accent-archive-2.2.0-russian

-0.01

-0.02

-0.06

-0.03

0.02

-0.17

0.06

0.03

-0.00

0.19

speech-accent-archive-2.2.0-serbian

0.07

0.12

0.10

0.10

0.02

-0.04

-0.10

-0.10

-0.08

-0.08

speech-accent-archive-2.2.0-spanish

-0.00

0.01

0.10

-0.01

0.07

0.02

-0.10

-0.01

-0.07

-0.03

speech-accent-archive-2.2.0-swedish

-0.01

-0.02

0.09

0.06

0.08

0.07

-0.09

-0.05

-0.07

-0.06

speech-accent-archive-2.2.0-tagalog

0.03

0.01

-0.14

-0.12

-0.03

-0.16

0.14

0.12

-0.01

0.14

speech-accent-archive-2.2.0-thai

-0.01

-0.01

0.06

0.04

0.05

0.07

-0.06

-0.05

-0.04

-0.06

speech-accent-archive-2.2.0-turkish

-0.01

-0.00

-0.13

-0.06

-0.15

-0.08

0.13

0.06

0.16

0.09

speech-accent-archive-2.2.0-ukrainian

0.01

-0.02

-0.01

0.12

0.11

0.10

0.01

-0.12

-0.12

-0.08

speech-accent-archive-2.2.0-urdu

-0.01

-0.02

-0.05

-0.13

-0.08

-0.09

0.05

0.13

0.10

0.11

speech-accent-archive-2.2.0-vietnamese

0.01

0.01

0.20

0.13

0.12

0.16

-0.20

-0.13

-0.13

-0.18

mean

0.00

-0.00

-0.00

-0.00

-0.01

-0.01

0.00

0.00

0.01

0.01

Visualization

w2v2-L

hubert-L

wavlm

data2vec

../../../_images/visualization_speech-accent-archive-2.2.0-amharic15.png
../../../_images/visualization_speech-accent-archive-2.2.0-amharic19.png
../../../_images/visualization_speech-accent-archive-2.2.0-amharic20.png
../../../_images/visualization_speech-accent-archive-2.2.0-amharic21.png
../../../_images/visualization_speech-accent-archive-2.2.0-arabic15.png
../../../_images/visualization_speech-accent-archive-2.2.0-arabic19.png
../../../_images/visualization_speech-accent-archive-2.2.0-arabic20.png
../../../_images/visualization_speech-accent-archive-2.2.0-arabic21.png
../../../_images/visualization_speech-accent-archive-2.2.0-bengali15.png
../../../_images/visualization_speech-accent-archive-2.2.0-bengali19.png
../../../_images/visualization_speech-accent-archive-2.2.0-bengali20.png
../../../_images/visualization_speech-accent-archive-2.2.0-bengali21.png
../../../_images/visualization_speech-accent-archive-2.2.0-bulgarian15.png
../../../_images/visualization_speech-accent-archive-2.2.0-bulgarian19.png
../../../_images/visualization_speech-accent-archive-2.2.0-bulgarian20.png
../../../_images/visualization_speech-accent-archive-2.2.0-bulgarian21.png
../../../_images/visualization_speech-accent-archive-2.2.0-cantonese15.png
../../../_images/visualization_speech-accent-archive-2.2.0-cantonese19.png
../../../_images/visualization_speech-accent-archive-2.2.0-cantonese20.png
../../../_images/visualization_speech-accent-archive-2.2.0-cantonese21.png
../../../_images/visualization_speech-accent-archive-2.2.0-dutch15.png
../../../_images/visualization_speech-accent-archive-2.2.0-dutch19.png
../../../_images/visualization_speech-accent-archive-2.2.0-dutch20.png
../../../_images/visualization_speech-accent-archive-2.2.0-dutch21.png
../../../_images/visualization_speech-accent-archive-2.2.0-english15.png
../../../_images/visualization_speech-accent-archive-2.2.0-english19.png
../../../_images/visualization_speech-accent-archive-2.2.0-english20.png
../../../_images/visualization_speech-accent-archive-2.2.0-english21.png
../../../_images/visualization_speech-accent-archive-2.2.0-farsi15.png
../../../_images/visualization_speech-accent-archive-2.2.0-farsi19.png
../../../_images/visualization_speech-accent-archive-2.2.0-farsi20.png
../../../_images/visualization_speech-accent-archive-2.2.0-farsi21.png
../../../_images/visualization_speech-accent-archive-2.2.0-french15.png
../../../_images/visualization_speech-accent-archive-2.2.0-french19.png
../../../_images/visualization_speech-accent-archive-2.2.0-french20.png
../../../_images/visualization_speech-accent-archive-2.2.0-french21.png
../../../_images/visualization_speech-accent-archive-2.2.0-german15.png
../../../_images/visualization_speech-accent-archive-2.2.0-german19.png
../../../_images/visualization_speech-accent-archive-2.2.0-german20.png
../../../_images/visualization_speech-accent-archive-2.2.0-german21.png
../../../_images/visualization_speech-accent-archive-2.2.0-greek15.png
../../../_images/visualization_speech-accent-archive-2.2.0-greek19.png
../../../_images/visualization_speech-accent-archive-2.2.0-greek20.png
../../../_images/visualization_speech-accent-archive-2.2.0-greek21.png
../../../_images/visualization_speech-accent-archive-2.2.0-hindi15.png
../../../_images/visualization_speech-accent-archive-2.2.0-hindi19.png
../../../_images/visualization_speech-accent-archive-2.2.0-hindi20.png
../../../_images/visualization_speech-accent-archive-2.2.0-hindi21.png
../../../_images/visualization_speech-accent-archive-2.2.0-italian15.png
../../../_images/visualization_speech-accent-archive-2.2.0-italian19.png
../../../_images/visualization_speech-accent-archive-2.2.0-italian20.png
../../../_images/visualization_speech-accent-archive-2.2.0-italian21.png
../../../_images/visualization_speech-accent-archive-2.2.0-japanese15.png
../../../_images/visualization_speech-accent-archive-2.2.0-japanese19.png
../../../_images/visualization_speech-accent-archive-2.2.0-japanese20.png
../../../_images/visualization_speech-accent-archive-2.2.0-japanese21.png
../../../_images/visualization_speech-accent-archive-2.2.0-korean15.png
../../../_images/visualization_speech-accent-archive-2.2.0-korean19.png
../../../_images/visualization_speech-accent-archive-2.2.0-korean20.png
../../../_images/visualization_speech-accent-archive-2.2.0-korean21.png
../../../_images/visualization_speech-accent-archive-2.2.0-macedonian15.png
../../../_images/visualization_speech-accent-archive-2.2.0-macedonian19.png
../../../_images/visualization_speech-accent-archive-2.2.0-macedonian20.png
../../../_images/visualization_speech-accent-archive-2.2.0-macedonian21.png
../../../_images/visualization_speech-accent-archive-2.2.0-mandarin15.png
../../../_images/visualization_speech-accent-archive-2.2.0-mandarin19.png
../../../_images/visualization_speech-accent-archive-2.2.0-mandarin20.png
../../../_images/visualization_speech-accent-archive-2.2.0-mandarin21.png
../../../_images/visualization_speech-accent-archive-2.2.0-nepali15.png
../../../_images/visualization_speech-accent-archive-2.2.0-nepali19.png
../../../_images/visualization_speech-accent-archive-2.2.0-nepali20.png
../../../_images/visualization_speech-accent-archive-2.2.0-nepali21.png
../../../_images/visualization_speech-accent-archive-2.2.0-polish15.png
../../../_images/visualization_speech-accent-archive-2.2.0-polish19.png
../../../_images/visualization_speech-accent-archive-2.2.0-polish20.png
../../../_images/visualization_speech-accent-archive-2.2.0-polish21.png
../../../_images/visualization_speech-accent-archive-2.2.0-portuguese15.png
../../../_images/visualization_speech-accent-archive-2.2.0-portuguese19.png
../../../_images/visualization_speech-accent-archive-2.2.0-portuguese20.png
../../../_images/visualization_speech-accent-archive-2.2.0-portuguese21.png
../../../_images/visualization_speech-accent-archive-2.2.0-romanian15.png
../../../_images/visualization_speech-accent-archive-2.2.0-romanian19.png
../../../_images/visualization_speech-accent-archive-2.2.0-romanian20.png
../../../_images/visualization_speech-accent-archive-2.2.0-romanian21.png
../../../_images/visualization_speech-accent-archive-2.2.0-russian15.png
../../../_images/visualization_speech-accent-archive-2.2.0-russian19.png
../../../_images/visualization_speech-accent-archive-2.2.0-russian20.png
../../../_images/visualization_speech-accent-archive-2.2.0-russian21.png
../../../_images/visualization_speech-accent-archive-2.2.0-serbian15.png
../../../_images/visualization_speech-accent-archive-2.2.0-serbian19.png
../../../_images/visualization_speech-accent-archive-2.2.0-serbian20.png
../../../_images/visualization_speech-accent-archive-2.2.0-serbian21.png
../../../_images/visualization_speech-accent-archive-2.2.0-spanish15.png
../../../_images/visualization_speech-accent-archive-2.2.0-spanish19.png
../../../_images/visualization_speech-accent-archive-2.2.0-spanish20.png
../../../_images/visualization_speech-accent-archive-2.2.0-spanish21.png
../../../_images/visualization_speech-accent-archive-2.2.0-swedish15.png
../../../_images/visualization_speech-accent-archive-2.2.0-swedish19.png
../../../_images/visualization_speech-accent-archive-2.2.0-swedish20.png
../../../_images/visualization_speech-accent-archive-2.2.0-swedish21.png
../../../_images/visualization_speech-accent-archive-2.2.0-tagalog15.png
../../../_images/visualization_speech-accent-archive-2.2.0-tagalog19.png
../../../_images/visualization_speech-accent-archive-2.2.0-tagalog20.png
../../../_images/visualization_speech-accent-archive-2.2.0-tagalog21.png
../../../_images/visualization_speech-accent-archive-2.2.0-thai15.png
../../../_images/visualization_speech-accent-archive-2.2.0-thai19.png
../../../_images/visualization_speech-accent-archive-2.2.0-thai20.png
../../../_images/visualization_speech-accent-archive-2.2.0-thai21.png
../../../_images/visualization_speech-accent-archive-2.2.0-turkish15.png
../../../_images/visualization_speech-accent-archive-2.2.0-turkish19.png
../../../_images/visualization_speech-accent-archive-2.2.0-turkish20.png
../../../_images/visualization_speech-accent-archive-2.2.0-turkish21.png
../../../_images/visualization_speech-accent-archive-2.2.0-ukrainian15.png
../../../_images/visualization_speech-accent-archive-2.2.0-ukrainian19.png
../../../_images/visualization_speech-accent-archive-2.2.0-ukrainian20.png
../../../_images/visualization_speech-accent-archive-2.2.0-ukrainian21.png
../../../_images/visualization_speech-accent-archive-2.2.0-urdu15.png
../../../_images/visualization_speech-accent-archive-2.2.0-urdu19.png
../../../_images/visualization_speech-accent-archive-2.2.0-urdu20.png
../../../_images/visualization_speech-accent-archive-2.2.0-urdu21.png
../../../_images/visualization_speech-accent-archive-2.2.0-vietnamese15.png
../../../_images/visualization_speech-accent-archive-2.2.0-vietnamese19.png
../../../_images/visualization_speech-accent-archive-2.2.0-vietnamese20.png
../../../_images/visualization_speech-accent-archive-2.2.0-vietnamese21.png