Fairness accent

Overall scores

w2v2-b

w2v2-L

w2v2-L-robust

w2v2-L-xls-r

w2v2-L-vox

Overall Score

98.4% passed tests (122 passed / 2 failed).

93.5% passed tests (87 passed / 6 failed).

89.2% passed tests (83 passed / 10 failed).

97.8% passed tests (91 passed / 2 failed).

94.4% passed tests (117 passed / 7 failed).

Mean Value

Threshold: 0.075

Data

Mean Value

w2v2-b

w2v2-L

w2v2-L-robust

w2v2-L-xls-r

w2v2-L-vox

speech-accent-archive-2.2.0-amharic

-0.07

-0.04

-0.05

-0.04

-0.07

speech-accent-archive-2.2.0-arabic

0.02

0.01

0.01

-0.00

0.00

speech-accent-archive-2.2.0-bengali

0.00

0.01

-0.01

-0.00

0.00

speech-accent-archive-2.2.0-bulgarian

-0.04

-0.04

-0.02

-0.05

-0.03

speech-accent-archive-2.2.0-cantonese

-0.01

0.00

-0.00

-0.01

-0.02

speech-accent-archive-2.2.0-dutch

0.01

0.00

0.02

0.00

-0.00

speech-accent-archive-2.2.0-english

0.04

0.02

0.04

0.01

0.03

speech-accent-archive-2.2.0-farsi

-0.01

0.00

-0.00

0.00

-0.00

speech-accent-archive-2.2.0-french

-0.04

-0.04

-0.04

-0.03

-0.03

speech-accent-archive-2.2.0-german

0.04

0.03

0.05

0.04

0.04

speech-accent-archive-2.2.0-greek

0.05

0.03

0.03

0.04

0.04

speech-accent-archive-2.2.0-hindi

0.02

0.01

0.02

0.02

0.02

speech-accent-archive-2.2.0-italian

0.02

0.01

0.00

0.01

0.01

speech-accent-archive-2.2.0-japanese

0.01

0.01

0.01

0.02

0.04

speech-accent-archive-2.2.0-korean

-0.01

-0.01

-0.03

-0.03

-0.02

speech-accent-archive-2.2.0-macedonian

0.03

0.06

0.04

0.04

0.04

speech-accent-archive-2.2.0-mandarin

0.07

0.05

0.05

0.05

0.05

speech-accent-archive-2.2.0-nepali

-0.04

-0.02

-0.02

-0.01

-0.02

speech-accent-archive-2.2.0-polish

0.01

-0.00

0.01

0.01

0.01

speech-accent-archive-2.2.0-portuguese

-0.01

-0.00

-0.00

0.00

-0.01

speech-accent-archive-2.2.0-romanian

-0.04

-0.02

-0.03

-0.02

-0.02

speech-accent-archive-2.2.0-russian

0.03

0.02

0.03

0.02

0.02

speech-accent-archive-2.2.0-serbian

-0.05

-0.04

-0.03

-0.04

-0.02

speech-accent-archive-2.2.0-spanish

-0.03

-0.03

-0.03

-0.03

-0.04

speech-accent-archive-2.2.0-swedish

-0.01

-0.01

0.00

-0.01

-0.00

speech-accent-archive-2.2.0-tagalog

0.05

0.02

0.02

0.02

0.03

speech-accent-archive-2.2.0-thai

-0.01

-0.00

-0.01

-0.02

-0.01

speech-accent-archive-2.2.0-turkish

0.04

0.03

0.04

0.04

0.03

speech-accent-archive-2.2.0-ukrainian

0.02

0.00

-0.01

0.01

-0.01

speech-accent-archive-2.2.0-urdu

0.01

0.01

0.02

0.02

0.02

speech-accent-archive-2.2.0-vietnamese

-0.05

-0.04

-0.04

-0.03

-0.04

mean

0.00

0.00

0.00

0.00

0.00

Relative Difference Per Bin

Bins with too few samples are skipped.

Threshold: 0.225

Data

(-inf, 0.25]

(0.25, 0.5]

(0.5, 0.75]

(0.75, inf]

w2v2-b

w2v2-L

w2v2-L-robust

w2v2-L-xls-r

w2v2-L-vox

w2v2-b

w2v2-L

w2v2-L-robust

w2v2-L-xls-r

w2v2-L-vox

w2v2-b

w2v2-L

w2v2-L-robust

w2v2-L-xls-r

w2v2-L-vox

w2v2-b

w2v2-L

w2v2-L-robust

w2v2-L-xls-r

w2v2-L-vox

speech-accent-archive-2.2.0-amharic

0.04

0.02

0.16

0.19

0.22

0.20

0.21

-0.20

-0.19

-0.22

-0.20

-0.23

speech-accent-archive-2.2.0-arabic

-0.02

-0.00

-0.05

-0.01

0.02

0.03

0.00

0.07

0.01

-0.02

-0.02

-0.00

speech-accent-archive-2.2.0-bengali

-0.02

-0.00

0.00

-0.05

0.05

0.06

0.03

0.02

0.05

-0.05

-0.06

-0.03

speech-accent-archive-2.2.0-bulgarian

-0.00

-0.00

0.17

0.24

0.15

0.17

0.20

-0.17

-0.24

-0.15

-0.17

-0.20

speech-accent-archive-2.2.0-cantonese

-0.02

-0.00

0.07

0.01

-0.01

0.07

0.13

-0.05

-0.01

0.01

-0.06

-0.13

speech-accent-archive-2.2.0-dutch

-0.02

-0.00

0.01

-0.04

-0.08

0.04

0.12

0.01

0.04

0.08

-0.04

-0.12

speech-accent-archive-2.2.0-english

-0.00

-0.00

-0.18

-0.17

-0.24

-0.20

-0.29

0.18

0.17

0.24

0.20

0.29

speech-accent-archive-2.2.0-farsi

-0.02

-0.00

0.18

0.01

0.04

-0.02

0.03

-0.17

-0.01

-0.04

0.02

-0.03

speech-accent-archive-2.2.0-french

0.04

0.01

0.11

0.18

0.16

0.12

0.12

-0.14

-0.18

-0.16

-0.12

-0.13

speech-accent-archive-2.2.0-german

-0.02

-0.00

-0.06

-0.19

-0.28

-0.19

-0.17

0.08

0.19

0.28

0.19

0.17

speech-accent-archive-2.2.0-greek

-0.02

-0.00

-0.11

-0.17

-0.17

-0.21

-0.20

0.13

0.17

0.17

0.21

0.21

speech-accent-archive-2.2.0-hindi

0.01

-0.00

-0.15

-0.04

-0.05

-0.04

-0.02

0.14

0.04

0.05

0.04

0.02

speech-accent-archive-2.2.0-italian

-0.00

-0.00

-0.08

-0.08

-0.02

-0.07

-0.06

0.09

0.08

0.02

0.07

0.06

speech-accent-archive-2.2.0-japanese

-0.02

-0.00

-0.03

-0.08

-0.05

-0.05

-0.16

0.03

0.08

0.05

0.05

0.16

speech-accent-archive-2.2.0-korean

0.00

-0.00

0.06

0.08

0.14

0.08

0.11

-0.07

-0.08

-0.14

-0.08

-0.10

speech-accent-archive-2.2.0-macedonian

-0.00

-0.00

-0.13

-0.26

-0.21

-0.15

-0.18

0.13

0.26

0.21

0.15

0.18

speech-accent-archive-2.2.0-mandarin

-0.02

-0.00

-0.19

-0.23

-0.27

-0.26

-0.23

0.20

0.23

0.27

0.26

0.23

speech-accent-archive-2.2.0-nepali

-0.02

-0.00

0.16

0.11

0.17

0.07

0.11

-0.14

-0.11

-0.17

-0.07

-0.11

speech-accent-archive-2.2.0-polish

-0.02

-0.00

-0.03

-0.01

-0.05

0.02

-0.07

0.05

0.01

0.05

-0.02

0.07

speech-accent-archive-2.2.0-portuguese

0.01

-0.00

0.01

0.01

0.08

0.01

0.01

-0.01

-0.01

-0.08

-0.01

-0.00

speech-accent-archive-2.2.0-romanian

-0.02

-0.00

0.15

0.20

0.18

0.09

0.11

-0.13

-0.20

-0.18

-0.09

-0.10

speech-accent-archive-2.2.0-russian

-0.02

-0.00

-0.08

-0.06

-0.27

-0.07

-0.08

0.09

0.06

0.27

0.07

0.08

speech-accent-archive-2.2.0-serbian

0.12

0.01

0.04

0.10

0.10

0.07

0.03

-0.16

-0.10

-0.10

-0.08

-0.04

speech-accent-archive-2.2.0-spanish

0.01

-0.00

0.08

0.10

0.12

0.11

0.16

-0.08

-0.10

-0.12

-0.10

-0.15

speech-accent-archive-2.2.0-swedish

0.03

-0.00

0.08

0.09

-0.02

0.14

0.08

-0.11

-0.09

0.02

-0.14

-0.08

speech-accent-archive-2.2.0-tagalog

0.03

-0.00

-0.27

-0.14

-0.14

-0.13

-0.28

0.25

0.14

0.14

0.13

0.28

speech-accent-archive-2.2.0-thai

-0.02

-0.00

0.05

0.06

0.09

0.08

0.03

-0.04

-0.06

-0.09

-0.08

-0.03

speech-accent-archive-2.2.0-turkish

-0.02

-0.00

-0.10

-0.13

-0.22

-0.16

-0.12

0.12

0.13

0.22

0.16

0.12

speech-accent-archive-2.2.0-ukrainian

-0.02

-0.00

-0.00

-0.01

0.09

0.05

0.09

0.02

0.01

-0.09

-0.05

-0.09

speech-accent-archive-2.2.0-urdu

-0.02

-0.00

-0.09

-0.05

-0.08

-0.10

-0.09

0.11

0.05

0.08

0.10

0.09

speech-accent-archive-2.2.0-vietnamese

-0.01

0.01

0.12

0.20

0.23

0.10

0.15

-0.12

-0.20

-0.23

-0.10

-0.15

mean

-0.00

0.00

-0.00

-0.00

-0.01

-0.00

-0.01

0.00

0.00

0.01

0.01

0.01

Visualization

w2v2-b

w2v2-L

w2v2-L-robust

w2v2-L-xls-r

w2v2-L-vox

../../../_images/visualization_speech-accent-archive-2.2.0-amharic12.png
../../../_images/visualization_speech-accent-archive-2.2.0-amharic15.png
../../../_images/visualization_speech-accent-archive-2.2.0-amharic16.png
../../../_images/visualization_speech-accent-archive-2.2.0-amharic17.png
../../../_images/visualization_speech-accent-archive-2.2.0-amharic18.png
../../../_images/visualization_speech-accent-archive-2.2.0-arabic12.png
../../../_images/visualization_speech-accent-archive-2.2.0-arabic15.png
../../../_images/visualization_speech-accent-archive-2.2.0-arabic16.png
../../../_images/visualization_speech-accent-archive-2.2.0-arabic17.png
../../../_images/visualization_speech-accent-archive-2.2.0-arabic18.png
../../../_images/visualization_speech-accent-archive-2.2.0-bengali12.png
../../../_images/visualization_speech-accent-archive-2.2.0-bengali15.png
../../../_images/visualization_speech-accent-archive-2.2.0-bengali16.png
../../../_images/visualization_speech-accent-archive-2.2.0-bengali17.png
../../../_images/visualization_speech-accent-archive-2.2.0-bengali18.png
../../../_images/visualization_speech-accent-archive-2.2.0-bulgarian12.png
../../../_images/visualization_speech-accent-archive-2.2.0-bulgarian15.png
../../../_images/visualization_speech-accent-archive-2.2.0-bulgarian16.png
../../../_images/visualization_speech-accent-archive-2.2.0-bulgarian17.png
../../../_images/visualization_speech-accent-archive-2.2.0-bulgarian18.png
../../../_images/visualization_speech-accent-archive-2.2.0-cantonese12.png
../../../_images/visualization_speech-accent-archive-2.2.0-cantonese15.png
../../../_images/visualization_speech-accent-archive-2.2.0-cantonese16.png
../../../_images/visualization_speech-accent-archive-2.2.0-cantonese17.png
../../../_images/visualization_speech-accent-archive-2.2.0-cantonese18.png
../../../_images/visualization_speech-accent-archive-2.2.0-dutch12.png
../../../_images/visualization_speech-accent-archive-2.2.0-dutch15.png
../../../_images/visualization_speech-accent-archive-2.2.0-dutch16.png
../../../_images/visualization_speech-accent-archive-2.2.0-dutch17.png
../../../_images/visualization_speech-accent-archive-2.2.0-dutch18.png
../../../_images/visualization_speech-accent-archive-2.2.0-english12.png
../../../_images/visualization_speech-accent-archive-2.2.0-english15.png
../../../_images/visualization_speech-accent-archive-2.2.0-english16.png
../../../_images/visualization_speech-accent-archive-2.2.0-english17.png
../../../_images/visualization_speech-accent-archive-2.2.0-english18.png
../../../_images/visualization_speech-accent-archive-2.2.0-farsi12.png
../../../_images/visualization_speech-accent-archive-2.2.0-farsi15.png
../../../_images/visualization_speech-accent-archive-2.2.0-farsi16.png
../../../_images/visualization_speech-accent-archive-2.2.0-farsi17.png
../../../_images/visualization_speech-accent-archive-2.2.0-farsi18.png
../../../_images/visualization_speech-accent-archive-2.2.0-french12.png
../../../_images/visualization_speech-accent-archive-2.2.0-french15.png
../../../_images/visualization_speech-accent-archive-2.2.0-french16.png
../../../_images/visualization_speech-accent-archive-2.2.0-french17.png
../../../_images/visualization_speech-accent-archive-2.2.0-french18.png
../../../_images/visualization_speech-accent-archive-2.2.0-german12.png
../../../_images/visualization_speech-accent-archive-2.2.0-german15.png
../../../_images/visualization_speech-accent-archive-2.2.0-german16.png
../../../_images/visualization_speech-accent-archive-2.2.0-german17.png
../../../_images/visualization_speech-accent-archive-2.2.0-german18.png
../../../_images/visualization_speech-accent-archive-2.2.0-greek12.png
../../../_images/visualization_speech-accent-archive-2.2.0-greek15.png
../../../_images/visualization_speech-accent-archive-2.2.0-greek16.png
../../../_images/visualization_speech-accent-archive-2.2.0-greek17.png
../../../_images/visualization_speech-accent-archive-2.2.0-greek18.png
../../../_images/visualization_speech-accent-archive-2.2.0-hindi12.png
../../../_images/visualization_speech-accent-archive-2.2.0-hindi15.png
../../../_images/visualization_speech-accent-archive-2.2.0-hindi16.png
../../../_images/visualization_speech-accent-archive-2.2.0-hindi17.png
../../../_images/visualization_speech-accent-archive-2.2.0-hindi18.png
../../../_images/visualization_speech-accent-archive-2.2.0-italian12.png
../../../_images/visualization_speech-accent-archive-2.2.0-italian15.png
../../../_images/visualization_speech-accent-archive-2.2.0-italian16.png
../../../_images/visualization_speech-accent-archive-2.2.0-italian17.png
../../../_images/visualization_speech-accent-archive-2.2.0-italian18.png
../../../_images/visualization_speech-accent-archive-2.2.0-japanese12.png
../../../_images/visualization_speech-accent-archive-2.2.0-japanese15.png
../../../_images/visualization_speech-accent-archive-2.2.0-japanese16.png
../../../_images/visualization_speech-accent-archive-2.2.0-japanese17.png
../../../_images/visualization_speech-accent-archive-2.2.0-japanese18.png
../../../_images/visualization_speech-accent-archive-2.2.0-korean12.png
../../../_images/visualization_speech-accent-archive-2.2.0-korean15.png
../../../_images/visualization_speech-accent-archive-2.2.0-korean16.png
../../../_images/visualization_speech-accent-archive-2.2.0-korean17.png
../../../_images/visualization_speech-accent-archive-2.2.0-korean18.png
../../../_images/visualization_speech-accent-archive-2.2.0-macedonian12.png
../../../_images/visualization_speech-accent-archive-2.2.0-macedonian15.png
../../../_images/visualization_speech-accent-archive-2.2.0-macedonian16.png
../../../_images/visualization_speech-accent-archive-2.2.0-macedonian17.png
../../../_images/visualization_speech-accent-archive-2.2.0-macedonian18.png
../../../_images/visualization_speech-accent-archive-2.2.0-mandarin12.png
../../../_images/visualization_speech-accent-archive-2.2.0-mandarin15.png
../../../_images/visualization_speech-accent-archive-2.2.0-mandarin16.png
../../../_images/visualization_speech-accent-archive-2.2.0-mandarin17.png
../../../_images/visualization_speech-accent-archive-2.2.0-mandarin18.png
../../../_images/visualization_speech-accent-archive-2.2.0-nepali12.png
../../../_images/visualization_speech-accent-archive-2.2.0-nepali15.png
../../../_images/visualization_speech-accent-archive-2.2.0-nepali16.png
../../../_images/visualization_speech-accent-archive-2.2.0-nepali17.png
../../../_images/visualization_speech-accent-archive-2.2.0-nepali18.png
../../../_images/visualization_speech-accent-archive-2.2.0-polish12.png
../../../_images/visualization_speech-accent-archive-2.2.0-polish15.png
../../../_images/visualization_speech-accent-archive-2.2.0-polish16.png
../../../_images/visualization_speech-accent-archive-2.2.0-polish17.png
../../../_images/visualization_speech-accent-archive-2.2.0-polish18.png
../../../_images/visualization_speech-accent-archive-2.2.0-portuguese12.png
../../../_images/visualization_speech-accent-archive-2.2.0-portuguese15.png
../../../_images/visualization_speech-accent-archive-2.2.0-portuguese16.png
../../../_images/visualization_speech-accent-archive-2.2.0-portuguese17.png
../../../_images/visualization_speech-accent-archive-2.2.0-portuguese18.png
../../../_images/visualization_speech-accent-archive-2.2.0-romanian12.png
../../../_images/visualization_speech-accent-archive-2.2.0-romanian15.png
../../../_images/visualization_speech-accent-archive-2.2.0-romanian16.png
../../../_images/visualization_speech-accent-archive-2.2.0-romanian17.png
../../../_images/visualization_speech-accent-archive-2.2.0-romanian18.png
../../../_images/visualization_speech-accent-archive-2.2.0-russian12.png
../../../_images/visualization_speech-accent-archive-2.2.0-russian15.png
../../../_images/visualization_speech-accent-archive-2.2.0-russian16.png
../../../_images/visualization_speech-accent-archive-2.2.0-russian17.png
../../../_images/visualization_speech-accent-archive-2.2.0-russian18.png
../../../_images/visualization_speech-accent-archive-2.2.0-serbian12.png
../../../_images/visualization_speech-accent-archive-2.2.0-serbian15.png
../../../_images/visualization_speech-accent-archive-2.2.0-serbian16.png
../../../_images/visualization_speech-accent-archive-2.2.0-serbian17.png
../../../_images/visualization_speech-accent-archive-2.2.0-serbian18.png
../../../_images/visualization_speech-accent-archive-2.2.0-spanish12.png
../../../_images/visualization_speech-accent-archive-2.2.0-spanish15.png
../../../_images/visualization_speech-accent-archive-2.2.0-spanish16.png
../../../_images/visualization_speech-accent-archive-2.2.0-spanish17.png
../../../_images/visualization_speech-accent-archive-2.2.0-spanish18.png
../../../_images/visualization_speech-accent-archive-2.2.0-swedish12.png
../../../_images/visualization_speech-accent-archive-2.2.0-swedish15.png
../../../_images/visualization_speech-accent-archive-2.2.0-swedish16.png
../../../_images/visualization_speech-accent-archive-2.2.0-swedish17.png
../../../_images/visualization_speech-accent-archive-2.2.0-swedish18.png
../../../_images/visualization_speech-accent-archive-2.2.0-tagalog12.png
../../../_images/visualization_speech-accent-archive-2.2.0-tagalog15.png
../../../_images/visualization_speech-accent-archive-2.2.0-tagalog16.png
../../../_images/visualization_speech-accent-archive-2.2.0-tagalog17.png
../../../_images/visualization_speech-accent-archive-2.2.0-tagalog18.png
../../../_images/visualization_speech-accent-archive-2.2.0-thai12.png
../../../_images/visualization_speech-accent-archive-2.2.0-thai15.png
../../../_images/visualization_speech-accent-archive-2.2.0-thai16.png
../../../_images/visualization_speech-accent-archive-2.2.0-thai17.png
../../../_images/visualization_speech-accent-archive-2.2.0-thai18.png
../../../_images/visualization_speech-accent-archive-2.2.0-turkish12.png
../../../_images/visualization_speech-accent-archive-2.2.0-turkish15.png
../../../_images/visualization_speech-accent-archive-2.2.0-turkish16.png
../../../_images/visualization_speech-accent-archive-2.2.0-turkish17.png
../../../_images/visualization_speech-accent-archive-2.2.0-turkish18.png
../../../_images/visualization_speech-accent-archive-2.2.0-ukrainian12.png
../../../_images/visualization_speech-accent-archive-2.2.0-ukrainian15.png
../../../_images/visualization_speech-accent-archive-2.2.0-ukrainian16.png
../../../_images/visualization_speech-accent-archive-2.2.0-ukrainian17.png
../../../_images/visualization_speech-accent-archive-2.2.0-ukrainian18.png
../../../_images/visualization_speech-accent-archive-2.2.0-urdu12.png
../../../_images/visualization_speech-accent-archive-2.2.0-urdu15.png
../../../_images/visualization_speech-accent-archive-2.2.0-urdu16.png
../../../_images/visualization_speech-accent-archive-2.2.0-urdu17.png
../../../_images/visualization_speech-accent-archive-2.2.0-urdu18.png
../../../_images/visualization_speech-accent-archive-2.2.0-vietnamese12.png
../../../_images/visualization_speech-accent-archive-2.2.0-vietnamese15.png
../../../_images/visualization_speech-accent-archive-2.2.0-vietnamese16.png
../../../_images/visualization_speech-accent-archive-2.2.0-vietnamese17.png
../../../_images/visualization_speech-accent-archive-2.2.0-vietnamese18.png