CTS Speaker Recognition Challenge


Contact Us

Please send questions to: sre_poc@nist.gov

For the CTS Challenge discussion please visit our Google Group. https://groups.google.com/a/list.nist.gov/forum/#!forum/cts-challenge

Summary

Following the success of the 2019 Conversational Telephone Speech (CTS) Speaker Recognition Challenge, which received 1347 submissions from 67 academic and industrial organizations, NIST is organizing a 2020 CTS Challenge, the next iteration of an ongoing series of speaker recognition evaluations conducted by NIST since 1996.

The basic task in the CTS Challenge is speaker detection, i.e., determining whether a specified target speaker is speaking during a given segment of speech. Similar to the 2019 CTS Challenge, the 2020 evaluation will feature a leaderboard-style challenge offering an open/unconstrained training condition, but using CTS recordings extracted from multiple data sources containing multilingual speech. In addition, unlike the 2019 CTS Challenge, no development set will be initially released.

For more information about the challenge please visit the announcement page or send questions to sre_poc@nist.gov.
News
11Jul'20
CTS Challenge is announced
7Aug'20
Evaluation plan is now available
August 7, 2020: Evaluation Plan Published

August 17, 2020 : CTS Challenge Registration Begins

August 17, 2020 : Data Available

Aug 2020-Jun 2021 (tentative): System Output Due to NIST

December 2021: Workshop

Test set results will be released periodically


Disclaimer:

Participants are allowed to publish the leaderboard results unaltered, but they must not make advertising claims about their standing/ranking in the evaluation, or winning the evaluation, or claim NIST or the U.S. Government endorsement of their system(s) or commercial product(s). See the evaluation plan for more details regarding the participation rules in the NIST CTS Challenge.

NIST CTS Challenge

Updated: 2021-12-01T12:35:39-05:00
RANK TEAM SET TIMESTAMP EER [%] MIN_C ACT_C
1 THUEE Progress 20210921-005400 2.23 0.063 0.076
2 STC Progress 20210826-020705 2.48 0.074 0.079
3 TEAM-MGQG-50 Progress 20210930-003746 2.01 0.078 0.085
4 I4U Progress 20210528-015904 2.53 0.077 0.094
5 TEAM-CDPE-28 Progress 20210616-024452 2.55 0.087 0.101
5 JHU-MIT Progress 20210413-171216 2.33 0.092 0.101
6 AAP Progress 20210621-103112 2.37 0.102 0.106
7 Veridas Progress 20210729-055711 2.60 0.105 0.110
8 ROXANNE Progress 20211021-131243 2.42 0.090 0.118
9 TEAM-QSYF-27 Progress 20210625-040137 2.58 0.117 0.122
10 BGU Progress 20210630-030728 2.85 0.124 0.132
11 ABC Progress 20210916-101147 2.35 0.095 0.135
12 LIA Progress 20210405-115008 2.83 0.129 0.136
13 XMUSPEECH Progress 20210702-015042 3.11 0.154 0.160
14 I2R Progress 20210203-223806 2.70 0.125 0.174
15 TEAM-ZBSM-52 Progress 20210729-075707 3.41 0.171 0.183
16 BiometricVox Progress 20201031-061334 3.40 0.140 0.207
17 Elektronika Progress 20210605-105329 4.51 0.203 0.218
18 NSYSU_CHT Progress 20210702-035631 3.52 0.177 0.222
19 TEAM-QTUY-05 Progress 20210218-041325 5.43 0.237 0.251
20 TEAM-IZYX-36 Progress 20211026-013456 4.35 0.190 0.294
21 dBLab Progress 20210210-044741 5.95 0.275 0.308
22 TEAM-FKFK-39 Progress 20210216-103739 6.99 0.280 0.399
23 TJU- Progress 20210601-023932 4.01 0.247 0.512
24 TEAM-FYYA-59 Progress 20211004-133818 10.57 0.464 0.632
25 TIT_SER Progress 20211025-012155 16.61 0.568 0.820
26 GRD Progress 20201021-022551 16.82 0.677 0.930
27 HIT Progress 20211021-103833 11.78 0.599 1.000
27 katon Progress 20210812-032522 49.88 1.000 1.000
27 TEAM-MQKX-64 Progress 20210902-175153 50.64 1.000 1.000
27 TEAM-QLSB-38 Progress 20210506-204611 63.86 1.000 1.000
28 TEAM-BPZI-72 Progress 20211027-103100 27.40 1.000 1.100
29 LEAP Progress 20210820-082901 4.51 0.244 1.137


Team Set Timestamp EER [%] min_C act_C Report
THUEE Test 20210921-005400 2.53 0.061 0.068 Download
I4U Test 20210528-015904 2.91 0.066 0.07 Download
STC Test 20210826-020705 2.81 0.067 0.081 Download
TEAM-CDPE-28 Test 20210616-024452 3.2 0.081 0.087 Download
JHU-MIT Test 20210413-171216 3.19 0.083 0.087 Download
Veridas Test 20210729-055711 3.09 0.09 0.092 Download
ROXANNE Test 20211021-131243 2.84 0.09 0.094 Download
TEAM-MGQG-50 Test 20210930-003746 2.33 0.089 0.095 Download
AAP Test 20210621-103112 2.92 0.1 0.101 Download
TEAM-QSYF-27 Test 20210625-040137 3.31 0.114 0.116 Download
BGU Test 20210630-030728 3.45 0.127 0.134 Download
LIA Test 20210405-115008 3.23 0.108 0.135 Download
ABC Test 20210916-101147 3.23 0.119 0.14 Download
XMUSPEECH Test 20210702-015042 3.78 0.142 0.146 Download
I2R Test 20210203-223806 3.29 0.131 0.161 Download
TEAM-ZBSM-52 Test 20210729-075707 3.88 0.154 0.164 Download
BiometricVox Test 20201031-061334 3.95 0.144 0.203 Download
Elektronika Test 20210605-105329 5.1 0.213 0.22 Download
NSYSU_CHT Test 20210702-035631 4.07 0.177 0.227 Download
TEAM-QTUY-05 Test 20210218-041325 5.92 0.24 0.25 Download
TEAM-IZYX-36 Test 20211026-013456 3.93 0.181 0.251 Download
dBLab Test 20210210-044741 6.9 0.293 0.333 Download
TEAM-FKFK-39 Test 20210216-103739 5.94 0.244 0.342 Download
TJU- Test 20210601-023932 5.04 0.218 0.43 Download
TEAM-FYYA-59 Test 20211004-133818 9.17 0.455 0.608 Download
TIT_SER Test 20211025-012155 15.55 0.539 0.863 Download
LEAP Test 20210820-082901 4.93 0.207 0.934 Download
GRD Test 20201021-022551 20.31 0.733 0.951 Download
HIT Test 20211021-103833 12.6 0.606 1 Download
katon Test 20210812-032522 49.32 1 1 Download
TEAM-QLSB-38 Test 20210506-204611 50 1 1 Download
TEAM-MQKX-64 Test 20210902-175153 48.95 1 1.033 Download
TEAM-BPZI-72 Test 20211027-103100 27.55 0.993 1.079 Download