Latest Updates (as of 2/1/2024):

Description of the imageDISPLACE-2024-challengeDescription of the image Click here

Registrations are open for DISPLACE-2024-challenge Click here

Description of the imageWelcome to The DISPLACE Challenge Description of the imageWe are looking forward to seeing everyone!


The following papers related to the challenge have been accepted at Interspeech 2023:

About


In multilingual communities, the social conversations often involve code-mixed and code-switched speech. The code-mixing refers to the scenario where words or morphemes from one language (secondary) are used within a sentence of another language (primary). However, the switching of languages at the sentence or phrase level is known as code-switching, where the conversational language is itself shifted. In such cases, the extraction of various analytics for speech-based systems, such as speaker and language information or automatic speech recognition (ASR) to generate rich transcriptions, becomes highly challenging. The current speaker diarization systems are simply not equipped to deal with multilingual conversations, where the same talker speaks in multiple code-mixed languages.

Focusing on the Interspeech-2023 theme, i.e., Inclusive Spoken Language Science and Technology – Breaking Down Barriers, the DISPLACE challenge aims to address research issues related to speaker and language diarization in an inclusive manner. The goal of the challenge is to establish new benchmarks for speaker diarization (SD) in multilingual settings and language diarization (LD) in multi-speaker settings, using the same underlying dataset. The previous works have addressed speaker and language diarization, but in isolation. A collective effort from worldwide researchers is required to address associated research issues. We look forward to your participation in reaching a new milestone in the speaker and language diarization areas. We also encourage general submissions in the field of speaker and/or language diarization under DISPLACE challenge/special session.

We also encourage general submissions related to speaker and language diarization in DISPLACE challenge / special session in Interspeech-2023.

Updates
  • [31-Dec-22] Challenge Registration is open.
  • [15-Jan-23] Submit the signed Terms & Conditions document to get access to the dataset and baseline systems.
  • [20-Jan-23] Development set is released.
    • [24-Jan-23] Track-1 (Speaker diarization for multilingual scenarios) baseline is released.
  • [25-Jan-23] Track-1 baseline is updated.
  • [03-Feb-23] Track-2 (Language Diarization in multi-speaker settings) baseline is released.
  • [08-Feb-23] Evaluation plan is released.
  • [08-Feb-23] Leaderborad is active.
  • [14-Feb-23] Eval Phase-1 Data is released.
  • [24-Feb-23] Phase-1 evaluation closes on 4 March.
  • [1-Apr-23] Phase-2 evaluation is started.
  • [10-May-23] Deadline for Phase-2 evaluation is extended.

Timeline

Registrations Opens:
31 Dec 2022  
Registrations Closes:
15 Apr 2023
Data Release (Dev):
20 Jan 2023
Baseline System Release:
24 Jan 2023
Leaderboard Active:
8 Feb 2023
Phase 1 Evaluation Data Release:
11 Feb 2023
Phase 1 Evaluation Closes:
4 Mar 2023
Phase 2 Evaluation Opens:
1 Apr 2023
Phase 2 Evaluation Closes:
22 May 2023
System Report submission:
4 Mar 2023
INTERSPEECH Paper Submission Deadline:
1 Mar 2023
INTERSPEECH Paper Update Deadline:
8 Mar 2023

Tracks

This challenge organises two tracks and you can participate in one or both of them. The Track-1 is dedicated to speaker diarization and Track-2 focuses on language diarization. You are encouraged to submit your experimental findings and observations to the DISPLACE Challenge at Interspeech 2023 for peer-review and subsequent consideration for presentation (and publication) in the conference. For this we require you to participate in one or both the tracks.


Track-1:
Speaker Diarization in multilingual scenarios.
  • a. The goal is to perform speaker diarization (who spoke when) in multilingual conversational audio data, where the same speaker speaks in multiple code-mixed and/or code switched languages. 
  • b. You will be provided with a dev set (far-field recordings), and a baseline system to enable design of your own models.
  • c. Subsequently, a blind evaluation set (far-field recordings), will be provided to all participants. You will need to submit your model predictions (in rttm format) on the blind set to a leaderboard interface (setup in Codalab). The leaderboard will be featuring performance of other teams on the same dataset.
  • d. The performance metric for evaluation will be Diarization Error Rate (DER). 
  • e. All participants will be required to submit a system description report (2-4 pages) to the organizers (Submission Deadline: click here). All participants are also encouraged to submit their findings to the DISPLACE challenge, Interspeech 2023 for peer-review (Submission page) .
  • f. The participating teams are encouraged to use any open datasets for training and developing the diarization systems. 
Track-2:
Language Diarization in multi-speaker settings.
  • a. The goal is to perform language diarization in multi- speaker conversational audio data, recorded in far-field settings. 
  • b. You will be provided with a dev audio dataset, and a baseline system to enable design of your own models. 
  • c. Subsequently, a blind evaluation dataset will be provided to all participants. You will need to submit your model predictions (in rttm format) on the blind set to a leaderboard interface (setup in Codalab). The leaderboard will be featuring performance of other teams on the same dataset. 
  • d. The performance metric for evaluation will be Diarization Error Rate (DER). 
  • e. All participants will be required to submit a system description report (2-4 pages) to the organizers (Submission Deadline: click here). All participants are also encouraged to submit their findings to the DISPLACE challenge, Interspeech 2023 for peer-review (Submission page) .
  • f. The participating teams are encouraged to use any open datasets for training and developing the diarization systems. 
For both tracks, the overall evaluation of submissions will be done in terms of Diarization Error Rate (DER) with overlap and without collar. A baseline system for both the tracks will be provided to the registered teams. The evaluation results of submissions will be displayed on a leaderboard for continuous monitoring of the progress.

Registration  

Thank you for your interest! Below are the two quick steps involved in registering your participation and get started in the challenge.
Step-1:
One representative of the participating team fills the form at: click here
Step-2:
Subsequently, you need to send a signed Terms & Conditions document to us at displace2023@gmail.com.
After a quick verification from our side, we will confirm your registration and send you the access details to the dataset. That's it!.

Resources  

Evaluation Plan:
Evaluation plan for this challenge is available Click here.
DISPLACE Leaderboard:
Click here.
Baseline Systems:
Click here.

Organizers

Sriram Ganapathy
Associate Professor, Indian Institute of Science, Bangalore, India
Deepu Vijayasenan
Associate Professor,National Institute of Technology Karnataka Surathkal,India
Shikha Baghel
Post Doctoral Researcher, Indian Institute of Science, Bangalore, India
Shreyas Ramoji
PHD Scholar, Institute of Science, Bangalore, India
Pratik Roy Chowdhuri
Research Scholar,National Institute of Technology Karnataka Surathkal,India
Somil Jain
Junior Research Fellow, National Institute of Technology Karnataka Surathkal,India

Frequently Asked Questions

Q. Which programming languages can I use?

A. You are free to use any programming language you like. For system evaluation we will require you to submit the output decisions as a Rich Transcription Time Marked (RTTM) file. 
Q. How do I get the DISPLACE audio dataset?

A. It is simple - by registering for the challenge. Please see the registration section in this webpage (above).
Q. Can I re-distribute the data?

A. No, you can not re-distribute the data even if you have participated in the challenge. However, you can use it for research purposes with proper citations
Q. In which format, do I need to submit output?

A. Output should be in text file with Rich Transcription Time Marked (RTTM) extension.
Q. How do I submit my findings obtained by participating in this challenge to Interspeech 2023?

A. That's great! You can follow the Interspeech 2023 paper submission portal here. Remember to select "DISPLACE Challenge” while uploading your paper there. 

Contact Us

You have more questions? Feel free to contact us at:

displace2023@gmail.com.