Bangla Lip-to-Text Sentence Level Dataset

The dataset is created as follows:

Step 1: Original Bangla videos were collected from YouTube Bangladesh National Television Debates and Talk Shows.

Step 2: Original videos are cut into 3-second duration mini videos.

Step 3: Face area from each frame of these mini videos is segmented using MediaPipe FaceMesh.

Step 4: Only those videos were chosen in which all of the frames contain exactly one face.

Step 5: Audios are cut and embedded in the chosen videos (folder: Face_Only_Videos_with_Audio).

Step 6: Bangla audio-to-text transcriptions of the chosen videos were generated using the Wav2Vec2 foundation model pre-trained on the Mozilla 300m Bengali Commonvoice dataset (folder: First_Transcriptions).

Step 7: Videos with no transcription texts were filtered out.

Step 8: A simplified transliteration for each of the translations was created (folder: Transliteration_Informal).