LSA-X: COCO Keypoints and Subtitles for Argentine Sign Language

COCO-Format Pose Landmarks and Aligned Subtitles for Argentine Sign Language (LSA) Video Recognition

Dataset Description

This dataset contains body pose landmarks extracted from publicly available Argentine Sign Language (LSA) videos, stored in COCO keypoint format. Landmarks cover the full upper body — including face, hands, and body pose — and were extracted using an automated pipeline applied frame by frame to each video.

The dataset comprises 1,553 videos totaling approximately 507.75 hours of footage, divided into two subsets:

Labeled (513 videos / 151.79h): Videos with associated Spanish subtitles, either downloaded directly from the platform, sourced from YouTube's auto-generated captions, or generated from audio using automatic speech recognition (ASR). Each sample includes the video ID, the temporally aligned subtitle text, and the corresponding landmark annotations per frame.

Unlabeled (1,040 videos / 355.96h): Videos that likely contain sign language content but lack audio or any available subtitle track, and therefore carry no associated text labels. Frame-level landmark annotations are provided without transcription.

This resource is intended to support research in sign language recognition, continuous signing translation, and multimodal learning for low-resource languages.

Dataset Format

The output file structure — including directory layout, file naming conventions, landmark schema, keypoint ordering, and annotation fields — is documented in the processing pipeline repository:

Sign-pipeline: Processing pipeline implementation and dataset format specification.

LSA-X: Dataset builder — tools and configuration used to produce this dataset.

Data Sources

All videos were collected from publicly accessible sources. The table below summarizes each source, its content type, and the subtitle acquisition method.

Source
URL / Origin
Content
Subtitle Method

Canales Asociación Civil
YouTube
Educational organization focused on quality education for deaf children in Argentina. Produces LSA content including the Videolibros project.
Generated from audio (ASR)

Videolibros LSA (public)
YouTube
A free platform featuring children's books and stories read in LSA by deaf signers, with Spanish voiceover. A project of Canales Asociación Civil.
Generated from audio (ASR)

Videolibros LSA (private)
videolibros.org (scraped)
Same content as the public Videolibros channel. These videos were publicly accessible on videolibros.org but unlisted on YouTube, and were collected via web scraping.
Generated from audio (ASR)

CNSordos
YouTube
Argentina's first news channel produced by deaf presenters,
Human-authored subtitles (downloaded)

Locufre
YouTube
A weekly streaming program.
YouTube auto-generated captions

Dataset Statistics

Source
Labeled
Unlabeled
Total

Canales Asociación Civil
231 videos (14.43h)
477 videos (24.35h)
708 videos (38.78h)

Videolibros LSA (public)
41 videos (0.87h)
63 videos (1.59h)
104 videos (2.46h)

Videolibros LSA (private)
63 videos (8.67h)
63 videos (8.67h)
126 videos (17.33h)

CNSordos
67 videos (28.50h)
109 videos (37.72h)
176 videos (66.23h)

Locufre
111 videos (99.33h)
328 videos (283.63h)
439 videos (382.95h)

TOTAL
513 videos (151.79h)
1,040 videos (355.96h)
1,553 videos (507.75h)

Usage Notes

Unlabeled subset: The unlabeled portion is not intended to be discarded or treated as auxiliary data. It is provided as a resource for semi-supervised learning approaches, where large amounts of unannotated signing footage can complement the labeled subset to improve model generalization — particularly given the limited availability of annotated LSA data.

Locufre source — signer attribution: Although Locufre videos feature clear signing, consistent video quality, and reliable Spanish subtitles, most episodes present two signers on screen simultaneously with a single shared voiceover track. Determining which signer produced each utterance — and therefore which portion of the subtitle corresponds to which individual — is non-trivial. Researchers should account for this when using this subset for tasks requiring speaker-level annotation, signer segmentation, or isolated signing sequence extraction.

Attribution

This dataset was conceptually based on LSA-T, a pioneering Argentine Sign Language translation dataset.