UPTI 2.0: A Multi Font Synthetic Urdu Nastaleeq Text Line Dataset with Expanded Ligature Coverage

Dataset Overview UPTI 2.0 Ground Truth Corpus contains 120,000 Urdu text line transcriptions prepared for training and evaluating optical character recognition systems for Urdu Nastaleeq script. This deposit contains the textual ground truth component associated with the UPTI 2.0 dataset. It provides 100,000 training text files and 20,000 testing text files. Each file contains one Urdu text line stored as Unicode text. The dataset was introduced in the paper titled Impact of Ligature Coverage on Training Practical Urdu OCR Systems. The purpose of UPTI 2.0 was to improve the linguistic and visual coverage available for practical Urdu OCR research. The original UPTI dataset contained approximately 6,000 unique ligatures. UPTI 2.0 expanded this coverage to more than 18,000 unique ligatures, representing more than 70 percent of the ligatures expected to occur in practical Urdu text. Scope of This Deposit The uploaded archive contains ground truth text only. It does not contain the rendered text line images described in the associated paper. It also does not contain the four font variants or the four degradation variants. The archive has the following structure: groundtruth/train/ This directory contains 100,000 text files numbered from 1.txt to 100000.txt. groundtruth/test/ This directory contains 20,000 text files numbered from 1.txt to 20000.txt. Every file contains one Urdu text line. The files are encoded as UTF-8 text and end with a standard line break. The archive does not include a validation directory. It also does not include a README, licence document, image manifest or font configuration file. Relationship to the Complete UPTI 2.0 Dataset The full UPTI 2.0 dataset described in the paper was designed around 120,000 collected text lines. The authors report: Training text lines: 100,000 Testing text lines: 20,000 Supported fonts: 4 Degradation levels per font: 4 Total variants per text line: 16 Expected total text line images: 1,920,000 The complete dataset was rendered in the following publicly available Nastaleeq fonts: Alvi Nastaleeq Jameel Noori Nastaleeq Nafees Nastaleeq Pak Nastaleeq Each font version was produced at four degradation levels: No degradation Low degradation Medium degradation High degradation The paper states that every text line was available in 16 image variants, created from four fonts and four degradation levels. These image variants are not present in the uploaded ZIP. The current deposit should therefore be described as a ground truth corpus rather than the complete rendered UPTI 2.0 image dataset. Purpose of the Dataset UPTI 2.0 was created to improve the development of practical OCR systems for Urdu Nastaleeq text. Urdu Nastaleeq is a highly cursive and context-sensitive script. The visual shape of a character can change according to its position within a word. Characters combine to form ligatures, and a single Urdu word may contain several ligatures. An OCR system trained on a limited number of ligatures may perform well on a controlled benchmark but fail when it encounters complex or less frequent ligatures in scanned books, newspapers, magazines or other real documents. UPTI 2.0 addresses this problem by increasing the number of ligatures represented in the training corpus. The expanded coverage allows OCR systems to learn a broader representation of the Urdu language and reduces the number of unfamiliar ligatures encountered during evaluation. Text Collection The source corpus was formed by collecting Urdu text samples from books, news articles and web sources. The paper does not identify the exact books, publishers, websites or news organisations used. It also does not provide the dates on which the source text was collected. A total of 120,000 text lines were gathered. These lines covered more than 18,000 unique Urdu ligatures. The collected text was then passed to the Pango text layout and rendering system to generate synthetic Nastaleeq text line images. Ground Truth Format Each ground truth file contains a single Urdu text line. The file name acts as the sample identifier. For example: groundtruth/train/1.txt groundtruth/train/50000.txt groundtruth/test/1.txt groundtruth/test/20000.txt In the complete image dataset, these identifiers would normally be used to associate the text transcription with its corresponding rendered image variants. The ground truth is stored in Unicode and should be processed using UTF-8 encoding. Software used to read the files must support right to left Urdu text and Arabic script Unicode characters. Training and Testing Split The archive provides a predefined training and testing split. Training set: 100,000 text lines Testing set: 20,000 text lines The predefined split should be retained when reproducing experiments associated with the original paper. A separate validation split is not supplied in the uploaded archive. Researchers who need validation data should create it from the training partition and clearly document the method used. The original experiments described in the paper used subsets of the dataset for particular training configurations. For example, one experiment used 80,000 images for training, 10,000 images for validation and 1,000 FZKR lines for external testing. Text Repetition and Split Quality Inspection of the uploaded ground truth files found repeated text lines within both partitions. The training partition contains 100,000 files but approximately 87,574 distinct complete text strings. The testing partition contains 20,000 files but approximately 12,881 distinct complete text strings. There are also 44 exact text strings that appear in both the training and testing partitions. Repeated lines may be intentional because the corpus was assembled from large text sources and may include common phrases, short expressions or repeated source material. However, this should be documented because exact repetition can affect evaluation if text content is used for language modelling or memorisation analysis. Researchers should avoid claiming that every text file contains a unique sentence unless the corpus has been deduplicated and checked again. Font Diversity The complete UPTI 2.0 design includes four Nastaleeq fonts. The purpose of using several fonts is to reduce dependence on one particular rendering style. Urdu OCR systems trained on one font may not generalise well to other fonts because Nastaleeq typefaces differ in character proportions, ligature shapes, slant, spacing and vertical arrangement. The paper reports that the Jameel Noori Nastaleeq model performed best among the individual font models. It also reports that the multi font model produced the strongest overall result on the FZKR evaluation dataset. The font files themselves are not included in the uploaded archive. Synthetic Image Generation The paper states that the collected text lines were rendered using Pango. Synthetic rendering makes it possible to produce a large number of labelled images without manually transcribing every image. The text used to render each image already serves as its ground truth. This method also makes it possible to generate the same text in several fonts and under controlled degradation settings. The rendering parameters, exact font versions, image dimensions and Pango configuration are not included in the uploaded archive. These details would be needed to reproduce the original 1,920,000 image variants exactly. Degradation Levels The complete dataset includes four degradation levels for every font. The paper describes these levels as: No degradation A clean rendered version of the text line. Low degradation A lightly modified version representing limited document wear or imaging noise. Medium degradation A more visibly altered version representing moderate degradation. High degradation A strongly altered version representing greater wear, fading, smudging, elasticity and related variation. The paper refers to degradation effects such as elasticity, smudge, fading and other forms of document variation. The uploaded ground truth archive does not contain labels identifying fonts or degradation levels because it does not contain the corresponding images. Recommended Uses The corpus can support: Urdu language corpus preparation Generation of synthetic Nastaleeq text images Training data preparation for sequence recognition models Evaluation of ligature coverage Development of Urdu OCR systems Language model training for OCR correction Testing of Unicode normalisation procedures Analysis of right to left text processing Creation of font diverse OCR benchmarks Study of text repetition and corpus coverage Comparison of single font and multi font recognition Research on printed Urdu digitisation Development of LSTM and CTC recognition systems Training of Transformer based text recognition models Recommended Evaluation Practice Researchers should clearly report whether they used: The uploaded ground truth text only A newly rendered image version The original four font image collection One degradation level Several degradation levels A single font model A multi font model Any newly generated images should be accompanied by information about the font name, font version, font size, rendering engine, image dimensions, colour mode, resolution and degradation settings. The predefined testing text should not be used to optimise model parameters. Any deduplication, punctuation removal, Unicode normalisation or spelling correction should be described because these operations can change the difficulty of the recognition task. Limitations This deposit contains text files only. It cannot reproduce the visual OCR experiments reported in the paper without corresponding image generation. The source books, news articles and websites are not identified. The exact font versions are not supplied. The Pango rendering configuration is not supplied. The degradation algorithm and its numerical parameters are not fully documented in the archive. The deposit has no separate validation partition. Repeated lines occur within the training and testing partitions. A small number of exact text strings occur in both partitions. The corpus contains some Latin characters, digits, punctuation marks and special Unicode characters in addition to standard Urdu text. Applications should not assume that every character belongs to the core Urdu alphabet. Ethical and Rights Considerations The dataset contains printed text rather than personal images, handwriting samples or biometric records. However, the text was collected from books, news articles and web sources. Copyright may continue to apply to some source text even though the text has been divided into individual lines. Associated Publication Muhammad Ferjad Naeem, Noor ul Sehr Zia, Aqsa Ahmed Awan, Faisal Shafait and Adnan Ul-Hasan. Impact of Ligature Coverage on Training Practical Urdu OCR Systems. 2017 14th IAPR International Conference on Document Analysis and Recognition, pages 131 to 136. DOI: 10.1109/ICDAR.2017.30