british-library-book-images

# British Library Book Images 1,080,814 images cut out of 49,455 digitised books (65,227 volumes, ~25 million pages) published between c. 1510 and c. 1900, digitised by the [British Library](https://www.bl.uk/) in partnership with Microsoft and released by [British Library Labs](https://web.archive.org/web/20230107150636/https://www.bl.uk/projects/british-library-labs) on Flickr Commons as the "1 Million Images from Scanned Books" release. The books cover geography, philosophy, history, poetry and literature, in several languages. ## The four image types British Library Labs deposited these as four separate records, split by an algorithmic estimate of what each cut-out region is. Each is a config here: | config | images | earliest date | | ---- | ---- | ---- | | `embellishments` | 416,935 | 1510 | | `plates` | 385,231 | 1528 | | `medium` | 217,100 | 1567 | | `covers` | 61,548 | 1510 | The type labels are **algorithmic, not curatorial**. The deposit states the images were "algorithmically gathered" but does not state the method, so `plates` / `medium` / `embellishments` are best read as size-and-position heuristics over the page, not as an art-historical taxonomy. Expect the boundary between `medium` and `plates` in particular to be fuzzy. Filter on your own classifier if the distinction matters to your task. ```python from datasets import load_dataset # pick a config — the whole thing is ~621 GB, so stream unless you mean it ds = load_dataset("biglam/british-library-book-images", "covers", split="train", streaming=True) print(next(iter(ds))) ``` ## Images per decade | decade | embellishments | plates | medium | covers | total | | --- | --- | --- | --- | --- | --- | | 1510s | 6 | 0 | 0 | 1 | 7 | | 1520s | 3 | 2 | 0 | 1 | 6 | | 1540s | 11 | 0 | 0 | 0 | 11 | | 1550s | 4 | 2 | 0 | 1 | 7 | | 1560s | 6 | 0 | 2 | 1 | 9 | | 1570s | 0 | 0 | 0 | 1 | 1 | | 1580s | 1 | 0 | 1 | 1 | 3 | | 1590s | 185 | 4 | 0 | 4 | 193 | | 1600s | 47 | 4 | 6 | 11 | 68 | | 1610s | 42 | 2 | 0 | 12 | 56 | | 1620s | 49 | 0 | 1 | 9 | 59 | | 1630s | 783 | 13 | 2 | 100 | 898 | | 1640s | 282 | 6 | 1 | 45 | 334 | | 1650s | 170 | 2 | 0 | 39 | 211 | | 1660s | 220 | 10 | 13 | 70 | 313 | | 1670s | 257 | 12 | 2 | 118 | 389 | | 1680s | 228 | 27 | 19 | 95 | 369 | | 1690s | 33 | 86 | 1 | 144 | 264 | | 1700s | 32 | 39 | 3 | 83 | 157 | | 1710s | 776 | 50 | 74 | 95 | 995 | | 1720s | 1,663 | 57 | 106 | 63 | 1,889 | | 1730s | 1,644 | 34 | 88 | 133 | 1,899 | | 1740s | 968 | 66 | 63 | 94 | 1,191 | | 1750s | 768 | 20 | 28 | 135 | 951 | | 1760s | 1,044 | 133 | 55 | 187 | 1,419 | | 1770s | 1,138 | 75 | 51 | 281 | 1,545 | | 1780s | 1,278 | 127 | 65 | 321 | 1,791 | | 1790s | 648 | 89 | 69 | 426 | 1,232 | | 1800s | 1,924 | 4,682 | 1,282 | 1,390 | 9,278 | | 1810s | 1,843 | 5,138 | 1,671 | 2,183 | 10,835 | | 1820s | 3,585 | 5,183 | 1,701 | 2,343 | 12,812 | | 1830s | 6,129 | 11,836 | 3,982 | 2,396 | 24,343 | | 1840s | 17,819 | 19,664 | 10,194 | 3,672 | 51,349 | | 1850s | 28,868 | 31,493 | 15,459 | 5,692 | 81,512 | | 1860s | 40,035 | 31,234 | 17,347 | 7,280 | 95,896 | | 1870s | 70,752 | 55,665 | 31,857 | 8,607 | 166,881 | | 1880s | 102,320 | 93,515 | 49,209 | 10,853 | 255,897 | | 1890s | 128,075 | 123,454 | 82,637 | 14,136 | 348,302 | The 1890s alone hold a third of the corpus, and everything before 1800 together holds about 1.6% of it. Anything trained on the whole set is overwhelmingly a model of late-Victorian book illustration. Note the `plates` column before and after 1800: 86 images in the entire 1690s against 4,682 in the 1800s. Full-page plates are a 19th-century printing phenomenon here, not an even feature of the date range. ## Data fields - `image` — the JPEG, at source resolution - `date` — publication year as a **string**, not an integer. 5,291 rows (0.5%) carry `"Unknown"`, and 2,151 rows carry a date after 1900 — up to 1946 — which the deposit's own c. 1510–c. 1900 framing marks as catalogue errors rather than genuinely 20th-century books. Cast and filter before doing anything arithmetic with it. - `fname` — the original filename. The leading digits are the British Library system number (see below), and the rest encodes volume/page position plus the book title. The exact layout differs between image types, so parse defensively. - `image_type` — one of the four above, redundant with the config but kept so the configs can be concatenated. ## Joining images to their page text The leading digits of `fname` are the British Library system number, which is the `record_id` in [`biglam/blbooks-parquet`](https://huggingface.co/datasets/biglam/blbooks-parquet) — the OCR text from the same digitisation programme. In a 20,000-row sample, 8,497 of 8,846 distinct system numbers (96%) matched a record there, so most images can be traced back to the book they were cut from and its text. ```python # system number -> the book this image came from fname = "002543810_3_Dzieje Narodu Polskiego...jpg" record_id = fname.split("_")[0] # "002543810" ``` The 4% that don't match are books present in the image deposit but absent from the OCR release. The join is at **book** level, not page level — `fname` encodes a page position, but it is not guaranteed to align with the `pg` column in the OCR corpus, so treat page-level alignment as something to verify rather than assume. ## What was selected, and by whom The date range is not a sampling frame. The corpus is what the British Library and Microsoft chose to digitise from what the Library had acquired from what happened to survive — three selection layers stacked before any image reached this dataset — and digitisation programmes of this era were driven partly by preservation priorities and out-of-copyright status rather than by any attempt at representativeness. The 19th century dominates. Treat the collection as a record of what a large British institution digitised in the early 2010s, not as a sample of printed illustration. Colonial-era publishing is heavily represented, and images carry the depictions, captions and categories of their period. Nothing here has been reviewed for offensive content. ## Related datasets The OCR text from the same digitisation programme is [`biglam/blbooks-parquet`](https://huggingface.co/datasets/biglam/blbooks-parquet) (14,011,953 pages). ## Licensing The images were published on Flickr Commons under **no known copyright restrictions** — the British Library placed them in the public domain, and the underlying works are out of copyright. Tagged `cc0-1.0` here for machine-readability; the deposit's own statement is the Public Domain Mark. No licence attaches to the images themselves that would restrict reuse. Attribution to the British Library is expected practice rather than a legal condition. ## Credit Mirrored and repackaged by [Daniel van Strien](https://huggingface.co/davanstrien) (Machine Learning Librarian, Hugging Face); all source data created by the British Library, digitised in partnership with Microsoft and released by British Library Labs. The four image types were deposited as four separate records on the British Library Research Repository (bl.iro.bl.uk), which is intermittently unavailable — this dataset is the more reliable route to the images. Against the counts those deposits state, this dataset is a handful of images short: 6 fewer plates, 1 fewer medium, 13 fewer covers. The shortfall predates this repackaging and no cause has been established for it. Point of contact for the original deposit: [e-mail] Maintenance: **Limited Maintenance** — this is a mirror of a static 2014 deposit and is not expected to change. ## Citation ```bibtex @misc{bl_labs_book_images, title = {Digitised Books. c. 1510 - c. 1900. JPG}, author = {{British Library Labs}}, year = {2014}, publisher = {British Library}, howpublished = {\url{https://bl.iro.bl.uk/}} } ``` ## SigLIP2 embeddings (`siglip2_embeddings` config) Every image in this dataset, embedded with [google/siglip2-so400m-patch16-256](https://huggingface.co/google/siglip2-so400m-patch16-256) (1152-d float32, images resized to 256x256 before encoding — the model's own preprocessing shape). One split per image config; rows are sorted to match the source parquet exactly, so **row N of split `plates` is row N of config `plates`**. Each row also carries `source_filename` and `file_row_number`, the direct coordinates of its image in this repository. ```python from datasets import load_dataset emb = load_dataset("biglam/british-library-book-images", "siglip2_embeddings", split="plates") ``` Because SigLIP2 is a dual encoder, text queries embed into the same space — encode a phrase with the model's text tower and rank rows by cosine similarity for free-text search over the collection. Notes: embeddings were produced with vLLM serving SigLIP2 as a pooling model on Hugging Face Jobs; all 1,080,814 images are covered, one embedding each. 768 image pairs (1,536 filenames) are near-duplicates differing only in title spacing (same system number/volume/page/block) — these are distinct rows here, as in the source configs. ## Crop masks (`crop_masks` config) Model-predicted instance masks and bounding boxes for 1,019,266 images — every image in the `embellishments`, `plates` and `medium` configs (`covers` is excluded: the model cannot abstain, and covers are overwhelmingly not illustrations). The ABBYY-derived crops in this dataset are loose — a `medium` crop typically carries lines of body text above the art and a printed caption below — and these masks are the tightening layer: 3,022,916 instances, one row per image, joinable on `fname`. Each row carries `objects` (index-aligned lists: `bbox` as `[x, y, w, h]` in source-frame pixels, `score`, `area`, `rectangularity`) and `masks_rle` (a JSON string of COCO RLE dicts in the same order; the RLE frame is the full-resolution source image, `src_width` x `src_height`). **Filter by `score` before display use.** Predictions were kept down to a deliberately low threshold of 0.10 so that consumers can choose their own operating point, and 59.7% of instances score below 0.3 — at 0.10, dense pages can carry dozens of low-confidence instances. `score >= 0.3` is a sensible display default; the confident singletons that dominate the corpus (76% of images have exactly one instance) are unaffected. ```python import json from datasets import load_dataset from pycocotools import mask as maskutil masks = load_dataset("biglam/british-library-book-images", "crop_masks", split="plates") row = masks[0] rles = json.loads(row["masks_rle"]) keep = [i for i, s in enumerate(row["objects"]["score"]) if s >= 0.3] m = maskutil.decode(rles[keep[0]]) # HxW numpy array, source frame ``` To join masks against the image configs' metadata, DuckDB reads both sides straight off the Hub with column pruning — no image bytes are fetched unless you select the `image` column: ```python import duckdb duckdb.sql(""" SELECT i.fname, i.date, m.objects, m.masks_rle FROM 'hf://datasets/biglam/british-library-book-images/plates/*.parquet' i JOIN 'hf://datasets/biglam/british-library-book-images/crop-masks/plates-*.parquet' m USING (fname) """) ``` Provenance: predicted by [davanstrien/bl-crop-tighten-rfdetrseg-clip10](https://huggingface.co/davanstrien/bl-crop-tighten-rfdetrseg-clip10), an RF-DETR-Seg student distilled from [Falcon-Perception](https://huggingface.co/tiiuae/Falcon-Perception-0.6B) weak labels on 8,400 of these crops. On a 40-image random human check, 97.4% of the student's predictions were judged acceptable — matching its teacher, at roughly 12x the throughput. Every row is stamped with `model_id` and `model_sha`.