Drawings Gemma-Enriched Dataset. Fotothek - Bibliotheca Hertziana

<h1 class="title">Zeichnungen Dataset - AI-Enhanced Art Historical
Descriptions</h1>
</header>
<nav id="TOC" role="doc-toc">
<ul>
<li><a
href="#zeichnungen-dataset---ai-enhanced-art-historical-descriptions-with-iconography"
id="toc-zeichnungen-dataset---ai-enhanced-art-historical-descriptions-with-iconography">Zeichnungen
Dataset - AI-Enhanced Art Historical Descriptions with Iconography</a>
<ul>
<li><a href="#dataset-overview" id="toc-dataset-overview">Dataset
Overview</a></li>
<li><a href="#processing-pipeline"
id="toc-processing-pipeline">Processing Pipeline</a></li>
<li><a href="#ai-prompts-used" id="toc-ai-prompts-used">AI Prompts
Used</a></li>
<li><a href="#kisski-cluster-resources"
id="toc-kisski-cluster-resources">KISSKI Cluster Resources</a></li>
<li><a href="#output-structure" id="toc-output-structure">Output
Structure</a></li>
<li><a href="#kisski-documentation" id="toc-kisski-documentation">KISSKI
Documentation</a></li>
<li><a href="#iconclass-resources"
id="toc-iconclass-resources">ICONCLASS Resources</a></li>
<li><a href="#data-usage-citation" id="toc-data-usage-citation">Data
Usage & Citation</a></li>
<li><a href="#quality-notes" id="toc-quality-notes">Quality
Notes</a></li>
</ul></li>
</ul>
</nav>
<h1
id="zeichnungen-dataset---ai-enhanced-art-historical-descriptions-with-iconography">Zeichnungen
Dataset - AI-Enhanced Art Historical Descriptions with Iconography</h1>
<p>This dataset contains 224x224 images and relative metadata extracted
from the MIDAS XML of the Catalogue of the Photographic Collection of
the Bibliotheca Hertziana enriched with AI-generated prose texts and
iconographic analysis. The dataset is limited to photographs of objects
classified as drawing (Zeichnungen), and has been processed using <a
href="https://huggingface.co/google/gemma-2-9b-it">Google Gemma 2 9B
Instruct</a> large language model on the <a
href="https://docs.hpc.gwdg.de/">KISSKI HPC cluster</a> of the GWDG.
Scripts to process the data on KISSKI have been elaborated with Claude
Code in Virtual Studio Code.</p>
<hr />
<h2 id="dataset-overview">Dataset Overview</h2>
<p><strong>Source Data:</strong></p>
<ul>
<li>Original dataset: <code>zeichnungen.tsv</code> (30,000 rows / 29,999
data rows)</li>
<li>Extracted from: MIDAS XML format (<code>combined.xml</code>)</li>
<li>Source institution: <a href="https://www.biblhertz.it/">Bibliotheca
Hertziana - Max Planck Institute for Art History</a></li>
<li>Image repository: <a href="https://fotothek.biblhertz.it/">Fotothek
der Bibliotheca Hertziana</a></li>
</ul>
<p><strong>Output:</strong></p>
<ul>
<li><strong>Enriched metadata:</strong> TSV files with AI-generated
German and English descriptions</li>
<li><strong>Iconographic analysis:</strong> Descriptions based on
ICONCLASS classification</li>
<li><ul>
<li><strong>224x224 images downloaded from IIIF Image Api of the
Photographic Collection</strong></li>
</ul></li>
</ul>
<hr />
<h2 id="processing-pipeline">Processing Pipeline</h2>
<h3 id="data-extraction">1. Data Extraction</h3>
<p>Source data was extracted with zeichnungen.xql from <a
href="https://edmond.mpg.de/dataset.xhtml?persistentId=doi:10.17617/3.8GPSDJ">MIDAS
XML format combined.xml</a> containing structured art historical
metadata including:</p>
<ul>
<li>Object titles and descriptions (<code>textobj</code>,
<code>textfoto</code>)</li>
<li>Artist information (<code>aob30</code>)</li>
<li>Location data (<code>aob26</code>, <code>aob28</code>)</li>
<li><strong>ICONCLASS codes</strong> (<code>a5500</code>) - Standardized
iconographic classification</li>
<li>Dating and provenance</li>
<li>Image references (<code>a8540</code>) The set was limited to 30000
entries.</li>
</ul>
<h3 id="iconclass-cache-preparation">2. ICONCLASS Cache Preparation</h3>
<p><strong>ICONCLASS System:</strong></p>
<ul>
<li><strong>Source:</strong> <a
href="https://iconclass.org/">ICONCLASS.org</a> - Multilingual
classification system for cultural content</li>
<li><strong>GitHub repository:</strong>
https://github.com/iconclass/data</li>
</ul>
<h4 id="images-download">Images Download</h4>
<p>224x224 images downloaded in advance from the IIIF Service based on
<code>gemalde.tsv</code>. The script processing for AI Text Enrichment
from the metadata checks that the image has been downloaded, so the
output data has a 100% certainty of having a matching image. 28.165
images downloaded from 29,999 rows. This is due to known missing digital
images. The dataset corresponds to published data and each row contains
the licence and accessibility of the single image, date of creation and
last update of the catalogue object.</p>
<h3 id="ai-text-generation">3. AI Text Generation</h3>
<p><strong>Model Used:</strong></p>
<ul>
<li><strong>Name:</strong> <a
href="https://huggingface.co/google/gemma-2-9b-it">Google Gemma 2 9B
Instruct</a></li>
<li><strong>Parameters:</strong> 9 billion</li>
<li><strong>Quantization:</strong> FP16 (no quantization)</li>
<li><strong>Context window:</strong> 8,192 tokens</li>
<li><strong>License:</strong> Gemma Terms of Use</li>
</ul>
<p><strong>Processing Workflow:</strong></p>
<ol type="1">
<li><strong>Input cleaning:</strong> Removal of numeric codes,
normalization of Unicode characters, increased CSV field size limit (10
MB)</li>
<li><strong>Paragraph generation:</strong> German text from structured
metadata</li>
<li><strong>ICONCLASS lookup:</strong> Offline cache-based iconographic
description retrieval</li>
<li><strong>Iconographic synthesis:</strong> AI-generated description
from ICONCLASS codes</li>
<li><strong>Translation:</strong> German → English</li>
<li><strong>Categories processed:</strong>
<ul>
<li><code>paragraph foto DE/EN</code> - Photograph description</li>
<li><code>paragraph obj DE/EN</code> - Object/artwork description</li>
<li><code>paragraph verwalter DE/EN</code> - Collection/custodian
information</li>
<li><code>paragraph standort DE/EN</code> - Location information</li>
<li><code>paragraph iconclass DE/EN</code> - <strong>Iconographic
content description (NEW)</strong></li>
</ul></li>
</ol>
<hr />
<h2 id="ai-prompts-used">AI Prompts Used</h2>
<h3 id="paragraph-generation-prompt">Paragraph Generation Prompt</h3>
<pre><code>Convert the following structured information into a coherent text in German.
The text contains field data that should be transformed into flowing prose while preserving all information.

IMPORTANT:
- Write a MAXIMUM of 2 paragraphs
- Do NOT include any URLs or web links
- Do NOT include reference codes or numerical codes
- Do NOT add any comments or explanations
- Only output the paragraph text itself

Field: {field_name}
Text: {cleaned_text}

German text (maximum 2 paragraphs):</code></pre>
<h3 id="iconclass-paragraph-prompt">ICONCLASS Paragraph Prompt</h3>
<pre><code>Based on the following Iconclass descriptions, write a brief German paragraph describing what the image depicts.

Descriptions: {'; '.join(descriptions)}

IMPORTANT:
- Start with "Das Bild zeigt" or similar phrasing
- Combine all descriptions into a flowing text
- Maximum 1-2 sentences
- Do NOT include iconclass codes or numbers
- Do NOT include reference codes starting with "bh"
- Only output the descriptive German text

German description:</code></pre>
<p><strong>Example ICONCLASS Processing:</strong></p>
<p><strong>Input from data:</strong></p>
<pre><code>a5500: 31 A 23 1 | 31 A 25 11 | 31 B 62 11</code></pre>
<p><strong>ICONCLASS lookup (from cache):</strong></p>
<pre><code>31 A 23 1 → "standing figure"
31 A 25 11 → "arm raised upward"
31 B 62 11 → "looking upwards"</code></pre>
<p><strong>AI-generated output (DE):</strong></p>
<pre><code>Das Bild zeigt eine stehende Figur mit erhobenem Arm, die nach oben blickt.</code></pre>
<p><strong>Translation (EN):</strong></p>
<pre><code>The image shows a standing figure with raised arm, looking upwards.</code></pre>
<h3 id="translation-prompt">Translation Prompt</h3>
<pre><code>Translate the following German text to English.
Preserve the meaning and style as much as possible.

IMPORTANT:
- Do NOT include any URLs or web links in the translation
- Do NOT include reference codes starting with "bh" followed by numbers
- Do NOT include numerical codes like 08012353
- Do NOT add any comments or explanations
- Only output the translated text itself

German text: {text}

English translation:</code></pre>
<hr />
<h2 id="kisski-cluster-resources">KISSKI Cluster Resources</h2>
<h3 id="hardware-configuration">Hardware Configuration</h3>
<p><strong>GPU:</strong> NVIDIA A100 (80GB VRAM)</p>
<ul>
<li>Architecture: Ampere</li>
<li>Tensor Cores: 432</li>
<li>FP16 Performance: ~312 TFLOPS</li>
<li>Memory Bandwidth: 2 TB/s</li>
</ul>
<p><strong>Allocation per job:</strong></p>
<ul>
<li>GPUs: 1× A100</li>
<li>CPUs: 4 cores</li>
<li>RAM: 64 GB</li>
<li>Time limit: 6 hours per job</li>
</ul>
<h3 id="job-array-configuration">Job Array Configuration</h3>
<p><strong>Array setup:</strong></p>
<ul>
<li><strong>Total jobs:</strong> 75 (indices 0-74)</li>
<li><strong>Chunk size:</strong> 400 rows per job</li>
<li><strong>Parallel jobs:</strong> 10 simultaneous</li>
<li><strong>Total rows processed:</strong> 30,000 (rows 0-29,999)</li>
</ul>
<h2 id="output-structure">Output Structure</h2>
<pre><code>data_zeichnungen/
├── enriched_data/
│ ├── zeichnungen_0-399.tsv # Rows 0-399
│ ├── zeichnungen_400-799.tsv # Rows 400-799
│ ├── zeichnungen_800-1199.tsv # Rows 800-1199
│ └── ...
├── images/
│ ├── {image_id_1}.jpg # IIIF thumbnail (224×224)
│ ├── {image_id_2}.jpg
│ └── ...
└── README.md # This file</code></pre>
<h3 id="output-fields">Output Fields</h3>
<p>Each TSV file contains the original metadata plus AI-generated
fields:</p>
<p><strong>Original fields:</strong> All fields from
<code>zeichnungen.tsv</code> including:</p>
<ul>
<li><code>a8540</code> - Image ID (BILDDATEI-NR.)</li>
<li><code>textobj</code> - Original object text</li>
<li><code>textfoto</code> - Original photo text</li>
<li><code>a5500</code> - ICONCLASS codes (primäre Ikonographie)</li>
<li><code>aob26</code>, <code>aob28</code>, <code>aob30</code> -
Relations</li>
<li>etc.</li>
</ul>
<p><strong>Generated fields:</strong></p>
<ul>
<li><code>paragraph foto DE</code> - German description of
photograph</li>
<li><code>paragraph foto EN</code> - English translation</li>
<li><code>paragraph obj DE</code> - German description of
object/artwork</li>
<li><code>paragraph obj EN</code> - English translation</li>
<li><code>paragraph verwalter DE</code> - German description of
collection</li>
<li><code>paragraph verwalter EN</code> - English translation</li>
<li><code>paragraph standort DE</code> - German description of
location</li>
<li><code>paragraph standort EN</code> - English translation</li>
<li><code>paragraph iconclass DE</code> - <strong>German iconographic
description</strong></li>
<li><code>paragraph iconclass EN</code> - <strong>English iconographic
description</strong></li>
</ul>
<h2 id="kisski-documentation">KISSKI Documentation</h2>
<ul>
<li><strong>Main documentation:</strong> https://docs.hpc.gwdg.de/</li>
<li><strong>GPU partitions:</strong>
https://docs.hpc.gwdg.de/how_to_use/compute_partitions/gpu_partitions/</li>
<li><strong>Account types:</strong>
https://docs.hpc.gwdg.de/start_here/account_types/</li>
</ul>
<hr />
<h2 id="iconclass-resources">ICONCLASS Resources</h2>
<ul>
<li><strong>Official website:</strong> https://iconclass.org/</li>
<li><strong>Data repository:</strong>
https://github.com/iconclass/data</li>
<li><strong>Help documentation:</strong>
https://iconclass.org/help/lod</li>
</ul>
<hr />
<h2 id="data-usage-citation">Data Usage & Citation</h2>
<p><strong>Source Institution:</strong> Bibliotheca Hertziana - Max
Planck Institute for Art History</p>
<ul>
<li>Website: https://www.biblhertz.it/</li>
<li>Fotothek: https://fotothek.biblhertz.it/</li>
</ul>
<p><strong>AI Processing:</strong></p>
<ul>
<li>Model: Google Gemma 2 9B Instruct</li>
<li>Infrastructure: KISSKI (GWDG Göttingen)</li>
<li>Processing date: November 2024</li>
</ul>
<p><strong>ICONCLASS:</strong></p>
<ul>
<li>ICONCLASS classification system</li>
<li>Data source: https://github.com/iconclass/data</li>
</ul>
<p><strong>License:</strong> Please refer to the Bibliotheca Hertziana
for source data licensing terms.</p>
<hr />
<h2 id="quality-notes">Quality Notes</h2>
<h3 id="general-ai-generated-content">General AI-Generated Content</h3>
<ul>
<li>AI-generated texts are meant to enhance discoverability and
accessibility</li>
<li>Generated descriptions may contain inaccuracies or
interpretations</li>
<li>Always refer to original structured metadata (<code>textobj</code>,
<code>textfoto</code>) for authoritative information</li>
<li>Translations preserve meaning but may not capture all nuances of art
historical terminology</li>
</ul>
<hr />
<p><strong>Generated:</strong> November 2024 <strong>Processing
location:</strong> KISSKI HPC Cluster, GWDG Göttingen
<strong>Contact:</strong> [e-mail]</p>