Accessibility and Digital Repositories: Describing Digital Collections using LLMs

In April 2024, the United States Department of Justice announced web accessibility regulations that require all state and local governments to make their websites, mobile apps, and content accessible as prescribed by U.S. law. This prompted the University of Illinois Urbana-Champaign (U of I) Library to both evaluate the accessibility of our repository systems and the content we steward, and also review how digital assets are created. Broadly, there are three areas of focus for digital accessibility in our repositories: user interfaces, existing digital assets, and future digital asset creation. The University of Illinois boasts one of the largest physical library collections in the United States and hosts more than more than 3 million digital assets across our repository services, including digitized newspapers, special collections, scholarship, and digitized books. The content creation, ingest pipelines, and homegrown repositories have not been designed with accessibility first, making adherence to the web accessibility requirements a daunting challenge. U of I librarians and staff are investigating whether multimodal large language models (LLMs) can be leveraged to meet the technical accessibility requirements for description of digital assets. The authors designed a pilot project to generate alternative text (i.e., alt text) using a local installation of Meta’s pre-trained Llama 3.2-Vision. Our preliminary findings suggest that, while it’s a viable tool to describe some types of images, there are certain technical issues that need to be addressed before it could be implemented into daily workflows. Strategies to address the ethical challenges of using LLMs also need to be addressed, including environmental impact, copyright issues with LLM training, and bias inherent in the description of cultural heritage alt text. Our presentation shares the challenges and successes of the pilot project and discusses our institution’s possible approach to alt text creation based on the findings of this pilot. (presented on 15 August 2025 at "Pushing Boundaries to Next Generation Cataloguing: Experiments at the Edge of AI and Metadata" session)