AI@IPCH

Humanity is at the dawn of a new era in artificial intelligence, one in which machines can speak and understand natural language. This will transform the nature of machine-assisted work in many domains, cultural heritage included. Indeed, because language is a cultural artifact, cultural heritage research is particularly vulnerable to transformation by this technology. And change is happening rapidly.

Led by Damon Crockett, AI@IPCH is endeavoring to meet the urgency of the present moment with a dedicated line of research into large language models and their multimodal variants. Of particular interest is shaping the behavior of these models to better align with the epistemological and ethical commitments of the field. We aim to make progress on two fundamental questions: 

  1. How do we evaluate the performance of generally intelligent systems in complex domains like cultural heritage research?
  2. Which interventions on model behavior—things like training curriculum, context engineering, architecture design, and loss/reward modeling—give us the control we need to align the models with our values?

Recent work on the nature of these values will be published in a forthcoming interactive essay, “Discourse Machines”, and an in-progress essay on the ethics of using AI for cultural heritage research.

Be the first to know about new publications

Sign up for the IPCH newsletter

Research Highlights

UCLA/Getty Distinguished Speaker Series

In May 2025, Dr. Crockett spoke at the UCLA/Getty Interdepartmental Program in the Conservation of Cultural Heritage on “AI & Cultural Heritage Research: Risks and the Sins of Omission.” He argued that large language models mark an inflection point in machine-assisted research, especially in interpretive fields like cultural heritage, and can reduce epistemic risk compared with earlier AI models. He also suggested that expanding access to heritage may make AI use not just permissible, but may be ethically better than refusal. These arguments will be published in two forthcoming essays.

A man stands to the right of a projector showing a presentation gesturing with his right hand.

Image Captioning in Vision-Language Models

In Fall 2024, Dr. Crockett led a study that ranked 10 vision-language models—variants of language models that can receive images as inputs—for their image captioning ability, using a difficult set of historical photographs from the Lens Media Lab’s collection. Human judges chose between candidate image captions for thousands of trials, resulting in a robust ranking of models. The study was published as an interactive web essay, “Image Captioning in Hostile Conditions”.

Rows of colorful rankings in green, blue, and yellow.

Screenshot from “Image Captioning in Hostile Conditions”, showing top vision-language models, ranked by human judges.

Paperbase

Dr. Crockett is the author of Paperbase, a web platform for exploring the Lens Media Lab’s collection of photographic paper samples, the largest of its kind in the world. The lab measured the collection for physical characteristics like color, texture, gloss, and thickness, and carried out comprehensive, high-resolution imaging of all collection items. Paperbase combines this measurement data with the collection catalog, makes it publicly available as a download, and provides visual access to the collection via an interactive web application.

An array of different tones of cream papers with the word Paperbase over it.