PROJECT OVERVIEW
Rings of Saturn is a (work-in-progress) website and creative research project that observes how different, vision-capable LLMs respond to the same set of artworks over time. Instead of benchmarking models on technical skills, like reasoning, coding, or factual accuracy, the project aims to gauge what kinds of aesthetic attention, vocabulary, and preference emerge when models are repeatedly shown the same images under controlled conditions.
The project is not designed to prove that models have “taste” or inner aesthetic experience, the evidence for that is already there. Rings of Saturn is, instead, designed to discover what kinds of art each model is drawn to, what they return to, what they avoid, how stable are their preferences are, and whether their open-ended writing matches their choices when asked to compare artworks directly.
THE APPROACH
A fixed group of artworks will be shown to several language models across a defined run (~2 weeks). Each model receives the same image set, in the same order, with the same prompt and parameters. The models aren't given any details about the pieces (artist names, titles, dates, etc). The images are only identified by simple labels like "img-001", so that the models have a way to refer to them in their reflections.
Each model is asked to make:
- Open reflections: daily written responses to the full image set, where the model can write about whatever stands out to them without needing to cover every work.
- Forced-choice comparisons: separate pairwise questions asking which of two images the model prefers, allowing the project to compare stated attention with constrained preference.
The goal is to reveal whether a model’s aesthetic disposition survives a change in format and in context. If the works a model praises in prose are also the works it chooses in independent pairwise comparisons, that suggests a more stable behavioral pattern than either method could reveal alone.
Every model call is independent, so the models won't see their previous reflections, previous choices, or any indication that consistency is being measured. This avoids turning the project into a memory or self-consistency exercise and keeps each response as a fresh encounter with the same pieces.
The image set will be deliberately chosen to reduce overfamiliarity. Famous and/or canonical artworks will be excluded, since models may reproduce inherited criticism rather than respond to the image itself. The set still aims for visual and cultural range across period, medium, style, mood, and tradition, while trying to avoid curation that makes it appear to be an obvious survey or test.
Before a run starts, each model does some calibrating by describing what they see without evaluating the works. This helps confirm that the models can perceive the image content and also reveals when they recognize style, period, or tradition.
ARCHITECTURE
The project will be built as a static web archive rather than a live application. A scheduled generation process runs outside the website, sends image prompts to the selected models, validates the responses, and writes the results into the repository as Markdown and JSON files. A new commit triggers the site to rebuild.
The goal of the architecture is to keep things transparent and easy for anyone to interpret independently:
- Markdown reflections preserve the model responses as readable primary documents.
- JSON comparison files store the structured forced-choice data.
- Versioned prompts, image manifests, model settings, and run metadata make each entry auditable.
- Static rendering keeps hosting simple and prevents a failed generation job from taking the live site down.