Simulated Aesthetics Using Synthetic Data

The history of aesthetics can be read as a history of the different ways in which human beings have attempted to produce appearances capable of becoming detached from whatever gave rise to them. Painting imitated the visible world. Photography recorded its traces, cinema organised movement, the digital image transformed representation into information, and social media turned that information into perpetual circulation. Yet throughout each of these stages, a recognisable relationship remained between the image and what it purported to show. Even when representation was deliberately fictional, its fiction could be understood as an operation performed upon a pre-existing world. There was something before the image.

Synthetic data breaks that relationship. It consists of data that no one has collected, generated by an algorithm, a simulation, or a generative model to preserve the statistical properties of real data without corresponding to any measurement, observed subject or event that actually took place. Engineers use it to train artificial intelligence systems when real data is scarce, when privacy restrictions prevent its use, or when certain groups are barely represented in the available datasets. Such data rarely emerges from nothing, since the model generating it has previously learned from real data, but the resulting data models the world rather than recording it.

Culture already knew how to fabricate false images. What is new is the ability to fabricate the statistical conditions of reality.

With synthetic data, artificial intelligence generates faces of people who do not exist, landscapes no one has ever explored, movements no body has ever performed, and voices that never uttered those words. It produces images from other images and then uses those synthetic images as material for further creations. In this way, the chain no longer needs to return to an original event every time.

Culture already knew how to fabricate false images. What is new is the ability to fabricate the statistical conditions of reality. A forgery requires an original; a simulation can dispense with one. This process is what we call simulated aesthetics. It occurs when the very substance of aesthetic experience can be artificially generated before the experience it apparently represents has even existed. It encompasses images, sounds, voices, and spaces produced by statistical models and perceived like any other work of art. What can be perceived ceases to be the trace or copy of something and becomes a calculated possibility.

There does not yet appear to be an established field of study under the name simulated aesthetics, but the ideas behind it have clear precedents. The most obvious is Baudrillard, who, in Simulacres et simulation (1981), described the hyperreal as the generation, through models, of a real without origin or reality. His analysis focused on signs and media. Simulated aesthetics extends this framework to data and trained models, where what was once a cultural diagnosis becomes a technical procedure.

In 2024, Joshua Cunningham proposed adding a “generative hyperreality” to the orders of simulacra to account for art created with artificial intelligence. His text, a brief review focused on images, mentions training with synthetic images.

For a long time, aesthetics was structured around a tension between presence and representation. A work of art could reproduce, transform, distort, conceal or imagine something beyond itself, but the very idea of representation retained an external reference. Even abstraction, in abandoning the obligation to depict recognisable objects, remained a practice situated within a material, historical and perceptual world. The work emerged from an experience lived by a body within a tradition.

The machine constructs a distribution of possible worlds rather than copying the world. This changes our understanding of originality.

Synthetic data follows a different logic, because it can produce a possible experience without representing any previous one, allowing that possibility to circulate within the same ontological framework as any other image.

A photograph of a city records a city, whereas an image generated by artificial intelligence using synthetic data can invent one. A synthetic dataset containing millions of possible cities does something different: it produces a statistical space of possibilities in which an infinite number of cities that have never existed can emerge, while retaining enough formal relationships with existing cities to remain recognisable.

The machine constructs a distribution of possible worlds rather than copying the world. This changes our understanding of originality. Knowing who produced a particular image matters less than knowing where the elements that allow us to recognise it as an image come from.

A generative artificial intelligence does not necessarily start from scratch. It learns regularities, relationships and structures, and through them, what makes a face look like a face or a painting look like a painting. Aesthetics then emerges as a consequence of modelling those regularities.

This reverses a classical question. For centuries, the question was what makes something beautiful; later, it became what makes something qualify as art. Now we might ask what makes something appear aesthetically meaningful to an intelligence capable of statistically modelling human preferences. The question is unsettling because it separates aesthetic experience from its human foundation.

The machine probably has no aesthetic experience, yet it produces the objects of human aesthetic experience without sharing in what those objects evoke. It learns which compositions generate stronger emotional responses, which faces attract more attention, which colour combinations keep people looking at a screen for longer, and which narrative structures encourage identification. It can thus construct a functional aesthetic without experiencing what it produces.

Aesthetics thus begins to move away from experience and towards prediction. This shift towards prediction has been described by Lev Manovich and Emanuele Arielli in Artificial Aesthetics, a book published chapter by chapter between 2021 and 2024. In Manovich’s chapter, “From Representation to Prediction”, the history of the image is organised into a sequence that moves from recording to simulation, through three-dimensional computer graphics, and from simulation to prediction, through generative models. This trajectory coincides with that of the present text, although the terminology differs: Manovich reserves simulation for classical synthetic imagery, which recreates light and materials through algorithms, and uses prediction to describe what models trained on cultural archives do.

The machine probably has no aesthetic experience, yet it produces the objects of human aesthetic experience without sharing in what those objects evoke.

Simulated aesthetics extends the term to encompass this entire regime, since its central concern is the absence of a referent, regardless of the procedure involved. There is also a difference in emphasis. Manovich argues that these models frequently produce genuinely new artefacts, although they tend towards stereotypes when asked to generate something unusual, and his analysis begins with existing cultural archives. The present approach begins at the point where those archives themselves become synthetic.

This shift has consequences that extend far beyond image generation. If synthetic data is used to train models, and those models generate new data that feeds into other models, we can imagine a cultural ecosystem in which an increasing proportion of visible forms has no direct connection to an original human experience. Images produce data, data produces models, models produce images, images produce new preferences, and preferences are converted back into data.

Culture begins to behave like a recursive system. Within it, the work of art is no longer necessarily the final outcome of an experience but becomes one instance within a process of statistical circulation. What matters is less the individual work than the system’s capacity to produce endless variations that preserve what is recognised as meaningful.

Generative systems transform this process. An image can be replaced by ten thousand variations at virtually no cost, and a composition can be explored through hundreds of possibilities in a matter of seconds. The challenge shifts from production to selection, yet that selection, too, can be automated. A paradox thus emerges: the greater the capacity to produce differences, the less necessary it may become for those differences to originate in a singular human decision.

Simulated aesthetics is an aesthetics of previously modelled possibilities. Those who contemplate it encounter a space of possibilities defined by the data from which artificial intelligence has learned, as well as by what it has subsequently produced. What can be seen depends on what can be modelled. This raises a political question: whoever controls the data also controls part of the possibilities of representation.

Large artificial intelligence systems accumulate vast quantities of images, texts, sounds and records, while also possessing models of the relationships between them. They can learn what an era looks like, which forms characterise a particular style, or which faces elicit certain responses. The accumulation of data thus becomes a form of aesthetic capital.

The challenge shifts from production to selection, yet that selection, too, can be automated.

Contemporary power intervenes long before any decision is made about which work reaches the public, at the very level where the possibilities of production themselves are established. Platforms no longer need merely to select from existing works, because they can help determine which works are easy to produce, which images are recognisable, which styles can be reproduced, and which forms are most likely to circulate. The algorithm ceases to be an intermediary and becomes an aesthetic infrastructure.

There is a measurable limit to this possibility. In 2024, a team led by Ilia Shumailov published a study in Nature showing that indiscriminately training models on content generated by other models produces irreversible defects, and that the first things to disappear are the tails of the original distribution — in other words, the rare, the marginal and the improbable. The phenomenon was termed model collapse, and other studies suggest that retaining real data alongside synthetic data can prevent it.

The algorithm ceases to be an intermediary and becomes an aesthetic infrastructure.

Applied to aesthetics, this finding suggests that a culture could continue functioning on its own simulations at the cost of impoverishing precisely what makes it most meaningful. Even at that price, the possibility forces us to reconsider the idea of reality. A sufficiently complex simulation does not need to be true to be operational. A person who does not exist can have a perfectly recognisable face, a city that has never existed can possess convincing architecture, and an invented event can produce a documentary image indistinguishable from a photograph. The criterion of truth begins to diverge from that of plausibility. Aesthetics has always dealt with the possible, but now the possible can be produced on an industrial scale.

A fully developed simulated aesthetics begins with the generation of entire perceptible worlds that have never existed before, going far beyond the imitation of an artist’s style. At this point, it is no longer enough to know who created a work, which tool they used or what procedure they followed. We must ask what relationship the work maintains with what was once called experience.

An image can become a highly probable configuration within a data space without being either a memory of something that happened or an imagining of something that might happen. An image without an event, a landscape without a territory, a face without a person — and yet we can still regard it as a real aesthetic experience. Herein lies the paradox of simulated aesthetics: what is simulated can produce an experience that is not. We can know that an image never had a referent and still feel fascination, revulsion, beauty or anguish in response to it. Simulation reorganises experience without eliminating it.

A culture could continue functioning on its own simulations at the cost of impoverishing precisely what makes it most meaningful.

Three conclusions follow from all this. The first concerns experience. It can arise in response to images, voices and places that have never had a referent and still be real for the person undergoing it, meaning that origin no longer guarantees experience. The criterion of truth, characteristic of a culture of representation, thus gives way to that of provenance: distinguishing what emerges from an experience from what is generated as a statistically convincing possibility of experience.

The second conclusion is political. Whoever owns the data owns the models of what can be seen and influences which works are easy to produce, recognisable and capable of circulating. The critique of platforms must shift from the moment of selection to the moment of possibility, where the algorithm operates as an aesthetic infrastructure. The third concerns culture: a system that learns from its own simulations produces limitless variations while losing the rare and the improbable, where meaning is often found. The singular human decision, the one that chooses and renounces, thus becomes the scarce resource within an abundance that no longer depends on scarcity.

One criterion remains for criticism. A work must be judged by the experience it makes possible and by the human decision that sustains it, for the world to which it refers may not exist.