The Feral Engine
An AI that learns to understand a world by acting on it.
Today's generative AI can produce images, simulate worlds and even act. Yet its learning is not rooted in a place, it reproduces the familiar rather than the new, and it acts mainly in controlled environments. The Feral Engine provides what is missing: a physical site, authors and actors.
Three dimensions, one shared engine.
The Feral brings together a collective artwork, an Institute and a studio through a shared technological infrastructure: the Feral Engine, an AI that learns to understand a world by acting on it.
An ongoing artwork unfolding in Epochs, each entrusted to an invited artist. It encompasses both a film, shot on site and edited by an AI learning to understand the place, and the transformation of the site itself.
It provides the Engine with its corpus and an author's artistic demands.
The Feral's platform for research, education and artistic practice: residencies, a school and an annual publication. It develops the project's concepts and experiments, exploring their philosophical, technological and political implications.
It enables young researchers and artists to experiment with the Engine.
The production infrastructure: the Membrane, indoors and outdoors, on-site computing and models trained on the site. It is designed to run the Engine and make its tools available to external productions.
It develops tools from the Engine for use in other productions.
A place, a world, actions.
What today's generative AI lacks, and what the Feral Engine brings to it.
Large models learn from images gathered from many places. Systems that reconstruct a real place (digital twins, splatting) capture a single moment, without tracking how it changes over time.
A 15-hectare natural environment made up of forests, streams, rocks and meadows.
World models generate coherent environments. Like all statistical models, they reproduce common patterns in their data more readily than rare or previously unseen situations.
Visual artists, writers and choreographers are invited each year to create radically new configurations of the world. These unprecedented situations push the machine to learn beyond familiar patterns.
When AI acts in the physical world through robotics, it usually performs specific tasks in controlled settings: laboratories, warehouses or factories. Long-term experiments in open natural environments remain rare.
Long-term participants in the project. At each training session, these men, women and children follow the machine's instructions on site, enabling it to act on the environment and learn from the changes it brings about.
An initial, temporary testing ground where the system is built, tested and demonstrated.
An AI that learns to understand a place by acting within it, designed for use at other sites and in other contexts.
Understanding by acting.
The starting hypothesis: we can only understand a world by acting on it. Observation reveals what exists; acting shows what makes things change.
An artist creates an event on site; the machine investigates what made it possible and revises its model.
The machine compares the imagined state (World) with the actual outcome (Place′): it learns from the difference and updates its model.
the physical site as it is: forest, rocks, stream, light, continuously captured by the machine.
the place as defined by an author's ontology. E.g. Epoch 1: the site as perceived by men and women of the year 1000.
actors, crew or machine (cameras, lights) carry out the approved instructions on the site.
The conceptual framework.
The Feral Engine brings current approaches to AI together in the service of artistic creation. These fall into three broad families, corresponding to the three modes of reasoning described by the philosopher Charles S. Peirce.
Generative AI and 3D reconstruction produce images and geometry of unprecedented quality. They identify patterns and associations more readily than the causes of change.
A curious agent selects the experiments it can learn the most from (P.-Y. Oudeyer). Faced with an unexpected event, it proposes an explanation for what caused it.
Observation alone cannot reveal what causes change: a causal model is needed, along with interventions to test it (J. Pearl, J. Woodward).
On site, the Place · World · Actions loop brings these three approaches together: explain, anticipate, verify.
The consortium.
Formed for the France 2030 funding application “Digital transformation of culture and adoption of AI” (first stage approved in July 2026). The Feral designs and leads the entire pipeline.
Reconstruction, specialised generation, perceptual image quality.
Capture and edge computing.
Artificial curiosity and autonomous experiment selection.
Actors, the film crew and the Membrane (the on-site capture and lighting rig) put the instructions into practice.
The consortium's overall decision-making framework.
Traceability of contributions and allocation of rights.
Responsible AI and sustainable digital practice: environmental impact measurement and eco-design of the system.
The Membrane: an interface with the site.
The Membrane is the interface with the physical environment: it records the site and relays instructions to the actors. A first prototype of the complete loop is in development.
The software pipeline: three transferable building blocks.
The software pipeline implements the Engine's cycle in seven steps, from recording the site to transforming it.
The Membrane records images, sounds, camera positions and lighting, with each state timestamped and spatially referenced.
Reconstruct the site's geometry and train generative models on its images and sounds.
Generate images and sounds of possible states of the site within the author's world.
The author selects one of the possible states.
Generate instructions for the actors and crew, and settings for the equipment (cameras, lights).
Actors, crew and equipment bring the selected state into being on site, within the Membrane.
Record Place′, measure the difference between the imagined and actual outcomes, and feed it back into the model.
Three transferable building blocks, usable individually or together at other sites and with other teams. Intended uses: generative memory for a series set or heritage site · planning lighting, sets or installations · comparing previsualisations with actual results.
Three evaluation criteria: fidelity to the site · predictive accuracy · value to the author.
Before, during and after the shoot.
The technical pipeline, step by step: installing the Engine on a set, preparing, shooting, explaining, verifying, then finishing and extending. Click a step to see its technologies.
Before the shoot: preparing
Scouting, writing, breaking down and previsualising in the measured set: what will be shot in reality is prepared in the Double Statistique.
1Block 1
Scout and measure
a scout that lasts
The set is surveyed once; then we decide where to place the Membrane and how many campaigns to run.
- Initial survey by drone and photogrammetry
- First model: SfM and Gaussian Splatting
- Geodetic anchoring outdoors, targets in the studio
- Capture plan: gantries, campaigns, seasons
2Authors
Define the world
the author's ontology
The author sets out what exists in their world, its names and its causes. E.g. Epoch 1: the year 1000.
- Structured world bible: entities, rules, vocabulary
- The author's reference corpus, under licence
- LoRA adaptation of the style on this corpus
- Rights and consents recorded before any training
3Block 2
Explore the possibilities
what the set can become
The author browses the states the set can take: times of day, seasons, weather, transformations.
- State generation: splat rendering and diffusion
- Navigable map of states, endogenous and exogenous
- Relighting (time, weather) learned from light probes
- Artificial curiosity: the least known states flagged
4Authors
Write and break down
a situated shot list
The shot list is made in the measured set: each shot has its position, focal length, height and state.
- Virtual camera matched to the FR7: sensor, lens, height
- Each shot linked to a state on the map
- Export of rail and column positions
- The author's notes carried into the bible
5Block 2
Previsualise
rehearsing before shooting
Camera, actors' movements and light are tried out in the double, then approved.
- Real-time rendering of the splat (game engine)
- Actor placement and blocking
- Animatic by video diffusion, at the shot's focal length
- Approval by direction, camera and production design
6Block 3
Schedule
shoot or generate
For each shot: to shoot, to generate, or to generate then shoot. The schedule follows the light.
- Shots sorted: shot, generated, exogenous to shoot
- Schedule matched to forecast light and weather
- Machine presets loaded: rail, framings, DMX
- Direction sheets for actors and crew
Once measured, the set remains available at any hour and in any season, without permits or moving a crew: you can return to it as often as needed.
What must be lived is shot; what the set already knows is generated; the exogenous is generated first, then shot to become endogenous.
During the shoot: augmenting
The augmented shoot: from a bare set to the first generated shot, a six-step loop, the direct route from Place to Place′.
1Block 1Place
Capture
installing the Membrane on the set
Gantries, rail and column use permanent reference points. Each take is timestamped and spatially referenced.
- Sony FR7 (S-Log3), motorised rail and column, logged positions
- 12 synchronised measurement cameras (Lucid Triton2, PTP clock)
- Controlled lighting (Vortex, DMX) and light-probe spheres
- AprilTag targets and fixed anchors: metric scale, shared reference frame
2Block 1Place
Index
building a timestamped corpus
Views are synchronised, colour-graded and organised into strata (one capture campaign per stratum), with provenance recorded.
- Take manifest: date, position, lens, lighting
- The film's LUT is applied during extraction, producing colour-graded images
- On-site storage and computing (technical basement), orchestration
- Provenance register of contributions and rights
3Block 1Place → World
Model
building the Double Statistique
Not a twin that represents the set: a double that learns what it can become.
- Poses: SfM (COLMAP / GLOMAP), scaled using the targets
- Geometry: 3D Gaussian Splatting; motion layer (canonical scene + deformation)
- Appearance: Flux diffusion model, LoRA adapted on selected views
- Perceptual quality control
4Block 2Place → World
Generate
proposing possible states
Endogenous possibilities, which extend the set's memory, and an exogenous possibility, which introduces something from outside it.
- Rendering the Gaussian splat with a virtual camera matched to the FR7
- Refinement by image and video diffusion (ComfyUI), masked inpainting
- Trajectory engine; artificial curiosity to choose experiments
- General-purpose model used only to generate exogenous possibilities
5Block 2World → Actions
Choose and instruct
the author retains the final decision
The author explores the possible states and selects one; the Engine translates it into actionable instructions.
- Navigation interface: map of states, distance from the measured state
- Generated instructions for actors and crew
- Equipment presets: rail positions, framing, lighting plans (DMX)
6Block 3Actions → Place′
Shoot and reintegrate
tested against reality
The chosen state is realised, recorded and compared with the prediction. Endogenisation: once filmed, the exogenous becomes endogenous.
- Filming in the Membrane with the same reference points and settings
- Imagined vs actual states: measured geometric distance
- Reintegration as a new stratum, provenance preserved
- Conform for editing: 25p, ProRes 422 HQ, original grain and blur
P(state | situated world) ∝ Pgeneral × Psituated: a prior from a general-purpose model, constrained by learning from the site.
Endogenous possibilities extend the set's memory; exogenous possibilities introduce something from outside it. Once filmed and recorded, they become endogenous.
During the shoot: explaining
An artist creates an unexpected event; the machine looks for what made it possible and revises its model.
1AuthorsPlace → Actions
Provoke
the artist creates an event
An artist intervenes on the site following their own protocol; the Engine knows only its record sheet.
- Intervention sheet: who, what, where, when, with what means
- Zone and time slot reserved in the Membrane, permanent reference points
- No instruction given by the machine: the event is external to it
- Rights and provenance of the contribution recorded
2Block 1Place
Capture the event
before, during, after
The Membrane records the place before, during and after the intervention, with the same cameras and the same settings.
- 12 synchronised measurement cameras (Lucid Triton2, PTP clock)
- Sony FR7 author's camera, rail and column positions logged
- Light-probe spheres: the lighting of every take is known
- Three dated strata: before, during, after
3Block 1Place
Isolate the change
what moved, and what did not
The before and after strata are registered on the same anchors, then compared in geometry and in image.
- Registration on AprilTag targets and permanent anchors
- Change detection between two reconstructions
- Masks of the changed zones; the rest of the set serves as a control
- Comparison with what the model had predicted
4Block 2Actions → World
Explain
working back from effect to causes
The machine imagines what may have produced the event and tests each hypothesis in the Double Statistique.
- Causal graph of the place: light, water, soil, vegetation
- Counterfactual renders: the place without the intervention
- Hypotheses ranked by plausibility (Psituated)
- Description of the event (vision-language), reviewed
5Block 1World → Place
Revise the model
the Double Statistique corrects itself
The event becomes a stratum: the exogenous becomes endogenous. The model corrects itself where it was wrong.
- New dated stratum, the artist's provenance preserved
- Targeted update: local splat and LoRA retrained on the changed zones
- Priors revised: less weight to the general-purpose model where the place has decided
- Revision log: what the model believed, what it now believes
6Block 2World → Actions
Propose an experiment
telling the hypotheses apart
The machine chooses the intervention it would learn the most from and proposes it to the author. Once approved, it becomes an instruction.
- Artificial curiosity: expected learning gain per experiment
- Experiments ranked by cost, risk to the place and interest
- Approval by the author; instructions for the crew and the machine (DMX, rail)
- Back to step 1: a new event, this time provoked
A transformation of the place that the Double Statistique judged unlikely: low Psituated(event). This is where the machine has the most to learn.
Peirce's abduction: working back from the observed effect to its possible causes, then choosing the experiment that tells them apart (artificial curiosity, P.-Y. Oudeyer).
During the shoot: verifying
Before the take, the Engine records what it anticipates; afterwards, the gap between World and Place′ is measured and learned.
1Block 2World
Record the anticipation
a falsifiable prediction
The chosen state is rendered from the author's camera viewpoint, with its uncertainties, then dated and sealed.
- Splat rendering and diffusion refinement, matched to the FR7
- Uncertainty map: where the model is confident, where it hesitates
- Timestamp and fingerprint of the prediction in the register
- Shooting instructions attached to the prediction
2Block 3Actions → Place′
Shoot
realising the chosen state
Actors, crew and machine carry out the instructions in the Membrane: same reference points, same settings.
- Rail and column positions, framings and lighting plans preset (DMX)
- Instructions to actors and crew, as approved
- Execution gaps noted: what was not done as planned
- FR7 rushes in S-Log3, the film's LUT
3Block 1Place′
Recapture Place′
measuring what happened
Right after the take, the Membrane recaptures the transformed place under the conditions of the prediction.
- 12 synchronised measurement cameras (Lucid Triton2, PTP clock)
- Same anchors, same metric scale, light-probe spheres
- Reconstruction of Place′ (SfM, Gaussian Splatting)
- New dated stratum, linked to the prediction
4Block 3World ↔ Place′
Measure the gap
between the imagined and what happened
Prediction and Place′ are overlaid from the same viewpoint: the gap is measured zone by zone.
- Distance between geometries (Chamfer, depth difference)
- Perceptual and photometric gap (LPIPS, FLIP), under known lighting
- Gap map, cross-checked with the uncertainty map
- Share due to the model, share due to execution
5Block 1Place′ → World
Learn from the gap
feeding it back into the model
The gap becomes a learning signal. Endogenisation: once filmed, the exogenous becomes endogenous.
- Targeted retraining of the splat and LoRA on the zones in error
- General-purpose model reweighted where the place has decided
- Uncertainties recalibrated: the model also learns to doubt
- Log: prediction, result, correction
6ValidationCycle after cycle
Track the gap
proof that the Engine learns
The same protocol is repeated at every shooting cycle: the mean gap must decrease, season after season.
- Test bench: the same situations replayed from one cycle to the next
- Gap curve by cycle, by zone, by type of state
- Validation reports for moving up the TRL scale (4 to 6, then 7)
- Results shared with the consortium and published
d(World, Place′): the distance between the anticipated state and the recaptured state, measured in geometry and in image. It is the Feral Engine's learning signal.
How far does feedback from the real world improve, from one cycle to the next, what the Engine anticipates? The gap must shrink.
After the shoot: finishing and extending
The set remains available: missing shots, continuity fixes and states never filmed are made in the double, and everything returns to the world's memory.
1Block 1
Log and index
situated rushes
Each rush is linked to the measured state of the set: we know where, when and under which light it was shot.
- Take metadata attached to the rushes: position, lens, light
- Synchronisation by timecode and PTP clock
- Searching the rushes by state, place, light
- Comparison with the planned shot list
2Block 2
Complete the edit
shots never filmed
A new camera move, a cutaway or a late continuity shot, made in the measured set.
- Splat rendering along an FR7 trajectory
- Refinement by video diffusion
- Integrating the actors: depth, rotoscoping
- Dated state restored, lighting included
3Block 2
Extend the states
night, rain, season, wear
The same shot can be declined into states never filmed, consistent with the measured set.
- Relighting learned from N known lightings (probes)
- Reusable atmosphere layers: mist, rain, dust
- Endogenous and exogenous transformations of the set
- Consistency check against the measured set
4Block 3
Grade and conform
matching generated and shot
Generated shots receive the film's look: same LUT, same frame rate, same grain as the shot footage.
- The film's LUT locked from capture onwards
- Conforming: 25p, ProRes 422 HQ, original grain and blur
- Documented grade, reusable from one season to the next
- Perceptual quality control of generated shots
5Block 1
Trace the origin
an exportable record
Each shot states what was filmed, what was generated, and from what. The record travels with the film.
- Shot-by-shot provenance: shot, generated, mixed
- Labelling of generated content (AI Act, art. 50)
- Rights and remuneration of contributions
- Export with the master: edit list, metadata
6Block 1
Enrich the memory
the world continues
Shot states, the author's choices, approved shots: everything returns to the double. The next production starts further ahead.
- Shot states reintegrated as dated strata
- The author's choices and rejections kept as data
- World bible updated
- Reusable world: sequel, season, spin-off
The set and its dated states remain: an episode cost becomes an asset amortised over seasons, gaining value with each production.
Shot, generated or mixed: the share of each shot is known, dated and delivered with the film, together with the rights of each contribution.
Inside the machine.
Memories, trajectories, materialisation, provenance: the Engine's internal architecture, after the diagrams of the France Télévisions dossier.
Two memories for one world
One comes from observing the real, the other from the author's project: where they meet lies the world's active memory, the Double Statistique.
The memory of the real is produced in the Membrane: the states of the set, the gestures and voices of the performers, recorded in a single frame of reference and within a rights framework agreed in advance.
The memory of the project gathers what the authors and the production have established: stories, rules of the world, continuity, editorial decisions. It is not inferred from images: it is written.
The Double Statistique arises from their articulation. It proposes endogenous possibilities, which extend what the world already holds, and, when its resources are not enough, an exogenous possibility guided by an author's intention.
Once selected, produced and fed back, the exogenous possibility enters the history of the world: it becomes endogenous.
The trajectory engine
Its central function: opening up, from the world's memory, several new and coherent trajectories. The author charts the one that is kept.
The engine connects the dimensions of the world: places, chronology, sets, lights, characters, events, rules, rights constraints.
It does not predict a single sequel: it maps the possibilities, what becomes of a place after an event, what a new element changes.
The author compares, combines, transforms, discards. The system does not choose a story in their place.
In practice: variations of a shot, missing shots, successive states of a set, a world bible that can be queried. Each proposal states its origin and how far it departs from what was observed.
Two routes to materialisation
A validated trajectory can be produced in digital space or re-staged on a set; the two routes can be combined within the same sequence.
The synthesis route makes all or part of the shot in the double: set variation, missing shot, continuity link. The performers' statistical doubles are used only within a contractual framework agreed in advance.
The rematerialisation route returns to production: the set is rebuilt or recreated in virtual production, the performers act, the Membrane captures the new state.
In both cases, the new state returns to the world's memory.
The choice depends on artistic intention, rights, cost and production conditions. It belongs to the authors, the performers and the production.
Provenance, computing and updating
Governance becomes a technical mechanism: every state keeps its status and its origin, and computing uses only as much generality as needed.
Provenance. Observed, documented, inferred or generated, selected, produced: for every proposal, we know the data used, the model version, the origin of the request, the validations and the destination.
Frugal computing. If the world's memory already holds what is needed, the specific model answers, on site. Otherwise, a general-purpose capability is called on occasionally, at an author's request.
Updating. Once selected and produced, the proposal is recaptured and integrated: the next time, the specific model uses it without calling on the general-purpose model.
The prototype: three phases, sixteen months.
The first complete prototype is under way: a 10 m outdoor Membrane, in three phases, each proving one part of the loop.
- 10 m outdoor Membrane: three gantries on screw piles
- 12 synchronised measurement cameras, network and storage
- Two real-time computing stations
- Author's camera on a manual support
- Capture v0, first training, interface v0
- Track, dolly and column, manual version
- Capture in motion, body and camera
- Exporting a state to virtual production, test shoot, recapture
- Second training
- Four-axis motorisation, absolute positioning, safety
- Commissioning and CE marking
- The Engine drives the machine
- Origin register, measured frugality, author interface v2
- Final evaluation
prototype demonstrated on site
commercial opening: first installations on other sets
complete, transferable model
After the prototype: the Membrane extended to 30 m, a portable kit for campaigns outside The Feral, computing installed in the building's technical basement. Indicative timeline; the prototype starts with current funding (France 2030, ERDF).
Core application: audiovisual creation.
For a film or a series, the Feral Engine aims to become the generative memory of a real set: preserving recorded states, proposing new ones and helping to film them.
The Membrane records the physical set.
The author defines the world of the film; the Engine proposes possible states within it.
Selected states are translated into instructions.
Filmed and generated footage are compared.
States accumulate from one episode to the next.
What the Engine aims to produce: images and sounds aligned with a real place · possible states to compare before shooting · instructions for the crew
One set, a starting state and four possible states. Epoch 1.1, conceptual illustrations.
“The Feral Engine does not replace the author: it shifts where authorship takes place.”
The first testing ground is The Feral's own film, shot on site and edited by the machine, Epoch after Epoch.
The vision: from finished content to evolving worlds.
Over the next five years, AI will not only change the way we make films and series. It will begin to change the very nature of what we make. We will gradually move from an industry that produces finished content (films, episodes, shots) to one capable of building persistent, generative worlds that evolve, retain a memory and produce new states.
In this new landscape, value will lie not only in completed works but also in a world's capacity to keep generating its own possibilities. The strategic question will extend beyond how AI can make production faster or cheaper: how can we build these worlds while preserving their distinctiveness, the role of authors, ownership of their memory and control over their possible futures?
A digital twin represents a set; the Double Statistique learns what this world can become, and generates states that have never been filmed.
Endogenous possibilities grow from the world's memory; exogenous possibilities come from outside it. The author chooses; corpus, computing and rights remain under control.
Once selected, filmed and recorded, the exogenous becomes endogenous: the world's generative memory grows in value from one episode to the next.