Delayed embedding, and holding it transversely stable via invertible generalized synchronization.
Here is a puzzle. Train a recurrent neural network (or a reservoir computer) on a single stream of measurements from a chaotic system — say one coordinate of the Lorenz attractor — and then close the loop so the model feeds on its own output. Remarkably, it will keep generating a trajectory that lives on the right attractor, reproducing the system's long-term statistics and even its Lyapunov exponents, despite never being given the governing equations. Why does this work? The answer braids together three classical ideas from dynamical systems.
Takens' theorem tells us that the geometry of a $d$-dimensional attractor can be reconstructed from a single scalar observable $s(t)$ by stacking its time-delays,
For $k$ large enough, the delay map is a diffeomorphism onto the true attractor: no information is lost, only re-coordinatized. A recurrent model does something similar but richer — instead of hand-picked delays, its hidden state $\boldsymbol{r}(t)$ integrates the input history through its own dynamics, learning an adaptive embedding.
Drive a recurrent system with an input signal from a source dynamical system with state $\boldsymbol{x}(t)$:
When the driven system settles, its state stops depending on its own initial condition and becomes a function of the driver: $\boldsymbol{r}(t) = \boldsymbol{\phi}(\boldsymbol{x}(t))$. This is generalized synchronization (GS). The network is now a living, continuously updated encoding of the world's hidden state — exactly the embedding we wanted, formed automatically by the coupling.
An embedding is only useful for forecasting if we can read the world back out of it. The key condition is that the synchronization map $\boldsymbol{\phi}$ be invertible — a diffeomorphism, not a many-to-one collapse. When $\boldsymbol{\phi}$ is invertible, a trained linear (or simple) readout $\psi$ can recover the next observable, and the encode–decode pair
composes into a faithful model of the flow. Closing the loop — feeding $\hat{s}$ back as the next input — turns the driven system into an autonomous replica of the original dynamics. I call this mechanism invertible generalized synchronization, and I argue it is a general principle for how recurrent systems, artificial and biological, implicitly learn dynamics (Lu & Bassett, Chaos 2020).
Synchronization is only worth having if it is stable. The set of states satisfying $\boldsymbol{r} = \boldsymbol{\phi}(\boldsymbol{x})$ forms a synchronization manifold in the joint space; learning succeeds precisely when this manifold is transversely stable — small perturbations off the manifold decay rather than grow. Formally, the conditional Lyapunov exponents (the transverse ones) must be negative:
Transverse stability is what lets a single trained readout generalize: any drive from the source is pulled onto the same reproducible embedding, so the model's predictions do not depend on where it happened to start. It is also the knob that separates a network that memorizes from one that learns the dynamics.
This is the first of two mechanisms in the talk. Recurrent models predict the dynamical world by synchronizing to it: forming an invertible, transversely-stable embedding that can be run forward. Modern transformers, by contrast, predict by hopping between basins on a learned energy landscape — the subject of Part II. Two very different architectures, two complementary geometric stories about the same goal: forecasting a changing world.
The full deck for both parts of the Santa Fe Institute talk:
Open the slides in a new tab / download (PDF) →
Related reading: Attractor reconstruction by machine learning · Reservoir observers · Model-free prediction of spatiotemporal chaos.