Presented at the Santa Fe Institute
Invited talk & visiting scientist — May 2026 · Host: Yuanzhao Zhang
This interactive essay re-derives single-head self-attention as one step of the
Concave–Convex Procedure (CCCP) on a Gaussian-mixture energy landscape, and shows
that the update is geometrically a “hop” along the surface normal followed by a
fixed linear readout. It also covers the tanh and linear kernel variants, with a live demo.
Open essay full-screen ↗
Jump to the interactive demo ↓
Slides (PDF) ↗
The slide deck covers both parts of the Santa Fe talk. This part (attention as
landscape hopping) is kept brief in the slides — I walked through it live using the
interactive demo above rather than static slides.