Neural networks as field theories.
An ensemble of neural networks with randomly drawn parameters defines a random function, that is, a statistical field theory. Its correlation functions can be computed directly in parameter space, without writing down an action. This project asks which field theories, and in particular which conformal field theories, can be built this way. It grew out of my master's thesis on the conformal bootstrap.
With Jim Halverson and Jiahua Tian, I built conformal fields in D dimensions from homogeneous neural networks in D + 2 dimensions with a Lorentz-invariant parameter distribution, restricted to the projective null cone of the embedding formalism. We computed exact four-point functions in several examples and decomposed them into 4D conformal blocks, using methods from Feynman integrals where needed. In the infinite-width limit, where the network becomes a Gaussian process, the construction gives generalized free CFTs and the free boson; for deep networks, recursion relations carry conformal fields and their four-point functions from layer to layer.
With Manas Dogra and Jim Halverson, I extended the construction to spinning conformal fields, computed their two-, three- and four-point functions, and recovered the free 4D Maxwell CFT from an ensemble of independent neurons at infinite width.
With Sam Leutheusser, I took a closer look at the operators of these theories. They contain an infinite tower of degenerate operators with the same scaling dimension, which has to be taken into account when matching conformal block decompositions, and their operator product expansion holds inside correlators rather than as an identity between operators. We also lay out a recipe for the algebra of the theory; since these theories are in general not unitary, we propose that it acts on a space with an indefinite inner product.
Generative models are increasingly trained on data that earlier models produced. When this repeats over several generations, the models can degenerate: rare events in the tails of the original distribution disappear first, the outputs lose diversity, and the learned distribution drifts away from the real one. This effect, called model collapse, has been seen in language models, variational autoencoders and Gaussian mixture models (Shumailov et al., Nature 2024).
I am currently studying it.
How these papers connect to each other, and to my other work: My Research Journey.