Susceptibilities and Patterning: A Primer on Linear Response in Bayesian Learning

These notes introduce the theory of susceptibilities for interpreting neural networks. The susceptibility of an observable to a data perturbation is defined as a derivative of a posterior expectation, which by the fluctuation-dissipation theorem equals a posterior covariance. Different choices of observable yield different objects: per-sample losses give the influence matrix (the Bayesian influence function), while component-localized observables give the structural susceptibility matrix that pairs model components with data patterns. The susceptibility matrix is (up to a factor) the Jacobian of the map from data distributions to structural coordinates; its pseudo-inverse provides a linearized solution to the patterning problem: finding data perturbations that produce a desired structural change. We motivate the theory from its statistical-mechanical foundations, then give a detailed exposition of susceptibilities, their empirical estimators, and their connection to the geometry of the loss landscape.