Hierarchical Models in the Brain

Hierarchical Models in the Brain

November 7, 2008 | Karl Friston
This paper introduces hierarchical dynamic models (HDMs) and reviews a generic variational scheme for their inversion. HDMs are general models that subsume many parametric models for continuous data, including the general linear model for static data and generalized convolution models for nonlinear time-series analysis. The models are arranged in a hierarchy, where the output of one layer serves as input to another, enabling the modeling of data of arbitrary complexity. Crucially, all these models can be inverted using dynamic expectation maximization (DEM), a scheme that optimizes the conditional density on model states, parameters, and hyperparameters. This inversion can be formulated as a simple neural network, providing a useful metaphor for inference and learning in the brain. The paper discusses the inversion of HDMs, showing how it can be implemented using a variational approach. It highlights the importance of hierarchical structures in inducing empirical priors that provide structural and dynamic constraints. The inversion process is shown to be effective for a wide range of models, from variance component estimation in classical linear models to blind deconvolution. The paper also emphasizes the role of hierarchical forms in generating empirical priors and how these can be used to interpret conditional dependencies in data. The paper presents a detailed analysis of the mathematical formulation of HDMs, including their state-space equations and the probabilistic models they entail. It discusses the use of Gaussian assumptions for fluctuations and state-noise, which provide likelihoods and empirical priors. The paper also explores the implications of these models for understanding brain function, suggesting that the brain may use sophisticated algorithms similar to those used in scientific data analysis. The paper concludes by emphasizing the importance of hierarchical structures in the brain for processing sensory information and the potential of HDMs in understanding brain function and dysfunction.This paper introduces hierarchical dynamic models (HDMs) and reviews a generic variational scheme for their inversion. HDMs are general models that subsume many parametric models for continuous data, including the general linear model for static data and generalized convolution models for nonlinear time-series analysis. The models are arranged in a hierarchy, where the output of one layer serves as input to another, enabling the modeling of data of arbitrary complexity. Crucially, all these models can be inverted using dynamic expectation maximization (DEM), a scheme that optimizes the conditional density on model states, parameters, and hyperparameters. This inversion can be formulated as a simple neural network, providing a useful metaphor for inference and learning in the brain. The paper discusses the inversion of HDMs, showing how it can be implemented using a variational approach. It highlights the importance of hierarchical structures in inducing empirical priors that provide structural and dynamic constraints. The inversion process is shown to be effective for a wide range of models, from variance component estimation in classical linear models to blind deconvolution. The paper also emphasizes the role of hierarchical forms in generating empirical priors and how these can be used to interpret conditional dependencies in data. The paper presents a detailed analysis of the mathematical formulation of HDMs, including their state-space equations and the probabilistic models they entail. It discusses the use of Gaussian assumptions for fluctuations and state-noise, which provide likelihoods and empirical priors. The paper also explores the implications of these models for understanding brain function, suggesting that the brain may use sophisticated algorithms similar to those used in scientific data analysis. The paper concludes by emphasizing the importance of hierarchical structures in the brain for processing sensory information and the potential of HDMs in understanding brain function and dysfunction.
Reach us at info@study.space
[slides and audio] Hierarchical Models in the Brain