Example of a simple learning model - TNU

Example of a simple learning model - TNU

Computational models for imaging analyses Zurich SPM Course February 14, 2014 Christoph Mathys What the brain is about What do our imaging methods measure? Brain activity. But when does the brain become active? When predictions have to be adjusted. So where do the brains predictions come from?

From a model. Feb 14, 2014 Model-based imaging, Zurich SPM Course, Christoph Mathys Page 2 What does this mean for neuroimaging? If brain activity reflects model updating, we need to understand what model is updated in what way to make sense of brain activity. Feb 14, 2014 Model-based imaging, Zurich SPM Course, Christoph Mathys

Page 3 The Bayesian brain and predictive coding Model-based prediction updating is described by Bayes theorem. the Bayesian brain Hermann von Helmholtz This is implemented by predictive coding. Feb 14, 2014 Model-based imaging, Zurich SPM Course, Christoph Mathys Page 4

Advantages of model-based imaging Model-based imaging permits us to infer the computational (predictive) mechanisms underlying neuronal activity. to localize such mechanisms. to compare different models. Feb 14, 2014 Model-based imaging, Zurich SPM Course, Christoph Mathys Page 5 How to build a model Fundamental ingredients: Prediction

Sensory input Hidden states Inference based on prediction errors Feb 14, 2014 Model-based imaging, Zurich SPM Course, Christoph Mathys Page 6

Example of a simple learning model Rescorla-Wagner learning: Learning rate Previous value (prediction) () =( 1) + ( ( ) ( 1) ) Prediction error () Inferred value of New input ( 1) Feb 14, 2014

() Model-based imaging, Zurich SPM Course, Christoph Mathys ( ) Page 7 From perception to action Agent Sensory input Inversion of perceptual generative model Inferred hidden states World

Generative process True hidden states Decision model Feb 14, 2014 Action

Model-based imaging, Zurich SPM Course, Christoph Mathys Page 8 From perception to action Agent Sensory input Inferred hidden states World

True hidden states Action In behavioral tasks, we observe actions (). How do we use them to infer beliefs ()? We introduce a decision model. Feb 14, 2014

Model-based imaging, Zurich SPM Course, Christoph Mathys Page 9 Example of a simple decision model Say 3 options A, B, and C have values , , and . Then we can translate these values into action probabilities via a softmax function: The parameter determines the sensitivity to value differences

=0.1 Feb 14, 2014 Model-based imaging, Zurich SPM Course, Christoph Mathys =0.6 Page 10 All the necessary ingredients Perceptual model (updates based on prediction errors) Value function (inferred state -> action value) Decision model (value -> action probability)

Feb 14, 2014 Model-based imaging, Zurich SPM Course, Christoph Mathys Page 11 Reinforcement learning example (ODoherty et al., 2003) ODoherty et al. (2003), Glscher et al. (2010) Feb 14, 2014 Model-based imaging, Zurich SPM Course, Christoph Mathys Page 12 Reinforcement learning example

Significant effects of prediction error with fixed learning rate ODoherty et al. (2003) Feb 14, 2014 Model-based imaging, Zurich SPM Course, Christoph Mathys Page 13 Bayesian models for the Bayesian brain Agent World Sensory input Inferred hidden states

True hidden states Action Includes uncertainty about hidden states.

I.e., beliefs have precisions. But how can we make them computationally tractable? Feb 14, 2014 Model-based imaging, Zurich SPM Course, Christoph Mathys Page 14 The hierarchical Gaussian filter (HGF): a computationally tractable model for individual learning under uncertainty State of the world

Model p(x3(k)) ~ N(x3(k-1),) ( 1) 3 ( ) 3 , Gaussian random walk

with constant step size Log-volatility x3 of tendency p(x3(k)) x3(k-1) p(x2(k)) ~ N(x2(k-1), exp(xx3+)) ( 1) 2

( 1) 1 ( ) 2 ( ) 1 Tendency x2 towards category 1 Stimulus

category x1 (0 or 1) Gaussian random walk with step size exp(xx3+) p(x2(k)) x2(k-1) Sigmoid transformation of x2 p(x1=1) = s(x2) p(x1=0) = 1-s(x2) 1 p(x1=1)

x2 0 Feb 14, 2014 Model-based imaging, Zurich SPM Course, Christoph Mathys Page 15 HGF: variational inversion and update equations Inversion proceeds by introducing a mean field approximation and fitting quadratic approximations to the resulting variational energies (Mathys et al., 2011). This leads to simple one-step update equations for the sufficient statistics (mean and precision) of the approximate Gaussian posteriors of the states . The updates of the means have the same structure as value updates in Rescorla-Wagner learning:

Prediction error ^ 1 1 Precisions determine learning rate Furthermore, the updates are precision-weighted prediction errors. Feb 14, 2014 Model-based imaging, Zurich SPM Course, Christoph Mathys Page 16 HGF: adaptive learning rate

Simulation: Feb 14, 2014 0.5, 2.2, 1.4 Model-based imaging, Zurich SPM Course, Christoph Mathys Page 17 Individual model-based regressors Uncertainty-weighted prediction error Feb 14, 2014 Model-based imaging, Zurich SPM Course, Christoph Mathys Page 18

Example: Iglesias et al. (2013) Feb 14, 2014 Model-based imaging, Zurich SPM Course, Christoph Mathys Page 19 Example: Iglesias et al. (2013) Model comparison: Feb 14, 2014 Model-based imaging, Zurich SPM Course, Christoph Mathys Page 20

Example: Iglesias et al. (2013) Feb 14, 2014 Model-based imaging, Zurich SPM Course, Christoph Mathys Page 21 Example: Iglesias et al. (2013) Feb 14, 2014 Model-based imaging, Zurich SPM Course, Christoph Mathys Page 22 Example: Iglesias et al. (2013)

Feb 14, 2014 Model-based imaging, Zurich SPM Course, Christoph Mathys Page 23 How to estimate and compare models: the HGF Toolbox Available at http://www.tranlsationalneuromodeling.org/tapas Start with README and tutorial there Modular, extensible Matlab-based Feb 14, 2014

Model-based imaging, Zurich SPM Course, Christoph Mathys Page 24 How its done in SPM Feb 14, 2014 Model-based imaging, Zurich SPM Course, Christoph Mathys Page 25 How its done in SPM Feb 14, 2014 Model-based imaging, Zurich SPM Course, Christoph Mathys

Page 26 How its done in SPM Feb 14, 2014 Model-based imaging, Zurich SPM Course, Christoph Mathys Page 27 How its done in SPM Feb 14, 2014 Model-based imaging, Zurich SPM Course, Christoph Mathys Page 30

How its done in SPM Feb 14, 2014 Model-based imaging, Zurich SPM Course, Christoph Mathys Page 31 Take home The brain is an organ whose job is prediction. To make its predictions, it needs a model. Model-based imaging infers the model at work in the brain. It enables inference on mechanisms, localization of mechanisms, and model comparison. Agent Sensory input Inferred

hidden states World True hidden states Action Feb 14, 2014 Model-based imaging, Zurich SPM Course, Christoph Mathys

Page 32 Thank you Feb 14, 2014 Model-based imaging, Zurich SPM Course, Christoph Mathys Page 33

Recently Viewed Presentations

  • Fundamentals of Human Resource Management 11e Chapter 11

    Fundamentals of Human Resource Management 11e Chapter 11

    Jobs placed in grades to compare their descriptions to the benchmarked jobs. Look for a common denominator (skills, knowledge, responsibility). Point method. Jobs are rated and allocated points on several criteria. Jobs with similar point totals are placed in similar...
  • Colorado Pain Society Update Pelvic Pain

    Colorado Pain Society Update Pelvic Pain

    Bladder Pain Syndrome/IC. The most successful rx. documented in RCCT: Pelvic Physical Therapy. WHY? Fitzgerald, MP et al, Randomized multicenter clinical trial of myofascial physical therapy in women with IC/PBS and pelvic floor tenderness. 2012.
  • Software Evolution Through Program Transformations: An ...

    Software Evolution Through Program Transformations: An ...

    Software Evolution Through Program Transformations: An Experience Report Kostas Kontogiannis, Johannes Martin Kenny Wong Richard Gregory Hausi Muller
  • HIGHLAND NUMERACY CPD SUPPORT PACKAGE EARLY LEVEL Please

    HIGHLAND NUMERACY CPD SUPPORT PACKAGE EARLY LEVEL Please

    Early Level information Count Me In Too NZ maths K-5 Maths Teaching Resources Solon City Schools Illuminations Useful websites to explore These activities can be used to support the following domains at 1 *, 2 ** and 3*** Number word...
  • The Liability & Risk behind International Listings Copyright

    The Liability & Risk behind International Listings Copyright

    When Realtors® prepare opinions of real property value or price they must: 1) be knowledgeable about the type of property being valued, 2) have access to the information and resources necessary to formulate. an accurate opinion, and. 3) be familiar...
  • Developmental Psychology - Psychology Home Page

    Developmental Psychology - Psychology Home Page

    Developmental Psychology Cognitive development in childhood (Ages 2 - 6 years) Piaget's Preoperational Child Preoperational Intuitive thought Symbolic Function Piaget's Preoperational Child Static Reasoning Focus on appearance Conservation Vygotsky's Theory Uses a approach Vygotsky's Theory Language use Use in the...
  • Gibbs Free Energy - Mesa Community College

    Gibbs Free Energy - Mesa Community College

    Gibbs Free Energy, G Multiply through by -T -T∆Suniv = ∆Hsys - T∆Ssys -T∆Suniv = change in Gibbs free energy for the system = ∆Gsystem Under standard conditions — ∆Gosys = ∆Hosys - T∆Sosys ∆Go = ∆Ho - T∆So Gibbs...
  • After Euridice Claudio Monteverdi (1567-1643) 1567-Born in Cremona

    After Euridice Claudio Monteverdi (1567-1643) 1567-Born in Cremona

    (except in Venice) New Monodic Idioms with Basso Continuo Strophic Aria Good for Poetry Standard formulas and patterns Romanesca Chaccone Passacaglia Concertato (NAWM 56) piece for voice with instruments Independent parts Cantata (NAWM 57) "To be Sung" Summary Solo Singing...