A second question was whether our participants were indeed engaging in hierarchical learning and updating their learning rate dynamically, as our Bayesian model assumed, or used a simpler learning mechanism. To clarify this, we added two more models to our comparison set. The models were a Bayesian model

with reduced hierarchical depth (HGF3) in which the third level was eliminated from the hierarchy, and a standard Rescorla-Wagner (RL) model with a fixed learning rate. Finally, we implemented a RL model with click here dynamic learning rate (Sutton, 1992) that was recommended by one of the reviewers as a non-Bayesian alternative to HGF1. See the Supplemental Experimental Procedures section C (available online) for more information on these models. Comparing these five models, we found that, across studies, HGF1 was the superior model in 86 out of our 118 participants. Examining each study separately, random effects BMS yielded posterior model probabilities of 84% (behavioral study), 74% (first fMRI study), and 72% (second fMRI study) for HGF1, which was five to ten times higher than for the next best model in

each case (Table S1). As a consequence, in each study, the exceedance probability in favor of HGF1 (i.e., the probability that its posterior probability was higher than that of any other model considered) (Stephan et al., 2009) was indistinguishable from 100%. These results provide strong evidence that our participants selleck did learn the task-relevant conditional probabilities of visual stimuli (instead of predicting the incidental reward) and were capable of updating their learning rate dynamically. We next examined the estimates of the free parameters (κ, ϑ, ζ) from the winning model (Table S2). These estimates were comparable across the three studies,

as demonstrated by ANOVA: none of the model parameters showed significant differences across studies (κ: F(2,115) = 1.04, p = 0.358; ϑ: F(2,115) = 0.91, p = 0.405; ζ: F(2,115) = 2.98, p = 0.055). Additionally, we used multiple regression to evaluate how well our model explained subjects’ behavior (percentage of correct responses). This quantified model performance Carnitine palmitoyltransferase II in terms of variance explained, complementary to the relative model comparison by BMS above. This analysis showed that the linear combination of the three model parameters predicted subjects’ task performance well (behavioral study: R2 = 0.64, F(3,42) = 25.3, p < 0.001; first fMRI study: R2 = 0.59, F(3,41) = 20.1, p < 0.001; second fMRI study: R2 = 0.63, F(3,23) = 13.2, p < 0.001). As detailed in the Experimental Procedures section, our fMRI analysis focused on precision-weighted PEs and uncertainty estimates across the hierarchical levels of the HGF.