The Anterior Cingulate Cortex Predicts Future States to Mediate Model-Based Action Selection.
- Authors
- Akam, Thomas; Rodrigues-Vaz, Ines; Marcelo, Ivo; Zhang, Xiangyu; Pereira, Michael; Oliveira, Rodrigo Freire; Dayan, Peter; Costa, Rui M
- Year
- 2021
- Journal
- Neuron
- PMID
- 33152266
- DOI
- 10.1016/j.neuron.2020.10.013
- PMCID
- PMC7837117
Behavioral control is not unitary. It comprises parallel systems, model based and model free, that respectively generate flexible and habitual behaviors. Model-based decisions use predictions of the specific consequences of actions, but how these are implemented in the brain is poorly understood. We used calcium imaging and optogenetics in a sequential decision task for mice to show that the anterior cingulate cortex (ACC) predicts the state that actions will lead to, not simply whether they are good or bad, and monitors whether outcomes match these predictions. ACC represents the complete state space of the task, with reward signals that depend strongly on the state where reward is obtained but minimally on the preceding choice. Accordingly, ACC is necessary only for updating model-based strategies, not for basic reward-driven action reinforcement. These results reveal that ACC is a critical node in model-based control, with a specific role in predicting future states given chosen actions.
Two-Step Task with Transition Probability Reversals(A) Diagram of apparatus and trial events.(B) State diagram of task. Reward and transition probabilities are indicated for one of the six possible block types.(C) Block structure; left side shows the three possible states of the reward probabilities, right side shows the two possible states of the transition probabilities.(D) Example session. Top panel: exponential moving average (tau = 8 trials) of choices. Horizontal gray bars show blocks, with correct choice (top, bottom, or neutral) indicated by y position of bars. Middle panel: reward probabilities in left-active (red) and right-active (blue) states. Bottom panel: transition probabilities linking first-step actions (top, bottom pokes) to second-step states (left/right active).(E) Choice probability trajectories around reversals. Pale blue line, average trajectory; dark blue line, exponential fit; shaded area, cross-subject SD. Left panel: reversals in reward probability; right panel: reversals in transition probabilities.(F) Second step reaction times following common and rare transitions (i.e., the time between the first-step choice and side poke entry). ∗∗∗p < 0.001 Error bars show cross-subject SEM.
Stay Probability and Logistic Regression Analyses(A–C) Mouse behavior. (A) Stay probability analysis showing the fraction of trials the subject repeated the same choice following each combination of trial outcome (rewarded [1] or not [0]) and transition (common [C] or rare [R]). Error bars show cross-subject SEM. (B) Logistic regression model fit predicting choice as a function of the previous trial’s events. Predictor loadings plotted are outcome (repeat choices following rewards), transition (repeat choices following common transitions), and transition-outcome interaction (repeat choices following rewarded common transition trials and non-rewarded rare transition trials). Error bars indicate 95% confidence intervals on the population mean, dots indicate maximum a posteriori (MAP) subject fits. (C) Lagged logistic regression model predicting choice as a function of events over the previous 12 trials. Predictors are as in (B).(D–F) As (A)–(C) but for data simulated from a model-free RL agent with forgetting and multi-trial perseveration.(G–I) As (A)–(C) but for data simulated from a model-based RL agent with forgetting and multi-trial perseveration.(J–L) As (A)–(C) but for data simulated from the best fitting RL model found by model comparison.Parameters for all RL model simulations were obtained by fits of the RL models to the mouse behavioral data.
Two-Step ACC Calcium Imaging(A) Example GRIN lens placement in ACC.(B) Fluorescence signal from a neuronal region of interest (ROI) identified by CNMF-E (top panel, blue) and fitted trace (orange) due to the inferred deconvolved neuronal activity (bottom panel).(C) Histogram showing the distribution of average event rates across the population of recorded neurons. Events were defined as any video frame on which the inferred activity was non-zero.(D) Average trial aligned activity for all recorded neurons, sorted by the time of peak activity. No normalization was applied to the activity. The gray bars under (D), (E), and (G) between choice and outcome indicate the time period that was warped to align trials of different duration.(E) Regression analysis predicting activity on each trial from a set of predictors coding the choice (top or bottom), second step (left or right), outcome (rewarded or not) that occurred in each trial, and their interactions. Lines show the population coefficient of partial determination (CPD) as a function of time relative to trial events. Circles indicate where CPD is significantly higher than expected by chance, assessed by permutation test with Benjamini-Hochberg correction for comparison at multiple time points.(F) Representation of the second-step state before and after the trial outcome. Points show second-step predictor loadings for individual neurons at a time point halfway between choice and outcome (x axis) and a time point 250 ms after trial outcome (y axis).(G) Time course of pre- and post-outcome representations of second-step state, obtained by projecting the second step predictor loadings at each time point onto the pre- and post-outcome second-step representations. The red and blue triangles indicate the time points used to define the projection vectors.(H) Representation of trial outcomes (reward or not) obtained at the left and right poke. Points show predictor loadings for individual neurons 250 ms after trial outcome in a regression analysis in which outcomes at the left and right poke were coded by separate predictors. The regression analysis was identical to that shown in (E) except that the outcome and second-step x outcome predictors were replaced by left outcome and right outcome predictors, which coded reward/non-reward in trials that reached the left or right second-step state, respectively.
ACC Represents the Full State-Action Space(A–C) Projection of the average population activity for different trial types into the low-dimensional space that captures the most variance between trial types. Trial types were defined by the eight combinations of choice, second step, and trial outcome. Letters on the trajectories indicate the trajectory start (S; 1,000 ms before choice), the choice (C), outcome (O), and trajectory end (E; 1,000 ms after outcome). (A) Three-dimensional plot showing projections onto first three principal components. (B) Projection onto PC1 and PC2, which represent second-step and choice, respectively. (C) Projection onto PC4 and PC5, which differentiate trial outcomes.(D and E) Decoding analysis assessing how accurately ACC population activity differentiates between different locations in task’s state-action space. (D) Diagram showing the ten different locations (red dots) in the tasks state-action space used in the decoding analysis. (E) Confusion matrix showing the cross-validated probability of decoding each location given the actual location the activity was from.
ACC Represents Model-Based Decision Variables(A) Regression analysis predicting neuronal activity as a function of events in the current trial (top panel) and their interaction with the transition probabilities (trans. probs.) mapping the first-step choice to second-step (sec. step) states (bottom panel) for a subset of sessions with sufficient coverage of both states of the transition probabilities. Predictors plotted in top panels are as in Figure 3E. Predictors plotted in the bottom panel are transition probabilities (which of the two possible states the transition probabilities are in; see Figure 1C), common/rare transition (whether the transition on the current trial was common or rare, i.e., the interaction of the transition on the current trial [e.g., top → right] with the state of the transition probabilities), choice × trans. probs. (the choice in the current trial interacted with the state of the transition probabilities, i.e., the predicted second-step state given the current choice), and sec. step × trans. probs. (the second-step state reached on the current trial interacted with the state of the transition probabilities, i.e., the action which commonly leads to the second-step state reached). Predictors shown in top and bottom panels of (A) were run as a single regression but plotted on separate axes for clarity. The gray bars between choice and outcome indicate the time period that was warped to align trials of different length. Circles indicate where CPD is significantly higher than expected by chance, assessed by permutation test with Benjamini-Hochberg correction for comparison at multiple time points.(B) Regression analysis predicting neuronal activity as a function of events on the current trial (top panel) and their interaction with the reward probabilities in the second-step states (bottom panel) for a subset of sessions with sufficient coverage of different states of the reward probabilities. Predictors plotted in the bottom panel are reward probabilities (which of the three possible states the transition probabilities are in; see Figure 1C), transition × reward probs. (interaction of the transition in the current trial with the state of the reward probabilities), choice × reward probs. (the choice in the current trial interacted with the state of the reward probabilities), and sec. step × trans. probs. (the second-step state reached in the current trial interacted with the state of the rewarded probabilities, i.e., the expected outcome [rewarded or not]. Predictors shown in top and bottom panels of (B) were run as a single regression but plotted on separate axes for clarity.
Optogenetic Inhibition of ACC in the Two-Step Task(A) LED implant (left) and diagram showing implant mounted on head (right); red dots on diagram indicate location of virus injections.(B) Normalized firing rate for significantly inhibited cells over 5 s illumination; dark blue line, median; shaded area, 25th to 75th percentiles.(C) Timing of stimulation relative to trial events. Stimulation was delivered from trial outcome to subsequent choice.(D) Logistic regression analysis of ACC inhibition data showing loadings for the outcome, transition, and transition-outcome interaction predictors for choices made on stimulated (red) and non-stimulated (blue) trials. ∗∗Bonferroni-corrected p < 0.01 between stimulated and non-stimulated trials. Error bars indicate 95% confidence intervals on the population mean, dots indicate maximum a posteriori (MAP) subject fits.(E) Correlation across subjects between the strength of model-based influence on choice (assessed using the RL model’s model-based weight parameter, Gmb) and the effect of optogenetic inhibition on the logistic regression model’s transition predictor.
No entities extracted from this document yet.
No uploaded files.
In this knowledge base
| Title | Year | PMID |
|---|---|---|
| Predicting Alcohol-Related Memory Problems in Older Adults: A Machine Learning Study with Multi-Domain Features. | 2023 | 37232664 |
External
| Title | Authors | Journal | Year | Link |
|---|---|---|---|---|
| An amygdala-cortical circuit for encoding generalized fear memories. | Vincent CJ et al. | — | 2026 | → |
| Anterior cingulate neurons combine outcome monitoring of past decisions with ongoing movement signals. | Oesch LT et al. | — | 2026 | → |
| Bridging neurobiological mechanisms and translational applications in cognitive science: inspirations from the MCCS-NYUAD meeting 2025. | Nami M et al. | — | 2026 | → |
| Delay discounting correlates with depression but does not predict relapse after antidepressant discontinuation. | Elad D et al. | — | 2026 | → |
| Dynamics of ACC excitatory neurons during reversal learning: Implications for cognitive flexibility. | Zhang M et al. | — | 2026 | → |
| Evidence accumulation from experience and observation in the cingulate cortex. | Chen R et al. | — | 2026 | → |
| Fentanyl reinforcement history has sex-specific effects on multi-step decision-making. | Garr E et al. | — | 2026 | → |
| Looking deeper into the algorithms underlying human planning. | Kuperwajs I et al. | — | 2026 | → |
| Neuronal Features of Visual Attention in the Mouse Superior Colliculus Depend on Learned Behavioral Relevance. | Wang L et al. | — | 2026 | → |
| Orbitofrontal Thickness and Network Associations as Transdiagnostic Signature of Amotivation Along the Bipolar-Schizophrenia Spectrum. | Franz M et al. | — | 2026 | → |
| Time-dependent deployment of medial prefrontal cortical representations in male mice. | Lopez-Yepez JS et al. | — | 2026 | → |
| An Open-Source Joystick Platform for Investigating Forelimb Motor Control, Auditory-Motor Integration, and Value-Based Decision-Making in Head-Fixed Mice. | Linares-García I et al. | — | 2025 | → |
| A selective top-down pathway from anterior cingulate cortex embeds a representation of saliency in hippocampus | Li Y et al. | — | 2025 | — |
| Context-dependent activation of V1 parvalbumin interneurons enhances visual discrimination. | Kukovska L et al. | — | 2025 | → |
| Deep neural network models of emotion understanding. | Thornton MA | — | 2025 | → |
| Discovering cognitive strategies with tiny recurrent neural networks. | Ji-An L et al. | — | 2025 | → |
| Distributed Representations for Cognitive Control in Frontal Medial Cortex. | Colin TR et al. | — | 2025 | → |
| Experience-Dependent Plasticity to Visual Sequences in Mouse Anterior Cingulate Cortex Reflects Familiarity. | Malci A et al. | — | 2025 | → |
| Modeling natural neural networks of decision making with artificial neural networks. | Funamizu A et al. | — | 2025 | → |
| Optogenetic silencing of hippocampal inputs to the retrosplenial cortex causes a prolonged disruption of spatial working memory. | Pinto-Correia B et al. | — | 2025 | → |
| Prefrontal transthalamic uncertainty processing drives flexible switching. | Lam NH et al. | — | 2025 | → |
| Schemas, reinforcement learning and the medial prefrontal cortex. | Bein O et al. | — | 2025 | → |
| The distinct functions of working memory and intelligence in model-based and model-free reinforcement learning. | Yang C et al. | — | 2025 | → |
| The neural basis for uncertainty processing in hierarchical decision making. | Wang MB et al. | — | 2025 | → |
| A cellular basis for mapping behavioural structure. | El-Gaby M et al. | — | 2024 | → |
| Active reinforcement learning versus action bias and hysteresis: control with a mixture of experts and nonexperts. | Colas JT et al. | — | 2024 | → |
| An Active-Inference Approach to Second-Person Neuroscience. | Lehmann K et al. | — | 2024 | → |
| A normative framework dissociates need and motivation in hypothalamic neurons. | Kim KS et al. | — | 2024 | → |
| Anterior cingulate cortex lesions impair multiple facets of task engagement not mediated by dorsomedial striatum neuron firing. | Vázquez D et al. | — | 2024 | → |
| Anterior cingulate cortex provides the neural substrates for feedback-driven iteration of decision and value representation. | Chen W et al. | — | 2024 | → |
| A stress-sensitive frontostriatal circuit supporting effortful reward-seeking behavior. | Fetcho RN et al. | — | 2024 | → |
| Bad habits-good goals? Meta-analysis and translation of the habit construct to alcoholism. | Giannone F et al. | — | 2024 | → |
| Divergent subregional information processing in mouse prefrontal cortex during working memory. | Sonneborn A et al. | — | 2024 | → |
| Dopamine-independent effect of rewards on choices through hidden-state inference. | Blanco-Pozo M et al. | — | 2024 | → |
| Dynamic ensemble balance in direct- and indirect-pathway striatal projection neurons underlying decision-related action selection. | Tang S et al. | — | 2024 | → |
| Effects of optogenetic silencing the anterior cingulate cortex in a delayed non-match to trajectory task. | Cruz AS et al. | — | 2024 | → |
| Global neural encoding of behavioral strategies in mice during perceptual decision-making task with two different sensory patterns. | Wang S et al. | — | 2024 | → |
| Hierarchical control over foraging behavior by anterior cingulate cortex. | Alejandro RJ et al. | — | 2024 | → |
| Lights, fiber, action! A primer on in vivo fiber photometry. | Simpson EH et al. | — | 2024 | → |
| Neural activity ramps in frontal cortex signal extended motivation during learning. | Regalado JM et al. | — | 2024 | → |
| Prediction-error signals in anterior cingulate cortex drive task-switching. | Cole N et al. | — | 2024 | → |
| Strategic stabilization of arousal boosts sustained attention. | de Gee JW et al. | — | 2024 | → |
| The successor representation subserves hierarchical abstraction for goal-directed behavior. | Wientjes S et al. | — | 2024 | → |
| Theta and alpha oscillatory signatures of auditory sensory and cognitive loads during complex listening. | Brilliant et al. | — | 2024 | → |
| Understanding dual process cognition via the minimum description length principle. | Moskovitz T et al. | — | 2024 | → |
| Volitional mental absorption in meditation: Toward a scientific understanding of advanced concentrative absorption meditation and the case of jhana. | Yang WFZ et al. | — | 2024 | → |
| A neural substrate of sex-dependent modulation of motivation. | Cox J et al. | — | 2023 | → |
| Control of sustained attention and impulsivity by G<sub>q</sub>-protein signalling in parvalbumin interneurons of the anterior cingulate cortex. | Jendryka MM et al. | — | 2023 | → |
| Differential processing of decision information in subregions of rodent medial prefrontal cortex. | Diehl GW et al. | — | 2023 | → |
| Effects of the number of competing responses on neural signatures of pre- and post-response conflict. | Dumsky J et al. | — | 2023 | → |
| Predicting Alcohol-Related Memory Problems in Older Adults: A Machine Learning Study with Multi-Domain Features. | Kamarajan C et al. | — | 2023 | → |
| Reward-Mediated, Model-Free Reinforcement-Learning Mechanisms in Pavlovian and Instrumental Tasks Are Related. | Moin Afshar N et al. | — | 2023 | → |
| State-transition-free reinforcement learning in chimpanzees (Pan troglodytes). | Sato Y et al. | — | 2023 | → |
| The Anterior Cingulate Cortex Promotes Long-Term Auditory Cortical Responses through an Indirect Pathway via the Rhinal Cortex in Mice. | Liang Y et al. | — | 2023 | → |
| The association of spouse interactions and emotional learning in interference related to chronic back pain. | Nees F et al. | — | 2023 | → |
| The human cerebellum in reward anticipation and outcome processing: An activation likelihood estimation meta-analysis. | Kruithof ES et al. | — | 2023 | → |
| The relation between implicit statistical learning and proactivity as revealed by EEG. | Sznabel D et al. | — | 2023 | → |
| Theta oscillations in anterior cingulate cortex and orbitofrontal cortex differentially modulate accuracy and speed in flexible reward learning. | Ye T et al. | — | 2023 | → |
| Training-induced circuit-specific excitatory synaptogenesis in mice is required for effort control. | Ulloa Severino FP et al. | — | 2023 | → |
| A Computational Model of Hopelessness and Active-Escape Bias in Suicidality. | Karvelis P et al. | — | 2022 | → |
| Calcium activity is a degraded estimate of spikes. | Hart EE et al. | — | 2022 | → |
| Cingulate-motor circuits update rule representations for sequential choice decisions. | Takeuchi D et al. | — | 2022 | → |
| Computational models of adaptive behavior and prefrontal cortex. | Soltani A et al. | — | 2022 | → |
| Decision Making as a Learned Skill in Mice and Humans. | Posner MI et al. | — | 2022 | → |
| Explicit knowledge of task structure is a primary determinant of human model-based action. | Castro-Rodrigues P et al. | — | 2022 | → |
| Metastable dynamics of neural circuits and networks. | Brinkman BAW et al. | — | 2022 | → |
| Open-source, Python-based, hardware and software for controlling behavioural neuroscience experiments. | Akam T et al. | — | 2022 | → |
| Planning in the brain. | Mattar MG et al. | — | 2022 | → |
| Prospective and retrospective values integrated in frontal cortex drive predictive choice. | Hamaguchi K et al. | — | 2022 | → |
| Rat Anterior Cingulate Cortex Continuously Signals Decision Variables in a Patch Foraging Task. | Kane GA et al. | — | 2022 | → |
| Spatial and Temporal Distribution of Information Processing in the Human Dorsal Anterior Cingulate Cortex. | Keogh C et al. | — | 2022 | → |
| The interaction between depression diagnosis and BMI is related to altered activation pattern in the right inferior frontal gyrus and anterior cingulate cortex during food anticipation. | Manelis A et al. | — | 2022 | → |
| The role of reinforcement learning and value-based decision-making frameworks in understanding food choice and eating behaviors. | Pearce AL et al. | — | 2022 | → |
| A low-cost open-source 5-choice operant box system optimized for electrophysiology and optophysiology in mice. | Kapanaiah SKT et al. | — | 2021 | → |
| Alzheimer's Disease and Empathic Abilities: The Proposed Role of the Cingulate Cortex. | Ávila-Villanueva M et al. | — | 2021 | → |
| Distinct roles for dopamine clearance mechanisms in regulating behavioral flexibility. | Korn C et al. | — | 2021 | → |
| The anterior cingulate cortex directs exploration of alternative strategies. | Tervo DGR et al. | — | 2021 | → |