at left poke, bottom if at rightChoicerepeat choiceCorrectrepeat correct choiceOutcomerepeat rewarded choiceTransitionrepeat choice followed by common transitionTransition-outcome interactionrepeat choice followed by rewarded common and non-rewarded rare transitionsRL Model Variablesrreward (0 or 1)cchoice taken at first step (top or bottom poke)c′choice not taken at first step (top or bottom poke)ssecond-step state (left-active or right-active)s′state not reached at second step (left-active or right-active)Qmf(c)model-free action value for choice cQmo(c,st−1)motor-level model-free action value for choice c following second-step state st−1Qmb(c)model-based value of choice cV(s)value of state sP(s|c)estimated transition probability of reaching state s after choice cc‾choice historym‾(st−1)motor action history (i.e., choice history following second-step state st−1)RL Model ParametersαQvalue learning ratefQvalue forgetting rateλeligibility trace parameterαTtransition learning ratefTtransition forgetting rateαclearning rate for choice perseverationαmlearning rate for motor-level perseverationGmfmodel-free action value weightGmomotor-level model-free action value weightGmbmodel-based action value weightBcchoice bias (top/bottom)Brrotational bias (clockwise/counterclockwise)Pcchoice perseveration strengthPmmotor-level perseveration strength
