This study investigates the neural substrates underlying exploratory decisions in humans. Using a gambling task, the researchers found that human subjects' decisions can be characterized by a computationally well-regarded strategy for addressing the explore/exploit dilemma. Functional magnetic resonance imaging (fMRI) showed that the frontopolar cortex and intraparietal sulcus are preferentially active during exploratory decisions, while the striatum and ventromedial prefrontal cortex are involved in value-based exploitative decision making. The results suggest a model of action selection under uncertainty that involves switching between exploratory and exploitative behavioural modes. The study used a 'four-armed bandit' task, where participants repeatedly chose between four slot machines with varying payoffs. The researchers compared three distinct reinforcement learning (RL) models to subjects' behavioural choices and found strong evidence for value-sensitive (softmax) over undirected (ε-greedy) exploration. The findings were not an artefact of assumptions about free parameters. The study also identified brain regions in which neural activity was significantly correlated with the model's internal signals. The medial orbitofrontal cortex was correlated with the magnitude of the obtained payoff, while the ventromedial prefrontal cortex was correlated with the probability assigned by the computational model to the subject's choice of slot. Additionally, the frontopolar cortex showed increased activation during exploratory compared with exploitative trials, and the intraparietal sulcus showed similar activation. The study found that the prefrontal cortex is the principal cortical region implicated in behavioural control, and the signal observed in the anterior frontopolar cortex could reflect a control mechanism facilitating the switching of behavioural strategies between exploratory and exploitative modes. The anterior intraparietal sulcus also showed differential activation during exploratory trials. The results have important implications for both computational and neural accounts of action selection. The findings suggest that exploration is accomplished by overriding an exploitative tendency, but are troubling for accounts that more tightly entangle exploration and exploitation. The study also highlights the importance of exploration in the acquisition of adaptive behaviour in changing environments.This study investigates the neural substrates underlying exploratory decisions in humans. Using a gambling task, the researchers found that human subjects' decisions can be characterized by a computationally well-regarded strategy for addressing the explore/exploit dilemma. Functional magnetic resonance imaging (fMRI) showed that the frontopolar cortex and intraparietal sulcus are preferentially active during exploratory decisions, while the striatum and ventromedial prefrontal cortex are involved in value-based exploitative decision making. The results suggest a model of action selection under uncertainty that involves switching between exploratory and exploitative behavioural modes. The study used a 'four-armed bandit' task, where participants repeatedly chose between four slot machines with varying payoffs. The researchers compared three distinct reinforcement learning (RL) models to subjects' behavioural choices and found strong evidence for value-sensitive (softmax) over undirected (ε-greedy) exploration. The findings were not an artefact of assumptions about free parameters. The study also identified brain regions in which neural activity was significantly correlated with the model's internal signals. The medial orbitofrontal cortex was correlated with the magnitude of the obtained payoff, while the ventromedial prefrontal cortex was correlated with the probability assigned by the computational model to the subject's choice of slot. Additionally, the frontopolar cortex showed increased activation during exploratory compared with exploitative trials, and the intraparietal sulcus showed similar activation. The study found that the prefrontal cortex is the principal cortical region implicated in behavioural control, and the signal observed in the anterior frontopolar cortex could reflect a control mechanism facilitating the switching of behavioural strategies between exploratory and exploitative modes. The anterior intraparietal sulcus also showed differential activation during exploratory trials. The results have important implications for both computational and neural accounts of action selection. The findings suggest that exploration is accomplished by overriding an exploitative tendency, but are troubling for accounts that more tightly entangle exploration and exploitation. The study also highlights the importance of exploration in the acquisition of adaptive behaviour in changing environments.