Bayesian Learning in Unstable Settings: Experimental Evidence
We study learning in a bandit task in which the outcome probabilities of six arms switch (“jump”) over time. In the experiment, optimal Bayesian learning tracks the jumps by discovering the probability of a jump or direct jump detection. Although more complex than the natural alternative of learning, through adaptive expectations, when combined with a partially myopic decision rule, Bayesian learning better matches the behavior observed in the lab. This result suggests complexity does not always hamper rational learning. We hypothesize that the latter can emerge if several conditions are met: (i) implementing Bayesian learning, rather than boundedly rational alternatives, markedly improves economic performance (as in our task and, arguably, in many financial contexts), and subjects are provided with significant monetary incentives; (ii) the task at hand is ecologically relevant; (iii) agents are “nudged” into paying attention to the right things
Swiss Finance Institute Research Paper No. 10 – 28 — Most recent version: February 2012
Risk, Unexpected Uncertainty, and Estimation Uncertainty: Bayesian Learning in Unstable Settings
The ability of humans to learn like Bayesians implies that they perceive, at a minimum, three levels of uncertainty: risk, which reflects imperfect foresight even after everything is learned; (parameter) estimation uncertainty, i.e., uncertainty about outcome probabilities; and unexpected uncertainty, or sudden changes in the probabilities. We describe how these levels of uncertainty evolve in a six-arm restless bandit task in which human choices reliably reflect Bayesian updating, and how their evolution changes the learning rate. We then zoom in on estimation uncertainty. The ability to sense estimation uncertainty (also known as ambiguity) is a virtue because, besides allowing one to apply Bayesian learning, it may guide more effective exploration; but aversion to estimation uncertainty may be maladaptive. Here, we show that participant choices reflected aversion to estimation uncertainty. We discuss how past imaging studies foreshadowed the ability of humans to distinguish the different notions of uncertainty. Also, we document that the ability of participants to do such distinction relies on sufficient revelation of the payoff generating model. When we induced structural uncertainty, participants did not gain awareness of the jumps in our restless bandit task, and fell back to model-free reinforcement learning.
With P. Bossaerts — PLoS CB, 2011
View online version Download
Estimation Uncertainty, Unexpected Uncertainty and Action
Uncertainty as Modulators of Decision Making: How Adaptive?
Using simulated data, we study the performances of model-free and model-based approaches to learning in a restless bandit problem.
