- About Us
- Affiliates and Liaisons
Colloquia occur: Selected Mondays at 4:00 pm - 5:00 pm - Room PY 101.
Colloquia titles will be posted as they become available.
Organizer: Rob Goldstone
Jan 23, 2017: Nathanial Daw, Princeton University
PY 101, 4 - 5 p.m.
Title: The neurocomputational substrates of action evaluation: habits and beyond
Abstract: Many decision problems -- especially those involving multi-step tasks like mazes or chess -- imply extremely difficult computations to anticipate the values of options. Because of the complexity of exactly computing outcomes in expectation over a series of future steps, it is believed that human and animal brains use a range of computational shortcuts to approximate action evaluation. In particular, pre-computing (via model-free reinforcement learning) and storing action preferences has long been taken as a formal model of the nigrostriatal dopamine system and its role in habits -- both for efficient, healthy behavior and in disorders such as drug abuse. I review evidence that humans and animals trade off deliberative ("model based") and habitual strategies to action evaluation, and that disorders of compulsion are associated with pathological imbalance between these strategies. I will then turn to in-progress research investigating additional possibilities outside this dichotomy. These include partial pre-computation by storing decision variable precursors, and nonparametric estimation based on memories of individual episodes. Options like these enrich the notions of habit and deliberation, and also further complicate attempts to build a resource-rational account of self-control via weighing the costs and benefits of different decision variable computations.