Dopamine

There has been a lot of confusing information about dopamine. I finally found a literature review-style article, and here is what I learned.

Source:

Dopamine: Everything you need to know

This is my attempt to give a more or less complete, up-to-the-minute account of what we know about dopamine. It’s long, but it’s thorough.

Erring Towards AnswersFrazer Mawson

What is Dopamine:

• Dopamine is a signaling molecule in the brain
• Volume transmission:
  • As opposed to the typical synaptic transmission (directly between upstream and downstream neurons), dopamine is mostly distributed via volume transmission, which is essentially where they release a ton of dopamine into the space outside the cell, without targeting any specific downstream neurons or receptor sites in particular. 
  • By being sprayed far and wide, dopamine may be able to identify and strengthen certain recently active neural pathways associated with behaviours that preceded some kind of a reward.
• It is primarily produced by two populations of neurons found in the SN and the VTA
  • The first population in the SN connects mostly to the dorsal (i.e. top part of the) striatum. Known as the nigrostriatal pathway, this pathway is mostly involved in action-selection, habits, and goal-directed behaviour.
  • The second population in the VTA connects mostly to the ventral (i.e. bottom part of the) striatum, as well as various parts of the prefrontal cortex (PFC). Known as the mesocorticolimbic pathway, this pathway is mostly involved in movement, motivation, and reinforcement learning.
• There are two main classes of dopamine receptors: D1-like receptors and D2-like receptors.

What does Dopamine do:

• Contrary to common belief, dopamine level isn't directly correlated with pleasure.
• Dopamine is responsible for learning and motivation (lots of corollaries with Reinforcement Learning):
  • Learning:
    • Dopamine neural activity is correlated with the unexpectedness of reward; this is the Reward Prediction Error (RPE) theory.
    • Example: a monkey learns a juice is delivered 20 seconds after a cue.
      • Before the learning, dopamine neurons fire at the time of the reward.
      • After the learning, dopamine neurons no longer fire at the time of the reward instead the respond earlier, at the time of the cue.
      • If the cue was presented but reward is withheld or below baseline, the firing rates below baseline.
      • If reward exceeded expectation or was unexpected (e.g. occur without cue), firing rates were enhanced.
      • If the reward was exactly as expected, firing rates upon receipt of the reward stay more or less steady (neither a spike nor a dip).
    • The unexpectedness is the reduction error that later causes the brain to update its model of the world.
  • Motivations
    • "These researchers showed that dopamine levels correlated strongly with instantaneous reward value (or ‘value,’ as they call it), defined as the expected reward minus the expected time needed to receive it. It has, therefore, been suggested that motivation levels are essentially equal to "value"."
    • Another interesting finding was that as the subjects received cues suggesting that rewards were either larger, closer, or more certain than previously expected, the dopamine ramp shifted upwards, representing increased motivation.
• Dopamine appears to encode two things: First, the expected value of a reward, discounted by the expected time needed to receive it. Second, reward prediction errors. Or, in other words, motivation and learning.
  • Insofar as this ties into the dopamine stuff we’ve learnt about so far, it’s been theorised that dopamine signals essentially provide information about how worthwhile it is to expend specific types of limited resource at any given time.
• The basal ganglia includes two pathways: the direct pathway and the indirect pathway. Though the circuitry isn’t as neat as we might like, it appears that the direct pathway is responsible for selecting a specific action, whereas the indirect pathway is involved in inhibiting all other courses of action.
  • So essentially what seems to happen is that, for example, when dopamine levels start to rise, two things occur:
  • Dopamine binds to D1 receptors in the striatum, which results in motivation, attention, physical energy, etc., getting assigned to a specific course of action.
  • Dopamine binds to D2 receptors in the striatum, which works to inhibit the assignment of motivation, attention, working memory, physical energy, etc. to other possible options.

My own side notes:

• There seems to be little basis about resetting or lower dopamine level (e.g. dopamine detox etc).
  • "Dopamine detox" is made up. CBT is the right thing to do (source)
• Drug addiction: dopamine causes us to want to repeat the experience
  • "It was once thought that surges of the neurotransmitter dopamine produced by drugs directly caused the euphoria, but scientists now think dopamine has more to do with getting us to repeat pleasurable activities (reinforcement) than with producing pleasure directly." (source)
• Why do some pleasant activities seem to be able to provide lasting enjoyment (e.g., making progress on learning a new skill), while others don't (e.g., watching TV)?
  • Due to dopamine, new pleasurable experience trains our brains to want to repeat the experience.
  • Effortful progression (learning new skills) seems to naturally move the pleasure to a new high (better skills/satisfaction) as efforts are put forth. It seems like a virtuous cycle that is also self-regulating (due to effort/pain expenditure) - no pain, no gain.
  • Zero-effort pleasures cause us to acquire larger quantities of such pleasure not with effort/pain but with money/time. In itself, this is okay but less self-regulating, and eventually, it causes us to spend our life (money and time) trapped in this (e.g., TV trapping us to the couch, gambling trapping us to poverty).

Misinformation/non-information

• Dopamine Nation
  • All metaphors of general human behavior and not really about how dopamine works. To understand how bad this book is, read the first few reviews.
• Huberman podcasts
  • No, there is no freaking "dopamine reservoir" involved, and I don't think anything self-consistent was presented either.
• And most online articles/podcasts that use way too many metaphors and/or don't cite academic references.