Dopamine-regulated Cue Salience: Towards the Formal Modelling of the Enactive Mind

Introduction

A current challenge facing enactive cognition is the modelling of empirically verifiable theories of how the brain, body and environment interact. In this paper I attempt to address this challenge. Specifically, I draw from Friston et al.’s (2012) neurocomputational work on dopamine-regulated cue salience in Parkinson’s disease and Bruineberg and Rietveld's (2017) model for selective openness to affordances in OCD to develop a dopamine-regulated cue salience model of Bipolar disorder. I suggest this approach provides support to both accounts and forwards the potential of a computationally and neurobiologically plausible enactivism.

In what follows, I will define key terms and outline key aspects of the two aforementioned studies I have drawn from in developing my model. Then I’ll provide an overview of Bipolar disorder, focusing on what is widely accepted to be an instance of a hyper-dopaminergic state -- Bipolar mania (henceforth BPM). I will then present my argument in two stages. Firstly, I will demonstrate that BPM involves overestimating the time delay on rewards and that this temporal discounting makes cues which signal immediate rewards more salient. Secondly, I will provide evidence that BPM is an instance of aberrant predictive coding -- specifically overly precise priors. This, I will demonstrate, leads to the discounting of sensory evidence, thus rendering cues signalling previously experienced states more salient. It is this convergence of cues signalling immediate reward and previously experienced rewards, I will argue, that underpins selective openness to affordances in BPM. Once these arguments have been presented I will consider an anticipated objection -- that enactivism remains empirically unverifiable. In response to this I will propose a testable hypothesis for my model informed by similar research into dopamine regulated cue salience. 

Enactivism and Affordances

To start I will provide a brief overview of the concepts I will be discussing. The enactivism I am referring to is a version of extended cognition (a group of theories that claim cognition extends from the skull to the body and in some cases to the environment). Enactivism holds that cognition arises through the dynamic interaction between an organism and its environment. The core idea is that we don’t passively experience the world. We actively create or “enact” a world through a constant dynamical interaction between our brain, body and the affordances offered to us by our environment (Gibson, 1979). This concept of affordances is central to enactive cognition. Affordances are resources offered by the environment to an organism that has the capacity to make use of them (Gibson, 1979). In other words, they are the opportunities for action we perceive in the world. For example, if you walk into a conference room before a talk, an appropriately placed chair might afford you the opportunity of sitting on it. In another setting, if you need to reach up high to change a lightbulb, that same chair might afford you the opportunity of standing on it. The core idea of affordances in enactivism is that you don’t initially perceive a value-free, objective thing called “chair” and then process how you want to use it. Instead, so the claim goes, you initially perceive the chair as soliciting a particular action policy given the organism you are and the context you are in. A question some enactivists, including Bruineberg & Rietveld, (2014) attempt to answer is why we are selectively open to some affordances in our environment and not others. As we go about our day to day lives the number of opportunities for action available to us are astounding. Why do some action possibilities seem to solicit us more so than others? This notion is called selective openness to affordances. A promising way of investigating why some actions solicit us more than others is by looking at disorders characterised by pathological decision making. For example, Bruineberg & Rietveld (2014) have studied why people with OCD feel utterly compelled to fulfil one particular course of action, perhaps repeatedly. Similarly, Friston (2012) has investigated why people who are addicted to substances are likewise so compelled to act on maladaptive cues in the environment. Now enactivism and affordances have been outlined I will move to a rough sketch of the concept of predictive coding (henceforth PC) relevant to this paper. 

Predictive Coding 

The question to address now is how enactivism and specifically the concept of affordances relates to predictive coding and active inference. PC is a theory of cognition garnering quite significant and swiftly increasing empirical support (Barrett, 2016). It holds that the brain generates predictions in a top down hierarchical fashion about what sensory stimuli an organism will encounter. The brain then compares these predictions to the actual sensory stimuli received, and if there is a difference between the two a prediction error signal is sent back up the hierarchy to update the organism’s model of how the world is, and to optimise future predictions (Friston, 2009). The purpose of minimising prediction error in this way is to infer the causes of sensory inputs using a version of Bayesian inference, a theory used to continually update the probability of a hypothesis being correct as more relevant information becomes available. Underpinning these processes is the core idea that organisms seek out predictable states as a way to maximise chances of survival in a world filled with noisy, unpredictable sensory input that may emit from hidden causes which could pose a potential threat to the organism. With this outline of PC in hand we can now turn to the related concept of active inference. 

Active Inference

The principle of minimising prediction error referenced above is also used to provide an account of the relationship between perceptual and motor systems. The view here is that organisms act to sample sensory stimuli in ways that better confirm prior beliefs, as a way to optimise the fulfilment of predictions (Friston, 2009). This particular view of predictive coding offers an explanation as to how an organism’s motor and perceptual systems work together as a single unit to predict sensory input. (Adams, Shipp, & Friston, 2013). Friston (2012) proposes an explanation as to how active inference relates to the enactive cognition that constitutes the focus of this paper. He notes that “Active inference can be seen as an embodied (enactivist) form of predictive coding in which perception minimises exteroceptive prediction errors and action minimises proprioceptive prediction errors” (Friston, 2012 pp.1). With an outline of the relationship between enactivism, affordances and predictive coding driven active inference in hand, we are now in a position to see how they come together in Friston et al.’s (2012) study on dopamine regulated cue salience in Parkinson’s disease. Recall I intend to draw from this study in developing the model I suggest may fruitfully be applied to BPM. 

Dopamine Regulated Cue Salience in Parkinson’s Disease

I will now outline the most relevant features of Friston et al.’s (2012) paper Dopamine, Affordance and Active Inference. This study tested the hypothesis that dopamine encodes the precision of cues that solicit action. Significantly, the authors proposed that dopamine balances top down and bottom up signals in such a way that it specifically signals which cues in the environment are soliciting enough to warrant an action policy. This hypothesis was tested and supported by the computational modelling of dopamine induced aberrant precision. Specifically, the authors were able to reproduce core symptomatic traits of Parkinson’s disease -- a disorder characterised by dopamine dysregulation. Physiologically, their findings were compatible with the short latency dopamine bursts evident in the basal ganglia which occur after any salient event. Friston et al. (2012) were therefore able to computationally model dopamine regulated affordance perception. This study was significant as it marked one of the first times the concept of affordances -- traditionally referenced in the fields of phenomenology and ecological psychology (Gibson, 1979), was modelled computationally and in a way that is neurobiologically plausible. I now turn to the second study which informed the development of my model for cue salience in BPM. 

Selective Affordance Perception in OCD

Bruineberg and Rietveld are two philosophers currently working on enactive cognition and particularly on the importance and nature of affordances in the brain-body-environment system. They are particularly interested in the mechanisms driving an organism’s selective openness to affordances in the environment (Bruineberg & Rietveld, 2014; 2018). In their 2014 paper Self-organization, free energy minimization, and optimal grip on a field of affordances, they hypothesise that this selective openness is compatible with Friston’s (2009) theory of active inference. Unlike Friston et al.’s (2012) computational work, Bruineberg and Rietveld (2014) do not commit to a neurobiological model. Instead, a central focus of their paper is the claim that mood disorders can be usefully understood as prolonged distortions in an individual’s field of affordances. That is to say that, unlike individuals who do not experience a mood disorder, their opportunities for action in the environment are limited, constraining their ability to follow through with adaptive action policies. Bruineberg and Rietveld (2014) report on a qualitative study they conducted with OCD patients who underwent deep brain stimulation. According to the authors, OCD can be characterised as an instance of a distorted field of affordances whereby limited, and sometimes singular opportunities for action are so soliciting that the individual is unable to choose another action policy. Bruineberg and Rietveld (2014) report that after treatment with DBS, the OCD patients who participated in the study reported phenomenological changes strongly indicative of changes in their responsiveness to a previously distorted field of affordances. I suggest that this qualitative work drawing on phenomenology and ecological psychology can be fruitfully compared with Friston et al.’s (2012) computational and biologically plausible model of selective affordance perception. Both studies offer different levels of explanation as to how organisms interact with cues in their environment. Despite the differences in methodology I currently perceive no reason to consider the essential thrust of the two approaches to cue salience as incompatible. Indeed, Bruineberg and Rietveld (2014), reference Friston et al.’s (2012) study as providing support for their theory of an active inference associated dynamic relationship between organism and environment. Now that these two studies have been outlined I will briefly define Bipolar disorder and demonstrate that BPM is considered an instance of a hyper-dopaminergic state, as this is relevant to my model of cue salience in BPM. 

Bipolar Mania as a Hyper-dopaminergic State

Bipolar is a disorder characterised by episodes of mood swings which range from depression to manic highs. Given the scope of this paper, my focus is on the manic phase, although there is evidence that the role of dopamine I suggest underlies BPM also plays a role in the cyclical nature of the disorder and that overcompensation in reversing hyper-dopaminergic states may be involved in the depressive phase (Ashok, 2017). The manic phase which I am focusing on is typically characterised by impulsivity, maladaptive decision making and abnormal reward processing. 

In the Bipolar literature it is widely accepted that BPM is an instance of a hyper-dopaminergic state (Ashok et al., 2017; Cousins, Butts & Young, 2009; O’Sullivan, Evans & Lees, 2009)

As noted by Ashok et al. ( 2017), “Converging findings from pharmacological and imaging studies support the hypothesis that a state of hyperdopaminergia, specifically elevations in D2/3 receptor availability and a hyperactive reward processing network, underlies mania” (Ashok et al., 2017, pp 1). This is supported by another critical evaluation of the literature “including a review of behavioural, neurochemical, receptor, and imaging studies, as well as genetic studies focusing on dopamine receptors and related metabolic pathways” (Cousins, Butts & Young, 2009, pp 1). These authors concluded that “Multiple lines of evidence, including data from pharmacological interventions and structural and functional magnetic resonance imaging studies, suggest that the dopaminergic system may play a central role in bipolar disorder and future research into the pathophysiological mechanisms of bipolar disorder and the development of new treatments for bipolar disorder should focus on the dopaminergic system.” (Cousins, Butts & Young, 2009, pp 1). There is thus ample evidence that dopamine plays a central role in Bipolar and specifically in the manic phase characterised as a hyper-dopaminergic state. With a grasp on this relationship between dopamine and BPM I’ll now turn to my claim that temporal discounting in BPM influences cue salience. 

Temporal Discounting and Cue Salience 

Ample evidence indicates that individuals with Bipolar disorder overestimate the time delay on rewards (Ahn et al., 2011; Monterosso, Ehrman, Napier, O'Brien, & Childress, 2001; Fellows and Farah, 2005). I suggest that this is relevant to affordance perception because this tendency would cause already temporally distant affordances or action cues to seem further away than they really are. This seems to lead to a plausible hypothesis that individuals experiencing BPM perceive future affordances as less soliciting as action cues than they typically should be compared to an individual who is not in a hyper-dopaminergic state. Therefore, temporal discounting would likely makes cues which signal immediate rewards more salient. Another influence on cue salience in BPM, I will now suggest, is aberrant predictive coding. 

BPM as Aberrant Predictive Coding

A review of recent accounts of aberrant predictive coding suggests that BPM can be usefully understood as an instance of this, and specifically as a case of overly precise priors. To date, aberrant predictive coding has been utilised in explaining traits in autism and schizophrenia (Adams, Huys & Roiser, 2016), OCD (Bruineburg & Reitveld, 2017), Parkinson’s disease (Friston, 2012) and depression (Barrett, Quigley, & Hamilton, 2016). More specifically relevant to this paper is evidence suggesting that dopamine dysregulation plays a central role in the aberrant predictive coding that seems to underlie Parkinson’s disease (Friston, 2012) and depression (Barrett, Quigley, & Hamilton, 2016). A good reason to consider the aberrant mechanism in BPM as overly precise priors is provided by Schwartenbeck et al. (2016), who argue that “hyperdopaminergic states—associated with pathologically high levels of (prior) precision—may induce maladaptive conditions such as overly optimistic state inference”. Recall that BPM is widely considered an instance of a hyperdopaminergic state. Additionally, BPM has been associated with optimism bias (Schönfelder, Langer, Schneider & Wessa, 2017). Furthermore, in research conducted by Cassidy et al. (2018) on hyper-dopaminergic states they propose underlie psychosis, findings suggest that prior expectations afforded extra weight could stem from a process controlled by dopamine. The authors note that this bias towards prior expectations could be induced pharmacologically by amphetamine and that there was a strong correlation between this induced bias and the striatal release of dopamine. It thus appears that there is sufficient evidence to posit that if we are going to look at BPM in the framework of predictive coding, what we are likely seeing is a case of overly precise prior beliefs. Now that these claims about influences on cue salience in BPM have been presented I will propose that they converge in support of my thesis.

Selective Openness to Affordances in BPM

The task at hand now is to bring these lines of enquiry together. As I have argued, for an individual in the manic phase of Bipolar, immediate as opposed to delayed rewards are more soliciting as action cues. Converging evidence also seems to point to BPM as an instance of aberrant perception and specifically overly precise priors. Such a bias towards prior expectations should lead to cues signalling previously experienced states being perceived as more soliciting than cues signalling as yet uninhabited states, even if sensory evidence indicates that these unfamiliar states are equally rewarding. 

I argue that this convergence of cues signalling immediate reward and cues signalling previously experienced reward provides an avenue to understanding what underpins selective openness to affordances in BPM. That is to say, cues which meet both criteria should be perceived as the most salient to an individual experiencing BPM, and this could constrain the opportunities for action they perceive in their environment. This may go some way to explaining the impulsivity, maladaptive decision making and abnormal reward processing which characterises BPM. This hypothesis is consistent with Friston et al.’s (2012) computational model of dopamine regulated cue salience and Bruineberg & Rietveld’s (2014) theoretical account of distorted affordance perception in mood disorders. 

If I am on the right track we should be able to model selective affordance perception in BPM as a function of a hyper-dopaminergic state leading to aberrant predictive coding and temporal discounting. Modelling cue salience in BPM in this way would allow us to connect useful concepts from phenomenology and ecological psychology with enactive cognition and predictive coding. The potential of such a synthesis may lie in it being both amenable to empirical testing and compatible with current research into the role of dopamine in Bipolar disorder. 

Fig 1. Model of dopamine regulated cue salience in BPM

An Anticipated Objection

There is currently lively debate about whether (and if so, how) the concept of mental representations can be accommodated by enactive cognition. Another topic receiving a lot of attention is how to position enactivism within increasingly well supported predictive coding theories of mind. However, the breadth of these topics falls well outside the scope of this paper. I will, therefore, hone in on a key objection to enactivism -- that it is not empirically verifiable and is therefore not particularly useful to the field of cognitive science. In The poverty of embodied cognition, Goldinger et al. (2016) sum up this view by claiming that embodied (and by inclusion, enactive cognition), is too vague, offers little scientific insight and is characterised by ideas that are trivially true. I will now attempt to respond to this claim by advocating the idea of computational enactivism. 

The Promise of Computational Enactivism

I argue that Friston et al.’s (2012) computational modelling of cue salience in Parkinson’s disease and his (2012) computational modelling of affordances in addiction demonstrate the potential of an empirically verifiable enactivism that can benefit cognitive science. It is likely, I suggest, that following these lines of enquiry will help to explain yet to be discovered facets of the brain-body-environment relationship. In support of this claim, I suggest that the model I have presented here provides an empirically testable hypothesis. If it is on the right track, then I propose that we should see the following results:

Firstly, utilising fMRI, we would expect to see increased activity in the midbrain (which encompasses dopamine projection neurons) when a hypomanic individual encounters an affordance that offers both an immediate reward and a state that the subject has previously experienced as rewarding. Dopamine projections should be more pronounced when both of these elements are present compared to just one or the other. If dopamine projections are larger when both conditions are met that would provide support for the idea that is this combination of factors which underpins the solicitation of cues which are most salient. We would also expect to see increased dopamine projections in hypomanic individuals compared to controls in the same conditions. 

Another testable hypothesis is that when given the option to choose between rewards which have been previously experienced and rewards which have not yet been experienced, but offer sensory evidence that they are just as valuable, hypomanic prone individuals should choose the familiar reward. In this way they would demonstrate less exploratory behaviour than non hypomanic individuals, who should be more willing to instigate action policies informed by sensory data. Both of these hypotheses could be tested with human subjects using fMRI, utilising a similar experimental design to Schwartenbeck et al. (2014), who used fMRI to measure dopaminergic firing in the midbrain during a task which involved subjects making decisions about potential rewards.

I suggest that if the results of these tests provided support for the hypotheses, it would indicate that the computational modelling of cue salience can be usefully applied to BPM, and this in turn would provide support for the claim that theories of affordance perception and enactive cognition are empirically verifiable. 

Conclusion

If my account of dopamine regulated cue salience in BPM is on the right track, it would provide support to both Friston et al.’s (2012) and Bruinefield and Rietveld’s (2017) models of selective affordance perception and enactive theories of cognition more generally. It would also suggest that it is beneficial to continue a two way exchange, whereby theories of selective openness to affordances can prompt future investigation of dopaminergic function and existing knowledge of dopamine-regulated cue salience can, in return, influence theoretical development.

References:

Ahn, W. Y., Rass, O., Fridberg, D. J., Bishara, A. J., Forsyth, J. K., Breier, A., ... & O'donnell, B. F. (2011). Temporal discounting of rewards in patients with bipolar disorder and schizophrenia. Journal of abnormal psychology, 120(4), 911.

Ashok, A. H., Marques, T. R., Jauhar, S., Nour, M. M., Goodwin, G. M., Young, A. H., & Howes, O. D. (2017). The dopamine hypothesis of bipolar affective disorder: the state of the art and implications for treatment. Molecular psychiatry, 22(5), 666.

Barrett, L. F., Quigley, K. S., & Hamilton, P. (2016). An active inference theory of allostasis and interoception in depression. Philosophical Transactions of the Royal Society B: Biological Sciences, 371(1708), 20160011.

Bruineberg, J., & Rietveld, E. (2014). Self-organization, free energy minimization, and optimal grip on a field of affordances. Frontiers in human neuroscience, 8, 599.

Bruineberg, J., Kiverstein, J., & Rietveld, E. (2018). The anticipating brain is not a scientist: the free-energy principle from an ecological-enactive perspective. Synthese, 195(6), 2417-2444.

Bermudez, M. A., & Schultz, W. (2014). Timing in reward and decision processes. Philosophical Transactions of the Royal Society B: Biological Sciences, 369(1637), 20120468.

Cassidy, C. M., Balsam, P. D., Weinstein, J. J., Rosengard, R. J., Slifstein, M., Daw, N. D., ... & Horga, G. (2018). A perceptual inference mechanism for hallucinations linked to striatal dopamine. Current Biology, 28(4), 503-514.

Cousins, D. A., Butts, K., & Young, A. H. (2009). The role of dopamine in bipolar disorder. Bipolar disorders, 11(8), 787-806.

Fellows, L. K., & Farah, M. J. (2005). Dissociable elements of human foresight: a role for the ventromedial frontal lobes in framing the future, but not in discounting future rewards. Neuropsychologia, 43(8), 1214-1221.

Friston, K. J., Shiner, T., FitzGerald, T., Galea, J. M., Adams, R., Brown, H., ... & Bestmann, S. (2012). Dopamine, affordance and active inference. PLoS computational biology, 8(1), e1002327.

Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Boston, MA: Houghton Mifflin.

Goldinger, S. D., Papesh, M. H., Barnhart, A. S., Hansen, W. A., & Hout, M. C. (2016). The poverty of embodied cognition. Psychonomic bulletin & review, 23(4), 959-978.

Kirchhoff, M. D. (2015). Experiential fantasies, prediction, and enactive minds. Journal of Consciousness Studies, 22(3-4), 68-92.

Monterosso, J., Ehrman, R., Napier, K. L., O'brien, C. P., & Childress, A. R. (2001). Three decision‐making tasks in cocaine‐dependent patients: Do they measure the same construct?. Addiction, 96(12), 1825-1837.

O’Sullivan, S. S., Evans, A. H., & Lees, A. J. (2009). Dopamine dysregulation syndrome. CNS drugs, 23(2), 157-170.

Rietveld, E., & Kiverstein, J. (2014). A rich landscape of affordances. Ecological Psychology, 26(4), 325-352.

Schönfelder, S., Langer, J., Schneider, E. E., & Wessa, M. (2017). Mania risk is characterized by an aberrant optimistic update bias for positive life events. Journal of affective disorders, 218, 313-321.

Schwartenbeck, P., FitzGerald, T. H., Mathys, C., Dolan, R., & Friston, K. (2014). The dopaminergic midbrain encodes the expected certainty about desired outcomes. Cerebral cortex, 25(10), 3434-3445.

Sozou, P. D. (1998). A Discounting framework for choice with delayed and probabilistic rewards. Proc. R. Soc. Lond. B 265, 2015–2020. doi: 10.1098/rspb.1998.0534

Talmi, D., Atkinson, R., & El-Deredy, W. (2013). The feedback-related negativity signals salience prediction errors, not reward prediction errors. Journal of Neuroscience, 33(19), 8264-8269

Teufel, C., Subramaniam, N., & Fletcher, P. C. (2013). The role of priors in Bayesian models of perception. Frontiers in computational neuroscience, 7, 25.

Whitton, A. E., Treadway, M. T., & Pizzagalli, D. A. (2015). Reward processing dysfunction in major depression, bipolar disorder and schizophrenia. Current opinion in psychiatry, 28(1), 7.