Empirical support for an involvement of the mesostriatal dopamine system in human fear extinction

Abstract

Exposure therapy for anxiety disorders relies on the principle of confronting a patient with the triggers of his fears, allowing him to make the unexpected safety experience that his fears are unfounded and resulting in the extinction of fear responses. In the laboratory, fear extinction is modeled by repeatedly presenting a fear-conditioned stimulus (CS) in the absence of the aversive unconditioned stimulus (UCS) to which it had previously been associated. Classical associative learning theory considers extinction to be driven by an aversive prediction error signal that expresses the expectation violation when not receiving an expected UCS and establishes a prediction of CS non-occurrence. Insufficiencies of this account in explaining various extinction-related phenomena could be resolved by assuming that extinction is an opponent appetitive-like learning process that would be mediated by the mesostriatal dopamine (DA) system. In accordance with this idea, we find that a functional polymorphism in the DA transporter gene, DAT1, which is predominantly expressed in the striatum, significantly affects extinction learning rates. Carriers of the 9-repeat (9R) allele, thought to confer enhanced phasic DA release, had higher learning rates. Further, functional magnetic resonance imaging revealed stronger hemodynamic appetitive prediction error signals in the ventral striatum in 9R carriers. Our results provide a first hint that extinction learning might indeed be conceptualized as an appetitive-like learning process and suggest DA as a new candidate neurotransmitter for human fear extinction. They open up perspectives for neurobiological therapy augmentation.

Citation: Transl Psychiatry (2011) 1, e12, doi:10.1038/tp.2011.10

& 2011 Macmillan Publishers Limited All rights reserved 2158-3188/11 http://www.nature.com/tp

Web End =www.nature.com/tp

Empirical support for an involvement of the mesostriatal dopamine system in human fear extinction

KA Raczka1, M-L Mechias1, N Gartmann1, A Reif2, J Deckert2, M Pessiglione3 and R Kalisch1

Exposure therapy for anxiety disorders relies on the principle of confronting a patient with the triggers of his fears, allowing him to make the unexpected safety experience that his fears are unfounded and resulting in the extinction of fear responses. In the laboratory, fear extinction is modeled by repeatedly presenting a fear-conditioned stimulus (CS) in the absence of the aversive unconditioned stimulus (UCS) to which it had previously been associated. Classical associative learning theory considers extinction to be driven by an aversive prediction error signal that expresses the expectation violation when not receiving an expected UCS and establishes a prediction of CS non-occurrence. Insufciencies of this account in explaining various extinction-related phenomena could be resolved by assuming that extinction is an opponent appetitive-like learning process that would be mediated by the mesostriatal dopamine (DA) system. In accordance with this idea, we nd that a functional polymorphism in the DA transporter gene, DAT1, which is predominantly expressed in the striatum, signicantly affects extinction learning rates. Carriers of the 9-repeat (9R) allele, thought to confer enhanced phasic DA release, had higher learning rates. Further, functional magnetic resonance imaging revealed stronger hemodynamic appetitive prediction error signals in the ventral striatum in 9R carriers. Our results provide a rst hint that extinction learning might indeed be conceptualized as an appetitive-like learning process and suggest DA as a new candidate neurotransmitter for human fear extinction. They open up perspectives for neurobiological therapy augmentation.

Translational Psychiatry (2011) 1, e12; doi:http://dx.doi.org/10.1038/tp.2011.10

Web End =10.1038/tp.2011.10 ; published online 7 June 2011

Introduction

Classical associative learning theory explains fear conditioning by an aversive prediction error signal dav generated when an initially non-predictive conditioned stimulus (CS) is unexpectedly followed by an unconditioned stimulus (UCS). This establishes a UCS prediction, or aversive value Vav, for

the CS that grows over successive pairings.1,2 If at some point

the UCS is unexpectedly omitted, this generates a negative (oppositely signed) aversive prediction error that will reduce Vav. The latter mechanism is thought to underlie the extinction of conditioned fear responses by repeated unpaired CS presentations.1 It is the theoretical basis of exposure therapy where a patient is repeatedly confronted with the trigger of his fears (the CS, for example, in an agoraphobic, an open space) and makes the experience that the predicted outcome (the UCS) is absent or less disastrous than expected (for example, he does not collapse).3

A frequent nding is that conditioned fear responses can return after successful extinction, indicating that the CSUCS association (Vav) is not simply unlearned or erased during extinction but rather complemented by a competing inhibitory CSnoUCS association that may, or may not, dominate the CSUCS association at future CS presentations.46 More

over, there is compelling evidence for a partial segregation in

the neural systems subserving conditioning and extinction.712

The above simple account of extinction, as being solely mediated by the same learning system that also mediates conditioning, cannot accommodate these observations.

Alternatively, the omission of an expected aversive UCS could be conceptualized as an appetitive-like or reward prediction error dapp and the consequential reduction of the

UCS prediction Vav during extinction as generation of a reward-like safety prediction Vapp. From this perspective, part of a solution for the above problem could be that extinction is driven by an opponent appetitive learning system. Reward learning has been strongly linked with the mesostriatal dopamine (DA) system.1315 There is evidence that dapp is

signaled by a phasic increase in the ring of DAergic neurons that originate in the ventral tegmental area and substantia nigra and project to the ventral striatum (VS).15 It has

therefore been hypothesized that VS DA release is involved in putative dapp signaling during fear extinction as well.16 One rodent study that showed that DA signaling via D1 receptors is necessary for extinction17 further supports the potential link between fear extinction and the reward system. One goal of this study was therefore to test whether the VS encodes appetitive-like prediction error signals during extinction in humans.

1Institute for Systems Neuroscience, University Medical Center Hamburg-Eppendorf (UKE), Hamburg, Germany; 2Department of Psychiatry and Psychotherapy, University of Wrzburg, Wrzburg, Germany and 3Centre de Recherche de lInstitut du Cerveau et de la Moelle pinire, INSERM UMRS 975, Paris, France Correspondence: Dr R Kalisch, Institute for Systems Neuroscience, University Medical Center Hamburg-Eppendorf (UKE), Martinistr 52, 20246 Hamburg, Germany. E-mail: mailto:[email protected]

Web End [email protected] Keywords: conditioning; DAT1; dopamine transporter; prediction error; safety learning; ventral striatum

Received 25 Feburary 2011; revised 11 April 2011; accepted 2 May 2011

Fear extinction and DAT1 KA Raczka et al

2

Extracellular DA levels in the striatum are prominently regulated by neuronal DA reuptake via the DA transporter (DAT).18 The human transporter gene DAT1 features a frequent and functional variable number of tandem repeat (VNTR) polymorphism in a region that encodes the 30 untranslated region.19 The 40-bp VNTR element is mainly repeated either 9 or 10 times, with the 9-repeat (9R) form most likely reducing DAT expression2024 (but see Dyck et al.25)

and thus presumably enhancing extrasynaptic striatal DA levels, in particular, during phasic DA release.18 Hence, if the mesostriatal DA system is involved in extinction in the fashion outlined above, one would expect the DAT1 9R allele to be associated with relatively enhanced extinction learning as well as with enhanced neural dapp signaling in the VS. To examine this hypothesis, we conducted the experiment in a sample of normal healthy volunteers that were preselected on the basis of their DAT1 genotype. In particular, we compared 9R carriers (genotypes 9/9 and 9/10) with non-9R carriers (genotypes 10/10). The grouping was chosen because of the relative scarcity of the 9R allele and to be in keeping with previous DA binding neuroimaging studies.20,21

We also examined effects of interindividual variation in COMT (catechol-o-methyltransferase) function. In contrast to DAT, COMT is most strongly expressed in the prefrontal cortex (PFC)26 where it degrades released DA, thereby regulating extracellular PFC DA levels.27 The human COMT gene contains a functional single-nucleotide polymorphism that codes for a Val to Met change at position 158,28 the Met

variant of the protein being less active29 and associated with higher prefrontal baseline synaptic DA.27 Prefrontal DA appears to have a role in extinction in rats,30 and a recent human study had suggested impaired extinction in COMT Met/Met carriers.31 Hence, including COMT genotype in the design allowed us to also explore potential contributions of extrastriatal DA to human fear extinction. As the Val and Met alleles are codominant,27 participants were preselected in a way to obtain three similarly sized groups of Val/Val, Val/Met and Met/Met carriers. This resulted in a 2 by 3 (DAT1 by COMT) factorial between-subject design.

Participants and methods

Participants. A total of 69 healthy male right-handed Caucasian participants with DAT1 and COMT genotypes 9R-Val/Val (n 13), 9R-Val/Met (n 10), 9R-Met/Met

(n 10), non-9R-Val/Val (n 14), non-9R-Val/Met (n 12)

and non-9R-Met/Met (n 10) were examined. As

participants were drawn from a bigger population to achieve a stratied and matched population, calculations of HardyWeinberg equilibrium (HWE) were only appropriate for the basic population (n 450). Genotype distributions

were as follows DAT1 9/9: n 28; 9/10: n 148;

10/10: n 252 (HWE w2 0.97), COMT Val/Val: n 80;

Val/Met: n 237; Met/Met: n 118 (HWE w2 4.18), and

thus above the HWE threshold of P 0.01. Details on

participant selection, sample characteristics and genotyping can be found in the Supplementary Methods and Supplementary Table 1. A different analysis of an overlapping sample has been reported before.32

Experiment. Participants performed a simple uninstructed fear conditioning, extinction and reacquisition task, which has been described in detail elsewhere.32 Briey, participants were rst habituated to the CSs, the task and the scanner noise by presenting each CS four times before the actual experiment. In the subsequent acquisition phase (Acq), participants saw 18 pseudorandomized 5-s presentations of each of two geometric symbols (a circle, a triangle), one of which (CS ) was paired in 80% of cases with a painful

electric stimulus (UCS) applied to the back of the right hand. The other symbol served as a control stimulus (CS )

for non-associative effects and was never paired with the UCS. Assignment of symbols as CS or CS was

counterbalanced across participants. In the extinction phase (Ext), both stimuli were again presented 18 times each, but in the absence of the UCS. The subsequent reacquisition phase (RAcq) was identical to the acquisition phase. The intertrial interval was jittered between 914 s, with an average of 11.5 s. We intermittently asked participants to give explicit ratings of their CS-evoked stress/fear/tension (at baseline, that is, after the habituation phase, and every 12 trials (six CS and six CS trials) thereafter, resulting in

three CS and three CS ratings per phase). Throughout

the experiment, the participants had to perform a speeded decision task on the geometric symbols (see Supplementary Methods). UCS intensity was individually adjusted before the experiment to achieve maximum tolerable pain.

Data acquisition and preprocessing. Acquisition and preprocessing of skin conductance and functional magnetic resonance imaging (fMRI) data followed standard procedures (see Supplementary Methods).

Data analysisFear ratings. All ratings were normalized by subtracting the baseline ratings given at the onset of the experiment (after habituation) such that positive ratings reected an increase in fear relative to baseline and negative ratings a relative decrease in fear. In four participants, ratings were not acquired or lost because of technical problems. The remaining sample size for analysis of ratings was n 65.

RescorlaWagner model. The RescorlaWagner (RW) model is a simple and established associative learning model that formalizes the laws of learning outlined in the introduction. If learning is about enabling an organism to predict relevant future events from present stimuli, then classical conditioning should result in the organism being able to predict a UCS from the presentation of the CS. That is, the CS should activate a UCS expectation (or CSUCS association or UCS prediction) that reects the probability and magnitude of the UCS. This associative strength or affective value of the CS is expressed in the V term of the RW equation below and will increase over the course of conditioning. It is thought to determine conditioned responding. Every violation of this expectation (for example, because a UCS occurs unexpectedly, as initially in the beginning of conditioning when the UCS prediction is still 0, or because an expected UCS does not occur, as in initial extinction) must result in an adjustment of the

Translational Psychiatry

Fear extinction and DAT1 KA Raczka et al

3

expectation, that is, learning. Therefore, the update of V in the RW model is directly proportional to a prediction error term d that reects the difference between actual and predicted reinforcement (that is, the expectation violation).1

This class of models is known to be relevant for learning about punishments and rewards, and has been successfully used to predict learning-related neural activation.3339

Specically, the RW model updates V at trial t 1

according to

Vt1 Vt aR Vt;

with a being the constant learning rate (01), R being a xed value assigned to the reinforcement/UCS and (R Vt)

corresponding to the prediction error d that is generated at the time of reinforcement.

We used this rule to model how participants change their aversive CS and CS values Vav depending on their

associated reinforcement pattern. A ow chart showing the separate analysis steps is provided in the Supplementary Methods. We rst range corrected each participants CS

and CS fear rating data (see Supplementary Figure 1a for

sample average) according to

xi;corr; CSor CS xi;

CSor CS min=max min; with xi,CS orCS (i 1,y,10) being the successive (CS

or CS ) fear ratings, min being the sample-wide minimum

of all ratings ( 58) and max being the sample-wide maximum

of all ratings (100). This resulted in individual rating time courses in which ratings ranged from 0 to 1 but retained interindividual differences in how participants used the rating scale. The (CS and CS ) baseline ratings x1,CS

orCS

Vav,CS and the change in fear ratings xcorr,CS and xcorr,CS

using a least-square approach. We did not use CS 4CS

difference scores, and Vav,CS and Vav,CS were estimated at the same time within the same model. Vav,CS and Vav,CS

time courses averaged across the entire sample are shown in Figure 1a. Average least sums of squares were similar between genotypes (DAT1: F(1,64) 0.55, P 0.461;

COMT: F(2,64) 0.13, P 0.878; and interaction:

F(2,64) 0.49, P 0.613). Model ts were substantially

worse when assuming one identical learning rate across all three phases of the experiment (data not shown).

Note that the original RW formula assumes different learning rates for reinforced and non-reinforced CS trials, but this differentiation is not critical for most of the models predictions1 and has not been made in neuroimaging studies.3339 Allowing different learning rates for reinforced and non-reinforced CS trials yielded worse ts (data not shown).

Imaging data. Analysis of imaging data was restricted to those 65 participants from which fear ratings were also available. In the comparison of DAT1 9R with non-9R carriers, group sizes were n 32 and n 33, respectively.

To prepare the analysis, we used the sample-averaged learning rates aAcq 0.16, aExt 0.21 and aRAcq 0.19 to

derive individual trial-by-trial Vav and dav estimates from the above modeling of the rating data, separately for acquisition, extinction and reacquisition. Averaging of learning rates was necessary to reduce noise in the data that resulted from a limited number of data points for tting (10 ratings), and thus to obtain robust estimates. An exemplary individual dav time

course is shown in Figure 1b. We emphasize that our estimated average learning rates are comparable to those used in previous neuroimaging studies.34

This information was fed into the imaging data analysis that used a standard approach for fMRI, involving a general linear model (multiple regression) at the single-subject level and a random-effects analysis at the group level within the SPM5 software (http://www.fil.ion.ucl.ac.uk/spm

Web End =www.l.ion.ucl.ac.uk/spm). 41 For each participant, regressors were dened that modeled the time course of the experimental events. Onsets of CSs, irrespective of whether they were a CS or a CS , were modeled as categorical

events, that is, one series of delta functions. This regressor was parametrically modulated in a trial-by-trial fashion by the individuals sequence of Vav estimates. Another categorical event regressor modeled CS (both CS and CS ) offsets

and was parametrically modulated by the individuals sequence of dav estimates. This was done for acquisition, extinction and reacquisition separately. CS and CS trials

were not differentiated in this analysis, as the concept of predictions and prediction errors is a purely quantitative one that does not make qualitative distinctions between types of stimuli. We therefore assumed identical neural substrates for predictions and prediction errors, whether associated with a CS or a CS . Additional categorical regressors modeled

UCSs (events), key presses (events), and the occurrence of fear ratings (14-s box car). Each regressor was convolved with a canonical hemodynamic response function. Using these regressors in a general linear model of brain activation at each voxel yields parameter estimates of the contribution of

(after habituation and before conditioning), which were 0 by denition in each participant (see above), became 0.36 ( x

1,corr,CS orCS ).

After complete learning, the aversiveness R of the UCS is reected in the aversiveness of the CS , that is, in the last

CS fear rating after acquisition (x

4,corr,CS ). With a partial

reinforcement schedule of 80%, a participants R in paired CS trials can thus be approximated as x

4,corr,CS /0.8.40 R

in CS trials (0% reinforcement) was set at each participants

x4,corr,CS rating. The same value of R was used for unpaired CS trials. See the Supplementary Methods for a more

detailed explanation. As mentioned above, UCS (pain) intensity in this experiment was individually calibrated to each participants subjectively tolerable maximum to eliminate interindividual differences in UCS processing. Concordantly, the calculated individual R-values for paired CS trials were

not affected by genotype (DAT1: F(1,64) 1.24, P 0.298;

COMT: F(2,64) 0.74, P 0.438; and interaction:

F(2,64) 0.06, P 0.978). We nevertheless used individual

R-values to factor out any potential interindividual differences in learning that might in fact merely result from differences in UCS processing.

Vav,CS and Vav,CS were modeled separately and set at0.36 ( x

1,corr,CS orCS ; see above) before learning. On the

basis of the idea of dissociable neural systems for fear acquisition and extinction (and possibly reacquisition as well), we used three free parameters aAcq, aExt and aRAcq (one for

each of the three experimental phases), which were adjusted to minimize the distance between the change in Vav,CS and

Translational Psychiatry

Fear extinction and DAT1

KA Raczka et al

4

Figure 1 Formal modeling of fear ratings. (a) Lines show the sample average of the modeled trial-by-trial estimates of aversive conditioned stimulus values (Vav,CS and

Vav,CS ). Dots show sample-average range-corrected fear ratings (made after every sixth CS and CS trial). 0: baseline rating after habituation. (b) An example of a

resulting individual time course of trial-by-trial aversive prediction error (dav) estimates associated with the CS . Black squares mark unpaired CS trials during acquisition

and reacquisition. CS trials during extinction were all unpaired. Prediction errors associated with the CS were always 0 and are not shown for simplicity. x axis: CS or

CS trials.

each regressor to the fMRI signal measured in each voxel. The convolved regressors of interest (Vav,Acq, dav,Acq, Vav,Ext,

dav,Ext, Vav,RAcq and dv,RAcq) were sufciently decorrelated

from each other and from the UCS regressor to allow for robust estimation (Pearsons Rs for the correlations between Vav,Acq and dav,Acq: 0.01; Vav,Ext and dav,Ext: 0.21; V

av,RAcq

and dav,RAcq: 0.01; Vav,Acq and UCS: 0.12; d

av,Acq and UCS:

0.43; Vav,Ext and UCS: 0; dav,Ext and UCS: 0; Vav,RAcq and

UCS: 0.12; and d

av,RAcq and UCS: 0.41). Note further that

the use of a comparatively high reinforcement ratio of 80% during acquisition assured a high initial amount of prediction error signaling during extinction in combination with a relatively steep approach toward zero signal (see Figure 1b). This characteristic time course was sufciently different in shape from the constant categorical CS offset regressor, of which it was a parametric modulator, for it to be able to explain additional variance. At the same time, choosing a partial reinforcement ratio also seemed preferable to a 100% schedule, because the latter would have generated a very steep approach toward zero, which would leave little room for modulation by individual-difference factors.

For the voxel-wise random-effects group analyses, the subject-specic parameter estimate images from the parametric dav and Vav regressors were spatially smoothed (FWHM 10 mm, resulting in total smoothing with an 11-mm kernel) to account for interindividual anatomical and functional variance, and to full the requirements for later correction for multiple comparisons following Gaussian random eld theory (see below). DAT1 and COMT genotype effects were assessed using SPMs full factorial model, which allows for correcting for possible non-sphericity of the error term (here unequal between-group variance). Separate models were

calculated for each effect of interest (for example, Vav,Acq).

A design matrix included six regressors, one for each possible genotype combination. The signicance of linear combinations of the regressors (for example, 1 1 1 1 1 1 when

asking which voxels show larger effects for DAT1 9R than for non-9R carriers in a given parameter estimate image) was assessed using one-tailed t-tests.

Correction for multiple comparisons following Gaussian random eld theory (family-wise error method) at Po0.05 was limited to the VS regions of interest (ROIs). Left- and right-sided ROIs were conservatively dened as spheres of 6-mm radius around coordinates.

The values around coordinates 27, 3, 9 and 27, 9, 9,

respectively, which were taken from the rst fMRI study that investigated neural reward prediction error coding using a formal associative learning model.34 Where no anatomical hypothesis existed (exploratory analyses across the entire scan volume), an uncorrected threshold of Po0.001 was used.

Results and discussion

Behavioral data. In the RW model of associative learning (see Participants and methods), the prediction error dav is

weighted by a constant a, the learning rate, that determines how much a deviation from prediction at trial t is taken into account when formulating the prediction for the next trial t 1. A high learning rate signies rapid prediction

adjustment and thus quick learning. If extinction relies (in part) on a different learning system than conditioning, it may well show a different dynamic. We thus assessed learning

Translational Psychiatry

Fear extinction and DAT1 KA Raczka et al

5

separately for the three phases of the experiment, allowing separate learning rates aAcq, aExt and aRAcq. These were

treated as free parameters, which we optimized so that the individual Vav time courses t changes in individual skin conductance responses (SCRs) or fear ratings across the entire experiment (see Supplementary Figure 1 for sample-average SCR and rating time courses).

An attempt to model SCRs failed because of excessive noise in the data. Modeling of fear ratings (Figure 1), followed by three separate two-way analyses of variance (one per phase, each with between-subject factors DAT1 and COMT, and learning rate as the dependent variable) revealed signicantly higher learning rates in DAT1 9R carriers compared with non-9R carriers during extinction (aExt: DAT1

main effect F(1,65) 4.57, P 0.037) but not during acquisi

tion (aAcq: F(1,65) 0.13, P 0.725) or reacquisition (aRAcq:

F(1,65) 3.27, P 0.075; Figure 2). This suggests DAT1 9R

carriers have a more sensitive extinction learning system and is consistent with the idea that striatal DA might positively contribute to extinction learning.

Learning rates were unaffected by COMT genotype (all P40.267), but there was a signicant DAT1 COMT inter

action effect on learning rates in the reacquisition phase (aRAcq: F(2,65) 4.48, P 0.015). A post hoc t-test showed

that DAT1 9R carriers had signicantly higher learning rates than non-9R carriers only in the COMT Val/Met group (9R:0.420.36 (means.d.) versus non-9R: 0.090.13; t(22) 2.55, P 0.025, two tailed; Supplementary Figure 2a).

This incidental nding will be discussed further below. A standard, non-computational analysis of SCR and rating responses for genotype effects yielded no signicant results.

Taken together, behavioral analysis suggested a signicant contribution to extinction of DAT1 in the predicted direction but found no evidence for an involvement of COMT. We note that the cited COMT study by Lonsdorf et al.31 also reported a negative result for SCR and that their COMT effect on startle potentiation might as well be explained by a modulation of fear memory consolidation rather than extinction learning itself. The current data from human subjects therefore do not lend

strong support to the idea30 that prefrontal DA function is important for extinction.

Imaging data: entire sample. In the entire sample, exploratory analysis of extinction data for dav signals (the aversive prediction error characterized by phasic relative decreases in activation when a UCS is unexpectedly omitted, compare with Figure 1b) yielded no signicant results. The putative appetitive-like prediction error dapp is the

mathematical inverse of dav and characterized by relative phasic increases at CS omission. Activity conforming to dapp

was found in, among others, left, and less so right, anterior insula, bilateral anterior lateral PFC, right ventrolateral PFC/ lateral orbitofrontal cortex and right VS (ventral putamen and/ or nucleus accumbens; x, y, z 14, 8, 6; z-score 3.22;

Po0.001 uncorrected; see Figure 3a; Supplementary Table 2), areas previously associated with dapp coding in reward studies42 and with phasic activations to UCS omission during fear conditioning.43 The VS activation was, however, not located within our conservatively dened (left or right) VS ROIs (see participants and methods for denition). As for dav,

there were no suprathreshold Vav signals (the aversive CS value that decreases across extinction, compare with Figure 1a). By contrast, the putative reward-like safety value of the CS, Vapp, which is the mathematical inverse of

Vav and accordingly builds up across extinction, was found to be encoded in left orbitofrontal cortex/ventrolateral PFC, dorsomedial and lateral PFC, temporal cortex, left caudate, cerebellum and others (Supplementary Table 2). These observations might suggest that extinction is indeed primarily driven by reward-like safety learning. Results for the acquisition and reacquisition phases are reported in Supplementary Table 2.

Imaging data: genetic analysis. In the genetic analysis, we focused on the comparison of DAT1 9R with non-9R groups, as COMT genotype had not affected extinction learning rates in the behavioral analysis. If DAT1 9R carriers weight prediction errors during extinction more strongly (have higher learning rates), then they can be expected to show larger neural dav and/or dapp signals during this phase. As predicted, 9R carriers had signicantly stronger signal increases to UCS omission than non-9R carriers, corresponding to stronger dapp coding, in our left VS ROI (x, y, z 32, 8, 6; z-score 2.99; P 0.03 corrected).

The activation was located in the putamen and extended into the anterior insula (Figure 3b). This result supports our conclusion from the behavioral analysis that DAT1 9R carriers have a more sensitive extinction learning system and is evidence for an involvement of the mesostriatal DA system in extinction. Further group differences, all going in the same direction, were observed in left anterior cingulate sulcus and other areas (exploratory analysis at Po0.001 uncorrected; Supplementary Table 3). Group comparisons of

CS value encoding (Vav or Vapp) surprisingly showed stronger Vapp signals in the non-9R compared with the 9R group in two frontal areas (Po0.001 uncorrected; Supplementary Table 3).

We stress the exploratory nature of the latter comparisons and the corresponding likelihood of false-positive results.

Figure 2 DAT1 genotype affects learning rates during extinction. Formal modeling of fear rating data showed that 9-repeat (9R) carriers have signicantly higher learning rates during extinction than non-9R carriers. Error bars: s.e.m. *Po0.05 (F test).

Translational Psychiatry

Fear extinction and DAT1 KA Raczka et al

6

Figure 3 Ventral striatal (VS) prediction error signaling during extinction. (a) Appetitive prediction error (dapp) signal in right VS in the entire sample. (b) Stronger dapp signal in DAT1 9-repeat (9R) compared with non-9R carriers in left VS. Activations superimposed on a canonical structural image. Display threshold: Po0.01 uncorrected. Bar graphs show average dapp parameter estimates in extinction, as well as, for comparison, in acquisition and reacquisition in the peak voxel indicated by the hair cross. Error bars: s.e.m.

The unexpected behavioral nding of higher reacquisition learning rates in 9R compared with non-9R carriers specically in the COMT Val/Met group (see above) was reected in relatively higher dapp signals in the right VS ROI in 9R-Val/Met participants (x, y, z 22, 8, 8; z-score 2.62; P 0.043

corrected; Supplementary Figure 2b). The trial-by-trial variance that is captured by the parametric dapp regressor during reacquisition mainly stems from the dapp increases to the three unexpected UCS omissions (compare with the inverse of the curve shown in Figure 1b, reacquisition part). A ventral striatal DAT1 effect on this type of signal suggests that the safety or appetitive-like mechanisms, which we propose are activated during extinction, carry over to the reacquisition situation. The observation that the DAT1 effect is limited to COMT Val/Met carriers might speculatively be attributed to a situationally dependent inuence of prefrontal DA on striatal DA function.44,45 In this context, it is worth noting that epistatic DAT1 by COMT interactions in VS reward signaling have been observed before.46 The exact nature of the effect must remain open until further investigation.

Supplementary Table 4 reports a genetic analysis of the VS ROIs from all experimental phases and contrasts for DAT1, COMT and DAT1 by COMT effects.

Conclusion. To summarize, our combined behavioral and imaging data hint toward signaling of UCS omission during extinction by phasic DA release in the VS, in analogy to the role of the mesostriatal DA system in reward learning.15 They

support a conceptualization of extinction as a reward-like safety learning process.16 More globally, such a conceptualization can be integrated within a perspective of opponent aversive and appetitive systems.47,48

Several limitations of the current study should be noted. First, there are still controversies with respect to the actual impact of DAT1 genotype on in vivo DAT function and striatal DA clearance (see introduction), and the prevailing hypothesis of stronger phasic DA signals in 9R carriers still has to be substantiated. Second, our approach is correlative, that is, we did not experimentally manipulate striatal DA signaling. Pharmacological manipulations in rodents have shown generally higher levels of conditioned freezing during extinction with DA antagonists given systemically16,49,50 or directly in the

amygdala51 or nucleus accumbens.16 A systemic DA agonist reduced conditioned freezing during extinction.50 Although these results could be taken to support a facilitatory role for DA in extinction learning, it should be noted that DAergic

manipulations also affect locomotion and baseline freezing, and it is therefore difcult to rule out that the enhanced conditioned freezing observed under DA antagonists might be explained by their motor side effects.16,4951 Further, these

studies have analyzed average freezing across the entire extinction session, a measure that might also be confounded by potential general fear-potentiating drug effects. It might therefore be advantageous to instead focus on rates of decay of freezing as a primary outcome measure for extinction in animals. In humans, where conditioned responding is normally not read out from motor behavior, pharmacological experiments might suffer from other confounds such as potential drug effects on arousal.36 Such experiments will therefore have to carefully control for non-specic effects but might nevertheless be a valuable source of evidence. Third, showing a contribution of the mesostriatal DA system does not necessarily prove that extinction is appetitive, as the DA system is not exclusively involved in reward learning. Here, a direct demonstration of the appetitive nature of extinguished CSs would be benecial. Another potentially interesting approach would be a direct formal comparison of extinction with a reward-learning task within the same sample. Fourth, our data do not exclude that non-appetitive, that is, aversive learning mechanisms also contribute to extinction. Fifth, although conditioning and extinction are generally considered relevant paradigms for the study of pathological anxiety and its therapy,3 it must be stressed that they cannot provide explanations for every aspect of anxiety and we can, in particular, not exclude that other mechanisms have a role in therapeutic fear relief. Sixth, our sample included mainly university students and exclusively comprised males. The latter selection criterion was introduced following reports of considerable gender and cycle effects on extinction52 and was

intended to reduce variance, thus allowing us to limit sample size. Reproduction in other samples is therefore required. Seventh, where we reported results from exploratory whole-brain analyses, these are not corrected for multiple comparisons, as emphasized earlier. Again, reproduction will be paramount.

It is worth noting that we did not nd evidence for a role for DA in fear acquisition, in line with one genetic study examining COMT genotype effects on conditioning.31 By contrast, a recent pharmacological fMRI study36 had reported enhanced aversive prediction errors dav in the caudate nucleus and the substantia nigra/ventral tegmental area during conditioning in participants under amphetamine compared with participants

Translational Psychiatry

Fear extinction and DAT1 KA Raczka et al

7

under haloperidol. As amphetamine participants also reported to feel less tired, drowsy and slowed, these results might also reect a general attentional effect. It should, however, also be noted that the absence of DAT1 or COMT effects on our and other peoples measures of conditioning does not exclude a contribution of DA to fear acquisition. Further research will be necessary to settle this question.

To conclude, our ndings highlight DA as a candidate neurotransmitter for fear extinction. This opens up interesting perspectives for neurobiological therapy augmentation, for instance, via adjunctive treatment with DAergic drugs. Experimental studies using the NMDA receptor agonist D-cycloserine to enhance the effects of exposure therapy have demonstrated the potential for such a strategy53 (reviewed in Grillon54). Pharmacological augmentation might be particularly useful in patients resistant to standard forms of behavior therapy. We would, however, caution against testing drugs in patients for which possible potentiating effects on fear expression or conditioning have not been carefully ruled out in previous non-clinical studies. Another promising lead for future studies would be to examine interactions with the endogenous opioid system, which, animal studies suggest, is another potential substrate of error signaling during fear extinction55,56 and therefore another interesting candidate neurotransmitter system for translational research.

Conict of interest

The authors declare no conict of interest.

Acknowledgements. We thank F Fassbinder for technical help and N Bunzeck and T Lonsdorf for comments and suggestions. This work was funded by the Deutsche Forschungsgemeinschaft (DFG Emmy Noether Grant KA1623/3-1 (KR, NG, MLM and RK); DFG Transregional Collaborative Research Centre Grant SFB TRR 58, subproject Z2 (AR and JD)) and the UKEs complementary funding program (RK).

1. Rescorla RA, Wagner AR. A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: Rescorla RA, Wagner AR, Black AH, Prokasy WF (eds). Classical Conditioning II: Current Research and Theory. Appleton-Century-Crofts: New York, NY, 1972, pp 6499.

2. Sutton RS, Barto AG. Reinforcement Learning: an Introduction (Adaptive Computation and Machine Learning). MIT Press: Cambridge, Massachusetts, 1998.

3. Rothbaum BO, Davis M. Applying learning principles to the treatment of post-trauma reactions. Ann N Y Acad Sci 2003; 1008: 112121.

4. Pavlov IP. Conditioned reexes: an investigation of the physiological activity of the cerebral cortex. Oxford University Press: Oxford, 1927.

5. Myers KM, Davis M. Behavioral and neural analysis of extinction. Neuron 2002; 36: 567584.

6. Bouton ME. Context and behavioral processes in extinction. Learn Mem 2004; 11: 485494.

7. Marsicano G, Wotjak CT, Azad SC, Bisogno T, Rammes G, Cascio MG et al. The endogenous cannabinoid system controls extinction of aversive memories. Nature 2002; 418: 530534.

8. Lin CH, Yeh SH, Lu HY, Gean PW. The similarities and diversities of signal pathways leading to consolidation of conditioning and consolidation of extinction of fear memory. J Neurosci 2003; 23: 83108317.

9. Gottfried JA, Dolan RJ. Human orbitofrontal cortex mediates extinction learning while accessing conditioned representations of value. Nat Neurosci 2004; 7: 11441152.

10. Heldt SA, Stanek L, Chhatwal JP, Ressler KJ. Hippocampus-specic deletion of BDNF in adult mice impairs spatial memory and extinction of aversive memories. Mol Psychiatry 2007; 12: 656670.

11. Herry C, Ciocchi S, Senn V, Demmou L, Muller C, Luthi A. Switching on and off fear by distinct neuronal circuits. Nature 2008; 454: 600606.

12. Jungling K, Seidenbecher T, Sosulina L, Lesting J, Sangha S, Clark SD et al. Neuropeptide S-mediated control of fear expression and extinction: role of intercalated GABAergic neurons in the amygdala. Neuron 2008; 59: 298310.

13. Montague PR, Hyman SE, Cohen JD. Computational roles for dopamine in behavioural control. Nature 2004; 431: 760767.

14. Pessiglione M, Seymour B, Flandin G, Dolan RJ, Frith CD. Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 2006; 442: 10421045.

15. Schultz W. Behavioral theories and the neurophysiology of reward. Ann Rev Psychol 2006; 57: 87115.

16. Holtzman-Assif O, Laurent V, Westbrook RF. Blockade of dopamine activity in the nucleus accumbens impairs learning extinction of conditioned fear. Learn Mem 2010; 17: 7175.

17. El Ghundi M, ODowd BF, George SR. Prolonged fear responses in mice lacking dopamine D1 receptor. Brain Res 2001; 892: 8693.

18. Cragg SJ, Rice ME. DAncing past the DAT at a DA synapse. Trends Neurosci 2004; 27:

270277.

19. Vandenbergh DJ, Persico AM, Hawkins AL, Grifn CA, Li X, Jabs EW et al. Human dopamine transporter gene (DAT1) maps to chromosome 5p15.3 and displays a VNTR. Genomics 1992; 14: 11041106.

20. Heinz A, Goldman D, Jones DW, Palmour R, Hommer D, Gorey JG et al. Genotype inuences in vivo dopamine transporter availability in human striatum. Neuropsychopharmacology 2000; 22: 133139.

21. Jacobsen LK, Staley JK, Zoghbi SS, Seibyl JP, Kosten TR, Innis RB et al. Prediction of dopamine transporter binding availability by genotype: a preliminary report. Am J Psychiatry 2000; 157: 17001703.

22. Fuke S, Suo S, Takahashi N, Koike H, Sasagawa N, Ishiura S. The VNTR polymorphism of the human dopamine transporter (DAT1) gene affects gene expression. Pharmacogenomics J 2001; 1: 152156.

23. Mill J, Asherson P, Browes C, DSouza U, Craig I. Expression of the dopamine transporter gene is regulated by the 30 UTR VNTR: Evidence from brain and lymphocytes using quantitative RT-PCR. Am J Med Genet 2002; 114: 975979.

24. VanNess SH, Owens MJ, Kilts CD. The variable number of tandem repeats element in DAT1 regulates in vitro dopamine transporter density. BMC Genet 2005; 6: 55.

25. van Dyck CH, Malison RT, Jacobsen LK, Seibyl JP, Staley JK, Laruelle M et al. Increased dopamine transporter availability associated with the 9-repeat allele of the SLC6A3 gene. J Nucl Med 2005; 46: 745751.

26. Matsumoto M, Weickert CS, Akil M, Lipska BK, Hyde TM, Herman MM et al. Catechol O-methyltransferase mRNA expression in human and rat brain: evidence for a role in cortical neuronal function. Neurosci 2003; 116: 127137.

27. Tunbridge EM, Harrison PJ, Weinberger DR. Catechol-o-methyltransferase, cognition, and psychosis: Val158Met and beyond. Biol Psychiatry 2006; 60: 141151.

28. Lachman HM, Papolos DF, Saito T, Yu YM, Szumlanski CL, Weinshilboum RM. Human catechol-O-methyltransferase pharmacogenetics: description of a functional polymorphism and its potential application to neuropsychiatric disorders. Pharmacogenetics 1996; 6: 243250.

29. Chen J, Lipska BK, Halim N, Ma QD, Matsumoto M, Melhem S et al. Functional analysis of genetic variation in catechol-O-methyltransferase (COMT): effects on mRNA, protein, and enzyme activity in postmortem human brain. Am J Hum Genet 2004; 75: 807821.

30. Morrow BA, Elsworth JD, Rasmusson AM, Roth RH. The role of mesoprefrontal dopamine neurons in the acquisition and expression of conditioned fear in the rat. Neurosci 1999; 92: 553564.

31. Lonsdorf TB, Weike AI, Nikamo P, Schalling M, Hamm AO, Ohman A. Genetic gating of human fear learning and extinction: possible implications for gene-environment interaction in anxiety disorder. Psychol Sci 2009; 20: 198206.

32. Raczka KA, Gartmann N, Mechias ML, Reif A, Buchel C, Deckert J et al. A neuropeptide S receptor variant associated with overinterpretation of fear reactions: a potential neurogenetic basis for catastrophizing. Mol Psychiatry 2010; 15: 1045, 10671074.

33. McClure SM, Berns GS, Montague PR. Temporal prediction errors in a passive learning task activate human striatum. Neuron 2003; 38: 339346.

34. ODoherty JP, Dayan P, Friston K, Critchley H, Dolan RJ. Temporal difference models and reward-related learning in the human brain. Neuron 2003; 38: 329337.

35. Seymour B, ODoherty JP, Dayan P, Koltzenburg M, Jones AK, Dolan RJ et al. Temporal difference models describe higher-order learning in humans. Nature 2004; 429: 664667.

36. Menon M, Jensen J, Vitcu I, Graff-Guerrero A, Crawley A, Smith MA et al. Temporal difference modeling of the blood-oxygen level dependent response during aversive conditioning in humans: effects of dopaminergic modulation. Biol Psychiatry 2007; 62: 765772.

37. Seymour B, Daw N, Dayan P, Singer T, Dolan R. Differential encoding of losses and gains in the human striatum. J Neurosci 2007; 27: 48264831.

38. Petrovic P, Kalisch R, Pessiglione M, Singer T, Dolan RJ. Learning affective values for faces is expressed in amygdala and fusiform gyrus. Soc Cogn Affect Neurosci 2008; 3: 109118.

39. Schiller D, Levy I, Niv Y, LeDoux JE, Phelps EA. From fear to safety and back: reversal of fear in the human brain. J Neurosci 2008; 28: 1151711525.

40. Dayan P. Classical conditioning and reinforcement learning. In: Dayan P, Abbott LF (eds). Theoretical Neuroscience. MIT Press: Cambridge, MA, 2001, pp 331.

Translational Psychiatry

Fear extinction and DAT1 KA Raczka et al

8

41. Friston KJ, Ashburner JT, Kiebel SJ, Nichols TE, Penny WD. Statistical Parametric Mapping: the Analysis of Functional Brain Images. Academic Press: San Diego, CA, 2007.

42. ODoherty JP. Reward representations and reward-related learning in the human brain: insights from neuroimaging. Curr Opin Neurobiol 2004; 14: 769776.

43. Spoormaker VI, Andrade KC, Schroter MS, Sturm A, Goya-Maldonado R, Samann PG et al. The neural correlates of negative prediction error signaling in human fear conditioning. Neuroimage 2011; 54: 22502256.

44. Bilder RM, Volavka J, Lachman HM, Grace AA. The catechol-O-methyltransferase polymorphism: relations to the tonic-phasic dopamine hypothesis and neuropsychiatric phenotypes. Neuropsychopharmacology 2004; 29: 19431961.

45. Meyer-Lindenberg A, Weinberger DR. Intermediate phenotypes and genetic mechanisms of psychiatric disorders. Nat Rev Neurosci 2006; 7: 818827.

46. Yacubian J, Sommer T, Schroeder K, Glascher J, Kalisch R, Leuenberger B et al. Gene-gene interaction associated with neural reward sensitivity. Proc Natl Acad Sci USA 2007; 104: 81258130.

47. Solomon RL, Corbit JD. An opponent-process theory of motivation. I. Temporal dynamics of affect. Psychol Rev 1974; 81: 119145.

48. Dickinson A, Pearce JM. Inhibitory interactions between appetitive and aversive stimuli. Psychol Bull 1977; 84: 690711.

49. Mueller D, Bravo-Rivera C, Quirk GJ. Infralimbic D2 receptors are necessary for fear extinction and extinction-related tone responses. Biol Psychiatry 2010; 68: 10551060.

50. Ponnusamy R, Nissim HA, Barad M. Systemic blockade of D2-like dopamine receptors facilitates extinction of conditioned fear in mice. Learn Mem 2005; 12: 399406.

51. Hikind N, Maroun M. Microinfusion of the D1 receptor antagonist, SCH23390 into the IL but not the BLA impairs consolidation of extinction of auditory fear conditioning. Neurobiol Learn Mem 2008; 90: 217222.

52. Milad MR, Goldstein JM, Orr SP, Wedig MM, Klibanski A, Pitman RK et al. Fear conditioning and extinction: inuence of sex and menstrual cycle in healthy humans. Behav Neurosci 2006; 120: 11961203.

53. Ressler KJ, Rothbaum BO, Tannenbaum L, Anderson P, Graap K, Zimand E et al. Cognitive enhancers as adjuncts to psychotherapy: use of D-cycloserine in phobic individuals to facilitate extinction of fear. Arch Gen Psychiatry 2004; 61: 11361144.

54. Grillon C. D-cycloserine facilitation of fear extinction and exposure-based therapy might rely on lower-level, automatic mechanisms. Biol Psychiatry 2009; 66: 636641.

55. McNally GP, Westbrook RF. Opioid receptors regulate the extinction of Pavlovian fear conditioning. Behav Neurosci 2003; 117: 12921301.

56. McNally GP, Pigg M, Weidemann G. Opioid receptors in the midbrain periaqueductal gray regulate extinction of pavlovian fear conditioning. J Neurosci 2004; 24: 69126919.

Translational Psychiatry is an open-access journal published by Nature Publishing Group. This work is licensed under the Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/

Supplementary Information accompanies the paper on the Translational Psychiatry website (http://www.nature.com/tp

Web End =http://www.nature.com/tp)

Translational Psychiatry