Increased Accuracy to c-Fos-Positive Neuron Counting

Abstract

There is not a described method to count the core label of c-Fos-positive neurons, avoiding false-positive and false-negative results. The aim of this manuscript is to provide guidelines for a secure and accurate method to calculate a threshold to select which core of c-Fos-positive neurons marked by immunofluorescence has to be scored. A background percentage was calculated by dividing the intensity value (0 to 255) of the core of c-Fos-positive neurons by its surrounding background from the 8-bit images obtained in a previous study. Using the background percentage from 20% up to 98%, raising 2% once for each score, as threshold to choose which core has to be counted, a script was written for the R program to count the number of the c-Fos-positive neurons and the comparison between control and experimental groups. The differences of the average number of the core counted c-Fos-positive neurons between control and experimental groups, at all thresholds studied, showed a rising value related to an increase of the background percentage threshold as well as a decrease of its $p$ value related to an increase of the threshold of background percentage. For the smallest thresholds (high intensity of label), the differences between groups are suppressed (false negative). However, for the biggest thresholds (nonspecific label), these differences are always the same (false positive). Therefore, to avoid the false-negative and the false-positive values, it was chosen as the threshold of 62% the inflection point of the linear regression, which is equally different from the biggest and smallest values of the differences between groups.

This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

1. Introduction

Fos protein is composed of 380 amino acids that form a heterodimer with Jun to bind itself to the DNA and promote a variety of actions in the cell. It can be produced 15 minutes after stimulation because it is a product of the immediate early gene c-Fos [1]. Its expression is an indirect marker for neuronal activity because it is frequently expressed after the neurons spike an action potential [2, 3]. Likewise, a conditional fear can inhibit the c-Fos mRNA expression in the animals stimulated to produce Fos protein. Such suppression indicates the repression of neural activity [4].

Immunofluorescence to Fos protein is a widespread procedure used by neuroscientists to study both excitatory [2, 3] and inhibitory [4] activations in the neural circuits [2, 5–7]. It has been used in qualitative studies to show the activation of the neurons in the central nervous system after a specific stimulus [8, 9]. Besides, it has also been used in quantitative studies, in which the number of c-Fos-positive neurons is counted to evaluate the recruitment of the neurons in the neural circuit nuclei. These quantitative studies have been used to evaluate the action of new medicines in the central nervous system, mainly to include a test to analgesic approach [10–14].

The methods used to core count c-Fos-positive neurons are not well described in the literature. The vast majority of the works do not provide details on the method used to core count c-Fos-positive neurons [6, 15].

To count the core label of c-Fos-positive neurons in the immunofluorescence images, it is necessary to overcome two issues: the variability of the core label intensity between different images (outside variability) and the gradient of the core label intensity inside of each image (inside variability). To control the influence of both the outside and the inside variabilities during the counting process of the core labels of c-Fos-positive neurons, there is not a described method capable of avoiding the false-positive and false-negative results. Aiming to fill that gap in the literature, this manuscript provides a secure and accurate method to count the core label of c-Fos-positive neurons marked by immunofluorescence.

2. Materials and Methods

2.1. Experimental Protocol

From a previous study [16], images of the experimental and control groups were selected to be used on the core count protocol proposed by this study. The core label of c-Fos-positive neurons was counted using several thresholds, from 20% up to 98%, raising 2% once, to compare control and experimental groups. After that, it was applied the inflection point of the linear regression, which is equally different from the biggest and smallest values of the differences between groups.

2.2. Image Acquisition

The 8-bit images were acquired with a CM10 camera of the IX81 Olympus microscopy, using 20x objective and fluorescence filter (U-FBWA) to Alexa Fluor 488. The frequency of the opening of the diaphragm was kept at 2 seconds.

2.3. Image Analysis

On each acquired image from the ventrolateral region of the PAG [16], the core of c-Fos-positive neurons was clicked, as well as the background around it, using a Cell Counter (Written by Kurt De Vos) plug-in for Image J software (NIH, USA). Subsequently, the intensity value (0 to 255) of the core of c-Fos-positive neurons was obtained, as well as the background around it, by clicking on the “Measure” button of the Cell Counter plug-in. These clicks on the core label and the background around it on the acquired image occurred, even when the core was overly weakly labeled. After that, the intensity value of the background around the core label was divided by the intensity value of the core label. As the intensity value of the core label was always higher than the intensity value of the background around it, the quotient obtained by that division was expressed by a percentage value which represents the background percentage of the core label. Therefore, the background percentage represents the quotient between the intensity of the background label (around the core label) and the intensity of the core label. Then, the R program was used to run a script, to write for the authors, to calculate the background percentage, and to count the marked c-Fos-positive neurons on the all thresholds (from 20% up to 98%, raising 2% once, of the background percentage) proposed in this study.

2.4. Statistical Analysis

All the calculations were made using the R statistic program [17]. The data obtained were expressed as the $\text{mean}\pm \text{standard}\text{error}$ of the mean, and the differences were evaluated through the Wilcoxon test [18]. Those with a $p$ value less than 0.05 were considered as significant.

3. Results

The background of image 1(b) has an intensity value (around 99) bigger than several core labels of image 1(a) (55 and 44). These differences of the intensity value between these images (1(a) and 1(b)) characterize the outside variability. In Figure 1, the intensity value of the core label of c-Fos-positive neurons inside picture 1(a) ranges from 44 to 178, and its background is around 50. This gradient of the core label intensity characterizes the inside variability, at some parts of the image 1(a); the smallest intensity label of the core can be smaller than its background label. Because of the variability (outside and inside), this work proposes to use the background percentage, obtained by division of the intensity value of the background image by core label, as a threshold to choose which core has to be scored. It was used several background percentages as a threshold, from 20% up to 98%, raising 2% once, to count the core label of the c-Fos-positive neurons (Figure 2).

[figures omitted; refer to PDF]

[figures omitted; refer to PDF]

Figure 2 shows the c-Fos-positive neurons (arrows) on the PAG in the mouse brain after either vehicle (a) or MEKR (b) treatment. The number of c-Fos-positive neurons activated on the PAG is raised by the MEKR ($p<0.05$) treatment, as demonstrated in previous research by our group [16]. Using the script below, the number of c-Fos-positive neurons was counted.

Code 1: Script for R proggram.

  #install.packages("ggplot2")

  #install.packages("Rmisc")

  #install.packages("rio")

  require(ggplot2)

  require(Rmisc)

  require(rio)

  setwd("C:\\Users\\PC\\Dropbox\\Trabalho\\W")

  data=read.table("Results.txt",head=TRUE) #Import Result.txt file

  #data=read.table(file.choose(),head=TRUE) #Import Result.txt file

  $\text{sig}=0.05$ #Significance Level

  threshold=seq(from=0.2, to=0.99, by=0.02) #Threshold limits

  ####### Organizes the data set, performs the neuron count and applies the Wilcoxon test ######

  i=1

  while(i <= nrow(data)){

    data[i,6]=data[i+1,5]/data[i,5]

    $i=i+2$

  }

  data<- subset(data, !is.na(data[,6]))

  l$=1$

  for($j$ in 1:length(threshold)){

    for($i$ in 1:nrow(data)){

     if($\text{data}i,6<=\text{threshold}j$)

      $\text{data}i,7=1$

     else

      $\text{data}i,7=0$

    }

    aggre1=aggregate(data[,7], by=list(data[,1],data[,2]), sum)

    if($j==1$){

     result.graph=data.frame(matrix(NA, ncol=3, nrow=nrow(aggre1)$\ast$length(threshold)))

     result.count=data.frame(matrix(NA, ncol=(length(threshold)+2), nrow=nrow(aggre1)))

     result.wilcoxon=data.frame(matrix(NA, ncol=3, nrow=length(threshold)))

     result.count[,1]=aggre1[,1];result.count[,2]=aggre1[,2]

     colnames(result.graph)=c("Animals","X","Threshold")

     colnames(result.wilcoxon)=c("Threshold","W","p-value")

     colnames(result.count)=c("Type","Slice",paste("Threshold", threshold$\ast$100,sep=""))

    }

    result.count[,j+2]=aggre1[,3]

    for($k$ in 1:nrow(aggre1)){

     if($\text{aggre}1k,1==1$)

      result.graph[l,1]="EXP"

     else

      result.graph[l,1]="CTRL"

     result.graph[l,2]=aggre1[k,3]

    result.graph[l,3]=threshold[j]$\ast$100

    w=wilcox.test(aggre1[aggre1[,1]==1,3], aggre1[aggre1[,1]==2,3],alternative="two.side")

    result.wilcoxon[j,1]=threshold[j]$\ast$100; result.wilcoxon[j,2]=w$statistic; result.wilcoxon[j,3]=w$$p$.value

    $l=l+1$

    }

  }

  #Export file with Wilcoxon test results

  export(result.wilcoxon, file = "Result_Wilcoxon.xlsx", overwrite = TRUE)

  #resultados das contagens

  export(result.count, file = "Result_Contagem.xlsx", overwrite = TRUE)

  ################### Averages Chart ###################

  tgc <- summarySE(result.graph, measurevar="X", groupvars=c("Animals","Threshold"))

  ggplot(tgc, aes(x=Threshold, y=X, colour=Animals)) +

    labs(x="Background percentage threshold (%)") +

    labs(y="Average number of the core count of c-Fos-positive neurons") +

    geom_errorbar(aes(ymin=X-se, ymax=X+se), colour="black", width=1) +

    theme_bw() +

    geom_point(aes(shape=Animals),size=3, colour="black") +

    annotate("text", x = result.wilcoxon[result.wilcoxon[,3]<sig,1],>

      y = tgc[tgc$Threshold >= min(result.wilcoxon[result.wilcoxon[,3]<sig,1])>

        tgc$Threshold <= max(result.wilcoxon[result.wilcoxon[,3]<sig,1])>

       tgc[tgc$Threshold >= min(result.wilcoxon[result.wilcoxon[,3]<sig,1])>

        tgc$Threshold <= max(result.wilcoxon[result.wilcoxon[,3]<sig,1])>

      label = "$\ast$", cex=6)

  ggsave(filename = "Average number of the core count of c-Fos-positive neurons.tiff",

      $\text{scale}=2.2$, $\text{dpi}=300$)

  ################### Graphic $p$-valor ###################

  tiff(filename = "$p$ value vs Background percentage threshold.tiff")

  plot(result.wilcoxon[,3] ~ result.wilcoxon[,1], type="p", pch=20,

    xlab="Background percentage threshold (%)", ylab="$p$ value")

  lines(c(sig, sig)~c(result.wilcoxon[1],result.wilcoxon[length(result.wilcoxon[,1]),1]), lty=3)

  dev.off()

  ############# Nonlinear Regression #########################

  tgc <- summarySE(result.graph, measurevar="X", groupvars=c("Animals","Threshold"))

  tgc2=subset(tgc, tgc$Animals=="EXP")

  reg=as.data.frame(matrix(NA, ncol=4, nrow=3))

  colnames(reg)=c("Parameters", "Estimate", "Std. Error", "Pr(>|t|)")

  y=tgc2$X

  x=tgc2$Threshold

  $\text{result}=\text{nls}(y~\underline{\text{A}}/1+\exp -\text{B}0+\text{B}1\ast x$,$\text{start}=\text{list}A=100,\text{B}0=-5,\text{B}1=0.1)$

  mod=summary(result)

  reg[,1]=rownames(mod$coefficients)

  reg[,2]=mod$coefficients[,1]

  reg[,3]=mod$coefficients[,2]

  reg[,4]=mod$coefficients[,4]

  reg[1, 5]="Inflection Point"

  reg[1, 6]="Count of c-Fos-positive neurons"

  reg[1, 7]="Background percentage threshold (%)"

  reg[2, 6]=mod$coefficients[1]/2

  reg[2, 7]=-mod$coefficients[2]/mod$coefficients[3]

  export(reg, file = "Result_Non_Linear_Regression.xlsx", overwrite = TRUE)

  ##

  tgc2[,8]=mod$coefficients[1]/(1+exp(-(mod$coefficients[2]+mod$coefficients[3]$\ast$x)))

  colnames(tgc2)[8]="est"

  # Gr?fico com todos os grupos

  ggplot(tgc, aes(x=Threshold, y=X, colour=Animals)) +

    labs(x="Background percentage threshold (%)") +

    labs(y="Average number of the core count of c-Fos-positive neurons") +

    geom_errorbar(aes(ymin=X-se, ymax=X+se), colour="black", width=1) +

    theme_bw() +

    xlim(min(tgc2$Threshold)-2, max(tgc2$Threshold)+2) +

    geom_point(aes(shape=Animals),size=3, colour="black") +

    geom_line(data = tgc2, aes(x = Threshold, y = est), size =1.5, color="red") +

    geom_segment(aes(x = reg[2, 7], y = 0, xend = reg[2, 7], yend = reg[2, 6]),

        size=1, colour="red", linetype = "dashed") +

    geom_segment(aes(x = min(tgc2$Threshold)-2, y = reg[2, 6], xend = reg[2, 7],

        yend = reg[2, 6]),$\text{size}=1$, colour="red", linetype = "dashed")

  ggsave(filename = "Nonlinear Regression Adjustment.tiff",

      $\text{scale}=2.2$, $\text{dpi}=300$)

In Figure 3, the average number of the core counts of the c-Fos-positive neurons in the PAG for all calculated background percentage thresholds (from 20% up to 98%, raising 2%) was plotted. Between control and experimental groups shows, on this plot, a rise in the difference of the average number of the core counts for c-Fos-positive neurons related to the increase of the background threshold, except for both extremes of the curves. For the smallest thresholds (high intensity of label), the differences between groups are suppressed (false negative). However, for the biggest thresholds (nonspecific label), these differences are always the same (false positive).

[figure omitted; refer to PDF]

After evaluating the core count average number, the value of $p$ on the Wilcoxon test for the differences between control and experimental groups at all thresholds was studied (Figure 4). It was observed a decrease in the $p$ values related to the increase in the threshold of the background percentage.

[figure omitted; refer to PDF]

Therefore, to avoid the false-negative and the false-positive values, it was chosen as the threshold of 62% (Figure 5). This value represents the inflection point of the linear regression, which is equally different from the biggest and smallest values of the differences between groups.

[figure omitted; refer to PDF]

4. Discussion

The proposed method in this manuscript uses mathematical language to calculate a safe threshold to the core count of c-Fos-positive neurons marked by immunofluorescence.

The results demonstrated that, for the weak core labels, the background percentage showed high intensity values (around 80%). On the other hand, for the strong core labels, the background percentage showed low intensity values (around 20%). Hence, to select which core label had to be counted, the background percentage was used as a threshold. Thereby, many counts of the core label were made with a threshold from 20% to 98%, raising 2% once. For each threshold, the amount of the core label of c-Fos-positive neurons for the control was measured, as well as for the experimental groups. Moreover, the $p$ value was calculated using a Wilcoxon test and analyzed whether there was a significant difference between the groups studied.

In Figure 3, that difference between control and experimental groups was observed because the increase on the average number of the core counted c-Fos-positive neurons was more intense in experimental than in the control group. This was an expected result because the control group does not have many c-Fos-positive neurons, whereas the experimental group has a lot of c-Fos-positive neurons. This higher core counts in the experimental group were due to the pharmacological effect produced by MEKR [16].

For thresholds ranging from 20% to 30%, the average number of the core count was not significantly different, whether it was compared to the control against the experimental groups (Figure 3), and the $p$ value was higher than 0.05 (Figure 4). For these threshold levels, only a few cores are counted in the experimental group because there are not many neurons with extraordinarily high intensity of the core label. These results indicate that counting only neurons with high intensity of the core label can provoke a type II error, i.e., a “false-negative” result.

For thresholds from 78% to 98% (Figure 3), the average number of the core count showed a significant difference between control and experimental groups with an extremely small $p$ value ($p<0.001$). However, the comparison inside the same group, either the control group or the experimental group, did not show differences in the average number of the core count. The probable reason for it is the weak stain of the core label for this threshold range. For a weak stain, it is difficult to differentiate the core label of the background around it. In Figure 1(a), the intensity of the background percentage around the core label was indicated by a circle for different thresholds. The core label cannot be clearly differentiated from the background around it for thresholds from 78% to 98%. Therefore, using this threshold range makes it hard to differentiate specific from nonspecific binding from the artifact labels. These difficulties can provoke a type I error, i.e., a “false-positive” result.

“False-negative” results are equally damaging to science as “false-positive” results, since both can generate a cascade of errors producing consequences that are often incalculable [19].

To choose which threshold should be the most indicated for the core count, a linear correlation was made, and the results showed its inflection point as threshold 62% in Figure 5.

In the literature, there are several methods to core count c-Fos-positive neurons. The most frequently used consists of counting cells by clicking on them, using the common sense to establish a threshold to separate the background from the specific label [6]. This method cannot be safe if the differences between experimental and control groups are tiny.

In our method to core count c-Fos-positive neurons, we also click on the core label of the neurons to count c-Fos-positive neurons. Thus, all core labels are clicked, even if the core label is not so intense. However, unlike the traditional methods, our method resides at the decision of which core has to be counted and which does not. In our method, a mathematical threshold is established to decide which core has to be counted and which does not. That way, all nucleus cells are clicked, but not all of them are counted. Just the cores with a background intensity of 62% or lower than the intensity of the core label is counted.

Other works have used a threshold limit with a fixed value [8, 20] to choose which cores have to be counted and which does not. That is a good strategy, but using the same threshold for all immunofluorescence pictures cannot be so efficient because the value of the background usually has a considerable variation, even if camera’s diaphragm is kept at the same frequency during the acquisition process. Because the method described in this paper uses a percentage background of the label as value, not a fixed value, all variation in the intensity of the core label can be normalized, avoiding “false-positive and false-negative” results.

The proposed method allows to evaluate more accurately the core count of c-Fos-positive neurons marked by immunofluorescence, allowing a reduction of the number of animals.

This manuscript offers an additional tool to improve the production of neuroscience studies, for which animals and humans will benefit.

5. Conclusions

Figure 5 shows that for background percentage values greater than 62%, there is no increase in the difference between the experimental and control groups. Therefore, 62% is the maximum threshold that allows counting the c-Fos-positive neurons safely. From that, considering the varying average number of the core label of c-Fos-positive neurons and the behavior of the $p$ value of Wilcoxon test on the threshold of the background studied, it should be advisable to use a threshold of 62% of the background percentage (Figure 5) to select which cores has to be counted.

Disclosure

The funding body had no role in the design, collection, analysis, and interpretation of data in this study or in writing the manuscript.

Acknowledgments

This work was supported by grants from the Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) from Brazil (Grant number 423394/2018-5). The Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) supports postgraduate programs in Brazil.

References

[1] T. D. Halazonetis, K. Georgopoulos, M. E. Greenberg, P. Leder, "c-Jun dimerizes with itself and with c-Fos, forming complexes of different DNA binding affinities," Cell, vol. 55, pp. 917-924, DOI: 10.1016/0092-8674(88)90147-X, 1988.

[2] M. Dragunow, R. Faull, "The use of _ c-fos _ as a metabolic marker in neuronal pathway tracing," Journal of Neuroscience Methods, vol. 29 no. 3, pp. 261-265, DOI: 10.1016/0165-0270(89)90150-7, 1989.

[3] M. VanElzakker, R. D. Fevurly, T. Breindel, R. L. Spencer, "Environmental novelty is associated with a selective increase in Fos expression in the output elements of the hippocampal formation and the perirhinal cortex," Learning & Memory, vol. 15, pp. 899-908, DOI: 10.1101/lm.1196508, 2008.

[4] H. E. W. Day, E. M. Kryskow, T. J. Nyhuis, L. Herlihy, S. Campeau, "Conditioned fear inhibits c-fos mRNA expression in the central extended amygdala," Brain Research, vol. 1229, pp. 137-146, DOI: 10.1016/j.brainres.2008.06.085, 2008.

[5] J. S. S. Quintans, R. G. Brito, P. G. V. Aquino, P. H. B. França, O. S. Siqueira-Lima, A. E. G. Santana, E. A. N. Ribeiro, M. J. Salvador, J. X. Araújo-Júnior, L. J. Quintans-Júnior, "Antinociceptive activity of Syzygium cumini leaves ethanol extract on orofacial nociception protocols in rodents," Pharmaceutical Biology, vol. 52 no. 6, pp. 762-766, DOI: 10.3109/13880209.2013.870582, 2014.

[6] R. E. Szawka, M. O. Poletini, C. M. Leite, M. P. Bernuci, B. Kalil, L. B. D. Mendonça, R. O. G. Carolino, C. V. V. Helena, R. Bertram, C. R. Franci, J. A. Anselmo-Franci, "Release of norepinephrine in the preoptic area activates anteroventral periventricular nucleus neurons and stimulates the surge of luteinizing hormone," Endocrinology, vol. 154 no. 1, pp. 363-374, DOI: 10.1210/en.2012-1302, 2013.

[7] H. van Dam, M. Castellazzi, "Distinct roles of Jun : Fos and Jun : ATF dimers in oncogenesis," Oncogene, vol. 20 no. 19, pp. 2453-2464, DOI: 10.1038/sj.onc.1204239, 2001.

[8] L. M. Parker, H. A. Damanhuri, S. P. S. Fletcher, A. K. Goodchild, "Hydralazine administration activates sympathetic preganglionic neurons whose activity mobilizes glucose and increases cardiovascular function," Brain Research, vol. 1604, pp. 25-34, DOI: 10.1016/j.brainres.2015.01.049, 2015.

[9] J. Petrak, J. Bundzikova, Z. Pirnik, B. Mravec, R. Kvetnansky, A. Kiss, "Fos expression in tyrosine hydroxylase containing hypothalamic neurons in CRH-KO mice: effect of immobilization stress," Endocrine Regulations, vol. 44 no. 3, pp. 89-99, DOI: 10.4149/endo_2010_03_89, 2010.

[10] J. Quintans, R. Barreto, W. de Lucca, C. Villarreal, C. Kaneto, M. Soares, A. Branco, J. Almeida, A. Taranto, A. Antoniolli, R. Freitas, L. Quintans-Júnior, "Evidence for the involvement of spinal cord-inhibitory and cytokines-modulatory mechanisms in the anti-hyperalgesic effect of hecogenin acetate, a steroidal sapogenin-acetylated, in mice," Molecules, vol. 19 no. 6, pp. 8303-8316, DOI: 10.3390/molecules19068303, 2014.

[11] R. G. Brito, P. L. Santos, D. S. Prado, M. T. Santana, A. A. S. Araújo, L. R. Bonjardim, M. R. V. Santos, W. de Lucca Júnior, A. P. Oliveira, L. J. Quintans-Júnior, "Citronellol reduces orofacial nociceptive behaviour in mice–evidence of involvement of retrosplenial cortex and periaqueductal grey areas," Basic & Clinical Pharmacology & Toxicology, vol. 112 no. 4, pp. 215-221, DOI: 10.1111/bcpt.12018, 2013.

[12] S. S. Nascimento, A. A. S. Araújo, R. G. Brito, M. R. Serafini, P. P. Menezes, J. M. DeSantana, W. Júnior, P. Alves, A. Blank, R. Oliveira, A. Oliveira, R. Albuquerque-Júnior, J. Almeida, L. Quintans-Júnior, "Cyclodextrin-complexed Ocimum basilicum leaves essential oil increases Fos protein expression in the central nervous system and produce an antihyperalgesic effect in animal models for fibromyalgia," International Journal of Molecular Sciences, vol. 16 no. 1, pp. 547-563, DOI: 10.3390/ijms16010547, 2015.

[13] S. S. Nascimento, E. A. Camargo, J. M. DeSantana, A. A. S. Araújo, P. P. Menezes, W. Lucca-Júnior, R. L. C. Albuquerque-Júnior, L. R. Bonjardim, L. J. Quintans-Júnior, "Linalool and linalool complexed in β -cyclodextrin produce anti-hyperalgesic activity and increase Fos protein expression in animal model for fibromyalgia," Naunyn-Schmiedeberg's Archives of Pharmacology, vol. 387 no. 10, pp. 935-942, DOI: 10.1007/s00210-014-1007-z, 2014.

[14] P. S. Siqueira-Lima, A. A. S. Araújo, A. M. Lucchese, J. S. S. Quintans, P. P. Menezes, P. B. Alves, W. de Lucca Júnior, M. R. V. Santos, L. R. Bonjardim, L. J. Quintans-Júnior, "β -Cyclodextrin complex containing Lippia grata leaf essential oil reduces orofacial nociception in mice–evidence of possible involvement of descending inhibitory pain modulation pathway," Basic & Clinical Pharmacology & Toxicology, vol. 114 no. 2, pp. 188-196, DOI: 10.1111/bcpt.12145, 2014.

[15] R. A. Holland, J. J. Leonard, N. A. Kensey, P. A. Hannikainen, B. C. De Jonghe, "Cisplatin induces neuronal activation and increases central AMPA and NMDA receptor subunit gene expression in mice," Physiology & Behavior, vol. 136, pp. 79-85, DOI: 10.1016/j.physbeh.2014.02.038, 2014.

[16] M. S. Melo, R. G. Brito, P. L. Santos, P. C. L. Nogueira, V. R. S. Moraes, M. C. P. Matos, J. N. S. Ferro, E. O. Barreto, W. Lucca Junior, M. A. Botelho, L. J. Quintans Junior, "Involvement of cerebral nervous system areas and cytokines on antihyperalgesic and anti-inflammatory activities of Kielmeyera rugosa Choisy (Calophyllaceae) in rodents," Phytotherapy Research, vol. 28 no. 12, pp. 1806-1815, DOI: 10.1002/ptr.5205, 2014.

[17] R Core Team, "R: a language and environment for statistical computing," . Sep 2017, https://www.gbif.org/tool/81287/r-a-language-and-environment-for-statistical-computing .

[18] D. F. Bauer, "Constructing confidence sets using rank statistics," Journal of the American Statistical Association, vol. 67, pp. 687-690, DOI: 10.1080/01621459.1972.10481279, 1972.

[19] K. G. Shojania, J. M. Grimshaw, "Evidence-based quality improvement: the state of the science," Health Affairs, vol. 24, pp. 138-150, DOI: 10.1377/hlthaff.24.1.138, 2005.

[20] H. Ikeda, S. Takasu, K. Murase, "Contribution of anterior cingulate cortex and descending pain inhibitory system to analgesic effect of lemon odor in mice," Molecular Pain, vol. 10,DOI: 10.1186/1744-8069-10-14, 2014.