Associations of a Metal Mixture Measured in Multiple Biomarkers with IQ: Evidence from Italian Adolescents Living near Ferroalloy Industry

Abstract

Background: Research on the health effects of chemical mixtures has focused mainly on early life rather than adolescence, a potentially important developmental life stage.

Objectives: We examined associations of a metal mixture with general cognition in a cross-sectional study of adolescents residing near ferromanganese industry, a source of airborne metals emissions.

Methods: We measured manganese (Mn), lead (Pb), copper (Cu), and chromium (Cr) in hair, blood, urine, nails, and saliva from 635 Italian adolescents 10–14 years of age. Full-scale, verbal, and performance intelligence quotient (FSIQ, VIQ, PIQ) scores were assessed using the Wechsler Intelligence Scale for Children-III. Multivariable linear regression and Bayesian kernel machine regression (BKMR) were used to estimate associations of the metal mixture with IQ. In secondary analyses, we used BKMR’s hierarchical variable selection option to inform biomarker selection for Mn, Cu, and Cr.

Results: Median metal concentrations were as follows: hair Mn, 0.08μg/g 0.08μg/g ; hair Cu, 9.6μg/g ; hair Cr, 0.05μg/g ; and blood Pb, 1.3μg/dL . Adjusted models revealed an inverted U-shaped association between hair Cu and VIQ, consistent with Cu as an essential nutrient that is neurotoxic in excess. At low levels of hair Cu (10th percentile, 5.4μg/g ), higher concentrations (90th percentiles) of the mixture of Mn, Pb, and Cr (0.3μg/g , 2.6μg/dL , and 0.1μg/g , respectively) were associated with a 2.9 (95% CI: −5.2 , −0.5 )–point decrease in VIQ score, compared with median concentrations of the mixture. There was suggestive evidence of interaction between Mn and Cu. In secondary analyses, saliva Mn, hair Cu, and saliva Cr were selected as the biomarkers most strongly associated with VIQ score.

Discussion: Higher adolescent levels of Mn, Pb, and Cr were associated with lower IQ scores, especially at low Cu levels. Findings also support further investigation into Cu as both beneficial and toxic for neurobehavioral outcomes.

Introduction

Recent progress in understanding how chemical mixtures affect health can be largely attributed to collaboration between scientific fields, innovation of statistical methods, and prioritization from funding agencies (Carlin et al. 2013; Taylor et al. 2016). Metals are an important class of chemicals in which to conduct mixtures research given that they are ubiquitous, commonly co-occur in the environment, and have interactive potential (Claus Henn et al. 2014; Valeri et al. 2017; Wright et al. 2006). Communities are exposed to mixtures of metals that may be both naturally occurring and from anthropogenic sources. Anthropogenic activities such as gasoline combustion, waste incineration and recycling, mining, fungicide application, and steel industry emissions, including those from ferroalloy plants, are known to emit metals into local environments (ATSDR 2012).

Manganese ferroalloys are metal mixtures primarily composed of iron (Fe) in combination with manganese (Mn) and other metals such as chromium (Cr), lead (Pb), copper (Cu), nickel (Ni), zinc (Zn), cadmium (Cd), and aluminum (Al) (Olsen et al. 2007). Most environmental epidemiology studies in communities living near ferromanganese industries have focused on Mn only, reporting associations with parkinsonism in adults and neurobehavioral decrements in children and the elderly, such as worse overall cognition, behavior, attention, learning and memory, olfactory function, and motor abilities (Bauer et al. 2017; Chiu et al. 2017; Haynes et al. 2015; Lucchini et al. 2007, 2014; Menezes-Filho et al. 2014; Rodrigues et al. 2018). Yet other metals in the ferroalloy mixture, such as Pb and Cu, are also known to be neurotoxic (Bulcke et al. 2017; Caito and Aschner 2017). Furthermore, prior studies have shown interactions between some of these metals on neurodevelopment (Claus Henn et al. 2012; Gorell et al. 1997; Lin et al. 2013; Liu et al. 2018; Menezes-Filho et al. 2018). Despite the presence of multiple metals in ferroalloy emissions that may have additive or synergistic associations with neurobehavioral outcomes, few prior studies have considered the health effects of exposures to multiple metals (Lucchini et al. 2012b; Menezes-Filho et al. 2018).

Neurobehavioral effects from exposure to environmental contaminants, such as metals, are understudied in adolescence and early adulthood although these are potentially important developmental life stages (Wasserman et al. 2018). Neurodevelopment occurs as a dynamic process from the in utero period to adulthood (e.g., ∼30 years of age) (Blakemore and Choudhury 2006; Sowell et al. 2001). In addition to the prenatal and early childhood periods, preadolescence and adolescence are periods of rapid growth and development during which the brain is sensitive to neurotoxicants. During this time frame, the brain changes dramatically via synaptic pruning, myelination, neuronal transmission, and neural circuitry restructuring (Blakemore and Choudhury 2006; Giedd et al. 1999; Wahlstrom et al. 2010). These processes lead to refinement in brain areas, particularly the prefrontal cortex and hippocampus, which are important for cognitive abilities such as higher-order executive functioning (Sander et al. 2012).

In this article, we examine the association of a mixture of Mn, Pb, Cu, and Cr with cognition in adolescents living near varied ferromanganese industrial activity that utilizes and emits these metals. Our study investigated this mixture association during adolescence, a potentially critical window for neurodevelopment. Data analysis included both traditional regression and Bayesian kernel machine regression (BKMR)—a statistical approach developed to assess the health effects of environmental mixtures (Bobb et al. 2015)—to estimate individual, joint, and interactive associations. As a secondary analysis, given the lack of scientific consensus on the best exposure biomarker for Mn, Cu, and Cr, the hierarchical variable selection option in BKMR was used to help inform the choice of metal biomarker.

Methods

Study Participants

Participants for this analysis were part of the ongoing Public Health Impact of Mixed Element exposure (PHIME) study, designed to investigate associations between multiple metal exposures from ferroalloy emissions and neurobehavior in adolescents 10–14 years of age. We have previously described the study design in detail (Lucchini et al. 2012a). Briefly, 720 participants were recruited from three demographically similar, but geographically distinct, sites in the province of Brescia, Italy, a region with varied ferroalloy activity: a) Bagnolo Mella, an area with currently active ferroalloy industry since 1974; b) Valcamonica, an area with historical ferroalloy production for over a century that ended in 2001; and c) Garda Lake, a tourist region with no history of ferroalloy activity (see Lucchini et al. 2007 for a map).

Participants were eligible for the study if a) they were living in the study area since birth; b) their families had lived in the study area since 1970; and c) their children were 10–14 years of age at the time of enrollment. Participants were excluded from the study if they a) had been diagnosed with neurological, hepatic, metabolic or endocrine diseases, or clinically evident hand/finger motor deficits; b) were currently prescribed psychoactive drugs or had psychiatric disturbances; c) had inadequately corrected visual deficits; or d) had ever received parenteral nutrition.

We enrolled 720 participants into the study: 312 participants in the first phase (2007–2010) and 408 in the second phase (2010–2014). The two phases reflect two waves of funding, but they were conducted by the same researchers using identical questionnaires and study protocols. The second phase included a third site (Bagnolo Mella), as well as measurement of the Home Observation Measurement of the Environment (HOME) Short Form, a measure of cognitive stimulation and emotional support in the home (National Longitudinal Surveys 1979). The second phase also included measurement of metals concentrations in additional biomarkers (saliva, urine, fingernails). Eligible children received a detailed description of the study procedures before consenting to participate. The institutional review boards at the Ethical Committee of the Public Health Agency of Brescia, the Icahn School of Medicine at Mount Sinai, and the University of California, Santa Cruz approved all PHIME study protocols.

Metals Concentrations in Biological Samples

Blood, hair, fingernails, urine, and saliva were collected from participants at 10–14 years of age, concurrent with neurobehavioral assessment. Methods of collection and analysis of biological samples have been previously described (Eastman et al. 2013; Lucchini et al. 2012a; Smith et al. 2007). Briefly, we collected venous whole blood samples using a 19-gauge butterfly catheter into lithium-heparin Sarstedt Monovette® Vacutainers; hair samples using stainless steel scissors (2–3cm section of hair from the nape of the neck, proximal to the scalp); fingernails using stainless steel nail clippers; spot urine samples in polyethylene containers; and passive saliva samples through a plastic straw into trace metal–free microfuge tubes after rinsing the mouth three times with ultrapure water and waiting 10 min. Hair and fingernail samples were cleaned of exogenous metal contamination using a validated method (Eastman et al. 2013; Lucas et al. 2015).

Concentrations of Mn, Pb, Cu, and Cr were measured in all samples using magnetic sector inductively coupled plasma mass spectrometry (Eastman et al. 2013; Lucchini et al. 2012a, 2012b; Smith et al. 2007). The analytical detection limits for all biomarkers were based on repeated measurements of procedural blanks on four separate analysis days (for details see Butler et al. 2019). Most of the biological measurements were above the limits of detection (LODs). Samples below the LOD were assigned a value of one-half the LOD (hair Mn, n=1 ; Mn urine, n=5 ; Mn nails, n=25 ; hair Cr, n=1 ; saliva Cr, n=1 ; Cr nails, n=31 ; Cu nails, n=2 ).

Neurobehavioral Outcomes

Neurobehavioral assessments were conducted in tandem with biological sampling. Trained neuropsychologists used the Italian language–validated version of the Wechsler Intelligence Scale for Children—3rd Edition (WISC-III) to assess cognitive and language development skills (Wechsler et al. 2006). Subtests from the WISC-III were combined to create three composite intelligence quotient (IQ) scores: a) the verbal IQ (VIQ) composite score, which provides an overall assessment of language-based skills summarizing scores from the Comprehension, Information, Vocabulary, Similarities, and Arithmetic subtests; b) the performance IQ (PIQ) composite score, which includes primarily visual-motor (i.e., nonverbal) performance based on the Block Design, Picture Completion, Coding, Picture Arrangement, and Object Assembly subtests; and c) the full-scale IQ (FSIQ) composite score, which summarizes VIQ and PIQ, encompassing all 10 subtests. Age-adjusted standard scores were used in the analyses.

Covariates

Sociodemographic factors were collected using a standardized questionnaire administered by trained researchers in person or via telephone. Information from the questionnaire included area of residence (Bagnolo Mella, Valcamonica, Garda Lake); birth order (first, second, third, or higher); participant alcohol consumption (yes, no); age and sex of the participant, and parent education and occupation. We categorized participants into low, medium, or high socioeconomic status (SES) based on methodology developed in Italy that combines parent education and occupation (Cesana et al. 1995; Lucchini et al. 2012b). Hemoglobin (in grams per deciliter) was measured in the same blood samples collected for metals analysis and was considered a proxy for iron status, which may influence Mn and Pb absorption and neurodevelopment (Park et al. 2013; Smith et al. 2013).

Statistical Analysis

We performed univariate and bivariate analyses for each variable. Metals concentrations were natural log (ln) transformed to satisfy model assumptions of normality of residuals and then standardized. Bivariate correlations between metals concentrations were examined using Spearman’s rank correlation coefficients. We used age-adjusted WISC IQ scores (FSIQ, VIQ, and PIQ) and modeled them as continuous outcome variables.

There is a lack of consensus about which biological sample is the most consistent and valid biomarker of exposure for Mn, Cu, and Cr (Bertinato and Zouzoulas 2009; Coetzee et al. 2016; Jursa et al. 2018; Lukaski 1999; Viana et al. 2014). In primary analyses, we considered concentrations of these three metals in hair because hair has been used for these metals in previous environmental epidemiologic studies (Bertinato and Zouzoulas 2009; Bouchard et al. 2011; Haynes et al. 2015; Randall and Gibson 1989; Riojas-Rodríguez et al. 2010) and had the largest sample size in our data. In secondary analyses, we used a data-driven approach (described below) to select the biological sample to use in models with IQ. Concentrations of Pb measured in blood were used for all analyses, given its wide acceptance as a Pb biomarker.

Confounder selection.

We used directed acyclic graphs to examine potential confounders that might be associated with metals concentrations and cognition but are not causal intermediates or colliders (see Figure S1). Continuous covariates and exposures were modeled in a way that best represented the shape of the association with IQ (e.g., HOME score was modeled with an additional quadratic term) based on visual assessment of penalized splines (constrained to 4 knots) in adjusted generalized additive models (GAMs). All final models were adjusted for sex, SES (high, medium, low), hemoglobin (in grams per deciliter), HOME score, and HOME score squared (due to a nonlinear relationship with IQ).

Multiple imputation.

Some participants were missing data on the HOME score and some biomarkers. We used multiple imputation to impute missing values using chained equations with the MICE package in R (van Buuren and Groothuis-Oudshoorn 2011).

We imputed missing values for all participants (n=709 ) (White et al. 2011) and included in the imputation process variables thought to be related to the missing data, including outcomes, metals concentrations in biological as well as environmental samples (Lucchini et al. 2012a), and potential confounders (see Table S1 for list of variables used in imputation). We assumed data were missing at random, that is, that the missingness did not depend on unobserved data. We generated 40 imputed data sets. In our analytic data set, we included only participants with measured IQ and hair metals concentrations (n=635 ).

GAMs and multivariable linear regression models.

We first examined the shape of the ln-metals (hair Mn, hair Cu, hair Cr, blood Pb) and IQ associations using GAMs with penalized splines (knots=4 ), adjusting for previously selected confounders. In GAMs, to visualize the association with the outcome, we used standard multiple imputation methods to combine the point estimates of coefficients parameterizing the nonlinear terms from multiple GAM fits across imputed data sets, averaging results from the 40 imputations. We calculated standard errors using Rubin’s rule that combines the within- and between-imputation variances of these coefficient estimates (Rubin 2004). To plot the association between each ln-metal and IQ, we estimated nonlinear effect estimates across each metal’s range of values in 0.1 increments while setting all other continuous variables in the model at their median and as linear combinations of these averaged coefficient estimates and computed point-wise 95% confidence intervals (CIs) of these linear combinations accordingly (see Figure S2).

Based on adjusted GAMs, all ln-metals appeared to be linearly associated with IQ except Cu, which appeared to have an inverted U-shaped association. To test nonlinearity, we compared model fit using a likelihood ratio test from the MICE R package (van Buuren and Groothuis-Oudshoorn 2011) for multivariable linear models. We compared model fit between models with an additional quadratic term for ln-Cu to a nested model with a linear ln-Cu term only.

To obtain effect estimates, we subsequently used multivariable linear regression, adjusting for selected confounders with Mn, Pb, and Cr modeled as continuous ln-transformed linear terms and ln-Cu modeled as a categorical variable (tertiles: high, low versus middle). We initially included all pairwise interactions between metals as cross-product terms in this model, but herein we report estimates from the final model only, which included only statistically significant (p<0.10 ) pairwise interactions. Our final multivariable linear regression model was specified as follows:

Y=β0+β1(lnMn)+β2(lnPb)+β3(lowCu)+β4(highCu)+β5(lnCr)+β6(lnMn×lowlnCu)+β7(lnMn×highCu)+β8(lnPb×lowCu)+β9(lnPb×highlnCu)+β10–14  (confounders).

We obtained pooled estimates and CIs by combining information from the 40 mean and variance estimates using Rubin’s method (Rubin 2004).

This approach of multivariable regression modeling, however, limits the investigation of the mixture to pairwise interactions. We therefore considered BKMR models to further examine this potentially complex relationship.

Bayesian kernel machine regression.

To allow for potential nonlinear associations between ln-metals and IQ as well as higher-order interactions between metals, we used Bayesian kernel machine regression (BKMR) to flexibly model the adjusted associations of the metal mixture with IQ. This method has been described previously in detail (Bobb et al. 2015, 2018; Valeri et al. 2017). Briefly, this approach estimates the joint association of mixture components and a health outcome such as IQ, allowing for nonlinear and nonadditive associations and potentially higher-order interactions among correlated mixture components, while adjusting for confounders. We used the component-wise variable selection option of BKMR, which controls for multiple testing and provides a relative measure of variable importance for each individual component of the mixture. The BKMR model for our analyses is given as follows:

Yi=h(Mni, Pbi, Cui, Cri)+βXi+ϵi.

The model consists of a health outcome Yi ; a smooth function h() that is modeled by a kernel function K that links the exposure contribution to the outcome for different subjects by the distance between individuals in the exposure space; a vector Xi that allows for adjustment of confounders; and a random error term ϵi . All metals were ln-transformed to remain consistent with the multivariable linear regression models. The choice of a Gaussian kernel function permits h() to potentially contain nonlinear and interactive effects among mixture components, allowing the data to dictate the shape of this exposure–response surface. For primary analyses, we obtained 10,000 posterior samples of all model parameters using a Markov chain Monte Carlo sampler and default noninformative priors specified in the R package. Credible intervals for estimates, which account for uncertainty both due to estimating the exposure–response function and due to variable selection from among a high-dimensional set of exposures, were calculated using the bkmr R package (Bobb et al. 2018). The bkmr package also calculates posterior inclusion probabilities (PIPs) to quantify the relative importance of each metal to the outcome.

We fit a separate BKMR model for each of the 40 imputed data sets. Using each BKMR fit, we estimated the posterior mean and variance for every contrast of interest. For each contrast, we obtained an overall estimate and credible interval by combining information from the 40 posterior mean and variance estimates using Rubin’s method. We developed publicly available R functions to combine information from multiple BKMR fits and to provide estimates of the overall association and credible intervals (Devick 2019) (see code file in supplemental material).

In order to describe associations from BKMR’s high-dimensional parameter space, we computed summary measures to quantify and visualize the exposure–response surface (Bobb et al. 2018). We estimated the following: a) the exposure–response relationship for each metal (e.g., Mn) on IQ when all other metals (i.e., Pb, Cu, and Cr) are fixed at their median; b) the association of an interquartile range (IQR; i.e., 25th to 75th percentile) increase in a particular metal (e.g., Mn) on IQ, at varying levels of the mixture (i.e., at the 25th, 50th, and 75th concurrent percentiles of Pb, Cu, and Cr); c) the pairwise dose–response relationship for each metal (e.g., Mn) on IQ at varying levels of a second metal (e.g., Cu at 25th, 50th, and 75th percentiles), when remaining metals (i.e., Pb and Cr) are fixed at their median; and d) the joint association of an incremental increase in all metals on IQ, compared with when all metals are their median.

Hierarchical variable selection option in BKMR.

Given the lack of consensus about which biological sample is the most consistent and valid biomarker of exposure for Mn, Cu, and Cr (Bertinato and Zouzoulas 2009; Coetzee et al. 2016; Jursa et al. 2018; Lukaski 1999; de Sousa Viana et al. 2014), we explored the use of hierarchical, or grouped, variable selection in BKMR as a secondary analysis to identify the biological sample for each metal that was most strongly associated with IQ. Details on the hierarchical variable selection approach of BKMR have been published previously (Bobb et al. 2015). Briefly, this function allows for the selection of one component within a prespecified group of correlated components. These secondary analyses proceeded in two steps. In the first step, we applied the hierarchical variable selection function to identify the most predictive biological sample type for each of the three metals (Mn, Cu, and Cr) when considering IQ as the outcome. In the second step, we fit new BKMR models with these newly selected biomarkers (i.e., one biological sample type per metal) along with blood Pb to evaluate associations with IQ. Specifically, we first entered the suite of biomarkers into the model as a group for each of the three metals (e.g., for Mn: blood Mn, hair Mn, saliva Mn, urine Mn, nail Mn). The hierarchical variable selection process quantifies the relative importance of each group (i.e., each metal) as well as each biological sample type within each group to the outcome using a PIP. For example, for Mn, conditional PIPs are estimated for blood Mn, hair Mn, saliva Mn, urine Mn, and nail Mn, given that the group of Mn biomarkers is an important group of exposures in the model. We selected the biological sample type with the highest conditional PIP to use, as a second step, in a standard BKMR model. This final model included only the selected biomarkers for Mn, Cu, and Cr as well as for blood Pb.

Sensitivity analyses.

In sensitivity analyses we examined the robustness of our findings in three ways: a) We used two alternative prior specifications for BKMR models to allow for varying degrees of smoothness of the function (Valeri et al. 2017); b) we examined the potential confounding effect of study site by adding this variable to models; and c) we performed analyses using complete data (i.e., without multiple imputation). All statistical analyses were conducted in R (version 3.5.1; R Development Core Team). BKMR was run using the bkmr package (Bobb et al. 2018).

Results

A total of 635 participants had available data for cognitive scores and hair metals. Half of the participants were female (50.6%), and about half (53.4%) were of medium SES. The average [standard deviation (SD)] for age was 12.3 y (0.9), hemoglobin was 13.8 g/L (0.9), and HOME score was 6.5 (1.7; scale ranged from 0–9) (Table 1). Descriptive characteristics did not vary substantially by study site (i.e., Bagnolo Mella, Valcamonica, or Garda Lake) (see Table S2). Median (25th, 75th percentile) metals concentrations were 0.08 (0.05, 0.2) μg/g for hair Mn; 9.6 (7.1, 15.4) μg/g for hair Cu; 0.05 (0.03, 0.8) μg/g for hair Cr; and 1.3 (1.0, 1.9) μg/dL for blood Pb (Table 2). Mean (SD) scores for FSIQ, VIQ, and PIQ were 107.1 (12.5), 103.4 (13.0), and 109.1 (12.8), respectively. Hair Mn, hair Cu, hair Cr, and blood Pb were weakly correlated [pairwise Spearman’s rho (r)<0.3 ; see Table S3]. Most characteristics in the original data were similar to those averaged across the 40 imputed data sets except for saliva metals: Concentrations averaged across imputed data sets were higher than in original data [median (IQR): saliva Mn, 16.5 (71.4) μg/L versus 4.9 (11.1) in original data; saliva Cu, 31.8 (73.2) versus 21.7 (41.6) μg/L ; and saliva Cr, 0.8 (2.2) versus 0.4 (0.7) μg/L ] (see Table S4).

Table 1 Participant characteristics (n=635 ).

Table 1, in two columns, lists categories and uppercase lowercase italic n (percent) or Mean (standard deviation).

Characteristic	n (percent) or mean (SD)
Demographics
Sex
Female	321 (50.6%)
Male	314 (49.4%)
Age (y)	12.3 (0.9)
Socioeconomic status index
High	154 (24.3%)
Low	141 (22.2%)
Medium	339 (53.4%)
Site
Bagnolo Mella	200 (31.5%)
Garda Lake	214 (33.7%)
Valcamonica	221 (34.8%)
HOME score	6.5 (1.7)
Hemoglobin (g/dL)	13.8 (0.9)
Biomarkers (median, 25th, 75th percentile)
Mn Hair (μg/g )	0.08 (0.05, 0.15)
Pb Blood (μg/dL )	1.3 (1.0, 1.9)
Cu Hair (μg/g )	9.6 (7.03, 15.4)
Cr Hair (μg/g )	0.05 (0.03, 0.8)
Neurocognitive outcomes
Full-scale IQ	107.1 (12.5)
Verbal IQ	103.5 (13.0)
Performance IQ	109.1 (12.8)

Note: Descriptive statistics averaged across 40 multiple imputed data sets. Cr, chromium; Cu, copper; HOME, Home Observation Measurement of the Environment; Mn, manganese; Pb, lead; SD, standard deviation.

Table 2 Metal concentrations in biomarkers.

Table 2 has five columns, namely, Percentile, Hair Manganese (micrograms per gram), Blood Lead (micrograms per deciliter), Hair Copper (micrograms per gram), and Hair Chromium (micrograms per gram).

Percentile	Hair Mn (μg/g )	Blood Pb (μg/dL )	Hair Cu (μg/g )	Hair Cr (μg/g )
Min	0.01	0.39	1.72	0.01
10th	0.03	0.80	5.46	0.02
25th	0.05	1.00	7.05	0.03
50th	0.08	1.32	9.56	0.05
75th	0.15	1.90	15.37	0.08
90th	0.26	2.60	24.83	0.13
Max	1.13	23.84	191.00	1.81

Note: Descriptive statistics were averaged across 40 multiple imputed data sets. Cr, chromium; Cu, copper; max, maximum; min, minimum; Mn, manganese; Pb, lead.

GAMs and Multivariable Linear Regression

We began by fitting GAMs and linear regression models for VIQ. We observed a nonlinear, inverted U-shaped association between ln-hair Cu and VIQ (see Figure S2). Using multivariable linear regression, the pooled estimate from 40 models with an additional quadratic term for ln-Cu was a better fit (likelihood ratio test p=0.03 ) compared with the pooled estimate of nested models with ln-Cu specified only as a linear term, supporting the observed nonlinear Cu-VIQ association. We then fit multivariable linear regression models with ln-Cu categorized into tertiles to obtain more interpretable estimates while maintaining the inverted U-shaped association. Compared with the middle tertile, low and high Cu were associated with 1.2 (−3.7 , 1.2) and 2.2 (−4.7 , 0.3)–point decreases in VIQ after adjusting for sex, age, SES, hemoglobin, HOME score, and HOME score squared as well as Mn, Pb, and Cr (Table 3). There was evidence of interaction between ln-Mn and ln-Cu (interaction term p -value between ln-Mn and low ln-Cu: pinteraction=0.04 ): a 1-SD increase in Mn was associated with decreased VIQ scores among individuals with low or mid-levels of Cu (β1+β6=−2.0 for low Cu; β1=−1.4 for mid-level Cu), whereas Mn was associated with a 1.1-point increase in VIQ among individuals with high Cu levels (β1+β7 ). Similarly, the adverse ln-Pb association was associated with a 2.1-point decrease in VIQ among individuals with low ln-Cu levels (β2+β8=−2.14 ), whereas the Pb association was close to null among those with mid- and high-level Cu. For PIQ, the association with Cu did not appear to be nonlinear and pairwise interactions with Mn or Pb were not evident (pinteraction=0.45–0.93 ) (Table 3). Cu was nonlinearly associated as an inverted U with FSIQ, but there was no evidence of pairwise interaction with Mn or Pb (pinteraction=0.23–0.77 ).

Table 3 Adjusted associations of verbal, performance, and full-scale intelligence quotient (VIQ, PIQ, and FSIQ) and metal biomarkers from multivariable linear regression models.

Table 3, in four columns, lists categories, verbal intelligence quotient, performance intelligence quotient, and full-scale intelligence quotient. Verbal intelligence quotient, performance intelligence quotient, and full-scale intelligence quotient columns are subdivided into one column, namely, lowercase beta (95 percent confidence interval).

	VIQ	PIQ	FSIQ
	β (95% CI)	β (95% CI)	β (95% CI)
Biomarkera
ln-Hair manganese	−1.4 (−3.2 , 0.4)	−0.2 (−2.1 , 1.6)	−0.9 (−2.7 , 0.9)
ln-Blood lead	−0.04 (−1.8 , 1.7)	−0.2 (−2.0 , 1.6)	−0.1 (−1.8 , 1.6)
ln-Hair chromium	−0.04 (−1.2 , 1.2)	0.5 (−0.7 , 1.7)	0.3 (−0.9 , 1.5)
ln-Hair copper by tertileb
Low	−1.2 (−3.7 , 1.2)	0.2 (−2.3 , 2.8)	−0.7 (−3.1 , 1.7)
High	−2.2 (−4.7 , 0.3)	−1.0 (−3.5 , 1.6)	−1.8 (−4.3 , 0.7)
ln-Hair manganese by copper tertile
Low	−0.6 (−3.1 , 1.9)	−0.5 (−3.1 , 2.0)	−0.4 (−2.9 , 2.0)
High	2.5 (0.1, 4.9)	1.0 (−1.5 , 3.4)	2.6 (0.2, 4.9)
ln-Blood lead by copper tertile
Low	−2.1 (−4.6 , 0.3)	−0.7 (−2.3 , 1.8)	−1.5 (−3.9 , 1.0)
High	−0.2 (−2.5 , 2.2)	0.1 (−2.3 , 2.5)	−0.3 (−2.7 , 2.0)

Note: Models were adjusted for sex, age, SES, hemoglobin, HOME score, and HOME score squared. Models are the following form: Y=β0+β1(ln Mn)+β2(ln Pb)+β3(low ln Cu)+β4(high ln Cu)+β5(ln Cr)+β6(ln Mn×low ln Cu)+β7(ln Mn×high ln Cu)+β8(ln Pb×low ln Cu)+β9(ln Pb×high ln Cu) . CI, confidence interval; HOME, Home Observation Measurement of the Environment; SES, socioeconomic status.

^aβ coefficients and 95% CIs averaged across 40 multiple imputed data sets. All metals concentrations were ln-transformed and standardized.

^bCu tertiles were created from ln-transformed variable; untransformed values of tertiles are low tertile=1.72–7.83μg/g ; middle tertile=7.83–13.2μg/g ; and high tertile=13.2–191.0μg/g .

Bayesian Kernel Machine Regression Analyses

We implemented BKMR to obtain an estimate of the joint exposure–response function of hair Mn, hair Cu, hair Cr, and blood Pb on IQ in adolescents. We began by investigating the dose–response relationships of each individual (ln-transformed) metal in the mixture with VIQ. Figure 1 displays the exposure–response functions for each metal, when all other metals are set at their medians, after adjusting for sex, age, SES, hemoglobin, HOME score, and HOME score squared. Similar to findings from GAMs and multivariable linear regression, findings from the BKMR analyses suggested a nonlinear, inverted U-shaped association between ln-Cu and VIQ, where both low and high concentrations of Cu were associated with lower VIQ scores when other metals were set to their medians. A decrease in Cu from the 40th to 10th percentile (8.5 to 5.4μg/g ; Table 2) was associated with a 0.8 (−1.9 , 0.3)–point reduction in VIQ score, whereas an increase in Cu from the 60th to 90th percentile (11.8 to 24.9μg/g ) was associated with a 1.3 (−2.7 , 0.2)–point reduction in VIQ. The association between Cr and VIQ was null when other metals were set to their medians.

[Image omitted - see PDF]

To more fully investigate possible effect modification by Cu in light of its inverted U-shaped association, we estimated the association of an IQR increase in an individual metal on VIQ score, at varying levels (25th, 50th or 75th percentiles) of all three other metals and stratified by level of Cu (at the 10th, 50th, and 90th percentiles) (Figure 2). Interaction was suggestive if there was variation in the effect estimate of a given metal at different percentiles of the other metals. There was variation between estimates for each metal across the three different plots of Cu percentiles, although CIs were relatively wide. For example, when Cu was set at its 10th, 50th, or 90th percentile (5.5, 9.6, or 24.9μg/g , respectively), an IQR increase in Mn (0.10μg/g ) was associated with respective reductions in VIQ of approximately 1.3 (−3.0 , 0.3), 1.0 (−2.5 , 0.4) and 0.5 (−2.2 , 1.1) points, regardless of Pb and Cr variation. This suggests that the adverse associations of Mn are more apparent at lower Cu levels. This finding is also reflected in Figure 3, showing pairwise interactions, where the negative slope of ln-Mn with VIQ is slightly steeper at lower (e.g., 25th percentile) Cu levels than at higher Cu levels. Associations of ln-Cr and VIQ were null (omitted from figure).

[Image omitted - see PDF]

Finally, we investigated the overall joint association of the metal mixture on VIQ score, stratified by Cu level (Figure 4). Each posterior mean estimate and the corresponding CI represented the change in VIQ when each metal in the mixture was concurrently set to a particular percentile (10th to 85th percentile) as compared with when each metal was set to its median value (50th percentile). Overall, increasing levels of Mn, Pb, and Cr were associated with decreases in VIQ score, especially at low levels of Cu. For example, at low Cu levels (10th percentile), metals set at their respective 70th, 75th, and 80th percentiles were associated with 1.1 (−1.9 , −0.2 ), 1.4 (−2.6 , −0.2 ), and 1.8 (−3.2 , −0.3 )–point decreases in VIQ, respectively, compared with when all metals were set at their 50th percentiles.

[Image omitted - see PDF]

In models of FSIQ and PIQ (see Figures S3 and S4), Mn and Pb associations were also negative and the Cu association also appeared to be nonlinear, but associations were smaller in magnitude and less precise (see Figures S3A and S4A). Associations of the overall mixture with FSIQ and PIQ were negative but weaker than associations with VIQ (see Figures S3D and S4D).

Hierarchical Variable Selection Using BKMR

We implemented hierarchical variable selection within BKMR to allow the model to select, for each metal, the biological sample that was most predictive of the VIQ score. We used 10 randomly selected data sets from the 40 imputed data sets. For (ln-transformed) Mn, Cu, and Cr (blood was forced into the model as the Pb biomarker), the highest average conditional PIPs (cPIPs) across the 10 data sets were estimated for saliva for Mn (cPIP=0.99 ), hair for Cu (cPIP=0.60 ), and saliva for Cr (cPIP=0.74 ). In BKMR models using these selected biomarkers plus blood Pb, saliva Mn was negatively associated with VIQ (Figure 5A,B) at all levels of the other metals: An IQR increase in saliva Mn (71.4μg/L ) was associated with reductions of 5.1 (95% CI: −8.6 , −1.7 ), 4.8 (−7.9 , −1.7 ), and 4.2 (−7.7 , −0.8 ) points in VIQ score, when other metals were at their 25th, 50th, and 75th percentiles, respectively (Figure 5B; Cu results were omitted from Figure 5B due to their nonlinear shape). There was also a suggested interaction between Mn and at least one other metal: Mn effect estimates varied slightly with increasing levels of the mixture, with the strongest association estimated when all other metals were set at their 25th percentile (Figure 5B). Jointly increasing levels of all metals were associated with decreases in VIQ score, which was stronger than the joint association observed previously in models using hair as the biomarker of exposure for Mn, Cu, and Cr (cf. Figure 5D versus Figure 4).

[Image omitted - see PDF]

Sensitivity Analyses

Results from complete case analyses for both primary analyses (using hair Mn, hair Cu, hair Cr, and blood Pb; n=338 ) and secondary analyses (using saliva Mn, hair Cu, saliva Cr, and blood Pb; n=330 ) were similar to models using multiple imputation, although estimates were attenuated (see Figures S5 and S6). Adjusting for study site as an additional confounder in models did not appreciably change results, nor did changing the prior specifications to allow for varying degrees of smoothness for the parameter of the h function in the BKMR model (see Figures S7–S9). Overall, our conclusions from primary analyses were unchanged after performing sensitivity analyses.

Discussion

These results provide new evidence for an association between a metals mixture (Mn, Pb, Cu, and Cr) and IQ assessed during adolescence. This mixture of metals is representative of exposures experienced by populations adjacent to ferroalloy production plants, a growing and globally prevalent industry supporting steel manufacturing (Holappa 2010; Olsen et al. 2007). The results lend insight into the impact of multi-metal exposure on the brain during adolescence, a developmental period that is understudied yet of potentially heightened susceptibility to neurotoxic insult.

We found that joint increases in metal concentrations were associated with lower IQ scores but particularly at lower Cu levels (e.g., fixed at its 10th percentile) and when other metals were above their respective 50th percentiles. Associations with VIQ were stronger than with PIQ. Associations of (ln-transformed) Mn, measured either in hair or saliva, with VIQ were linear and negative, suggesting adverse effects on aspects of cognition that involve language comprehension, verbal expression, attention, working memory, and executive function. Although other studies of children 7–12 years of age with industrial airborne Mn exposure did not estimate the association of Mn on VIQ in the context of a metals mixture, they have similarly reported adverse, linear associations between hair Mn and VIQ (Menezes-Filho et al. 2011; Riojas-Rodríguez et al. 2010; Wright et al. 2006) as well as with domains that are influenced by verbal abilities, including verbal learning, memory (Carvalho et al. 2018; Torres-Agustín et al. 2013; van Wendel de Joode et al. 2016; Wright et al. 2006), and verbal working memory (Carvalho et al. 2014; Haynes et al. 2018; van Wendel de Joode et al. 2016). These findings are also consistent with some (Bouchard et al. 2011; Wasserman et al. 2011), but not all (Bouchard et al. 2018), pediatric studies of environmental Mn exposure via drinking water and VIQ. Inconsistencies among the drinking-water studies might be due in part to reduced variability of Mn exposure between subjects and differences in outcome measurement given that some of the studies used a condensed neurobehavioral assessment battery to calculate VIQ (Bouchard et al. 2018).

An inverted U-shaped association was observed between ln-hair Cu and VIQ, suggesting that both low and high levels of Cu may have adverse effects on neurobehavior. This nonlinear association is consistent with the properties of Cu as both an essential micronutrient and, in excess, a neurotoxicant (Otten et al. 2006). There are few environmental studies of the association of Cu and neurobehavior, although Cu contamination in the environment is common: Cu is mined extensively in the United States to produce a variety of products (i.e., electrical wiring, sheet metal, preservatives), is widely used as a pesticide, and is a common drinking-water contaminant (ATSDR 2004; U.S. EPA 2008). In addition, environmentally elevated Cu levels have been identified at more than half of 1,647 current or former U.S. Superfund sites (ATSDR 2004).

Animal behavioral research has demonstrated that increasing Cu dose by oral gavage or drinking water reduces spatial memory, attention, learning, motor coordination, and strength (Behzadfar et al. 2017; Kalita et al. 2018; Kumar et al. 2015). In humans, cross-sectional studies have reported higher Cu accumulation in hair and nails of children with autism spectrum disorder (ASD) compared with typically developing children (Lakshmi Priya and Geetha 2011; Obrenovich et al. 2011). Yet few epidemiologic studies have investigated preclinical decrements of neurobehavior in relation to Cu concentrations. Among 606 Belgian adolescents living near industrial areas, whole blood Cu was inversely related to sustained attention and working memory (Kicinski et al. 2015). Likewise, a cross-sectional study of 826 Chinese children 10–14 years of age reported associations of high total serum Cu (≤110μg/dL ) and reduced working memory among boys only (Zhou et al. 2015). In an ecological study of fourth-graders in New Orleans, Louisiana, soil Cu was associated with lower academic performance (Zahran et al. 2012).

On the other hand, Cu is biologically essential, and there is evidence that Cu supports brain development and possible neuroprotection (Bica et al. 2014; Hung et al. 2012; Kaler 2011). In vitro studies have demonstrated neuroprotective effects of Cu, in particular that Cu enhances neurite elongation (Bica et al. 2014) and improves Parkinson’s disease outcomes (Hung et al. 2012). At nutritionally insufficient low levels, Cu has been linked with adverse health outcomes in humans, including Menkes disease, a rare X-linked recessive disorder where the enzyme ATP7A is absent, leading to severe muscle dystonia, gray matter neurodegeneration, and infantile death (Kaler 2011). Collectively, the aforementioned research supports both a neuroprotective and neurotoxic role for Cu, consistent with our finding of an inverted U-shaped association between hair ln-Cu and VIQ in GAMs, multivariable linear regressions, and BKMR models.

When examined on its own, Cu appears to be optimal for VIQ at midrange concentrations in our data. However, when examined in the context of a mixture, Cu may mitigate some of the neurotoxicity of Mn and Pb. In our study, we found suggestive interactions between Mn and Cu and between Pb and Cu, where the adverse associations of Mn with VIQ were strongest at lower Cu levels (i.e., 10th percentile, or 5.5μg Cu/g hair). This suggests an opposing effect of Cu: Mn and Pb toxicity is most apparent when Cu, which has beneficial effects, is at its lowest levels. At higher levels of Cu (e.g., 50th or 90th percentiles in BKMR models or third tertile in multivariable linear regression models), the adverse associations of Mn and Pb are less pronounced. These findings underline the importance of considering metals in the context of a mixture.

Interactions between Mn and Cu as well as Pb and Cu are plausible given the potential for overlapping toxicological mechanisms. For example, Mn and Cu are important for brain growth and development but in excess they can both cause cellular oxidative stress and degeneration in the brain as well as induce parkinsonian-like symptoms (Ala et al. 2007; Burton and Guilarte 2009; D’Ambrosi and Rossi 2015). In addition, Pb, Mn, and Cu can disrupt calcium-dependent pathways and cellular redox processes (Burton and Guilarte 2009; Lidsky and Schneider 2003; Scheiber et al. 2014; Tjalkens et al. 2006). However, the epidemiologic literature on interactions between Cu and either Mn or Pb is sparse. One recent epidemiologic study of children in Mexico City used BKMR and reported a similar interaction between Cu and Pb and neurobehavior, where higher concentrations of Pb reduced the beneficial effect of Cu on infant neurodevelopment (Liu et al. 2018). Despite several differences in study design, including exposure timing (prenatal versus adolescent), age at outcome assessment (6–24 months versus adolescence), and sources of contaminants (unknown versus ferroalloy industry), this finding of a Cu–Pb interaction using BKMR is consistent with ours.

Some evidence exists for interaction between Cu and Mn in experimental studies. In rodents and nonhuman primates, Mn exposure has altered endogenous Cu homeostasis, whereby Mn exposure either reduced (Fu et al. 2015; Robison et al. 2013) or enhanced Cu brain tissue concentrations (Guilarte et al. 2006, 2008; Zheng et al. 2009). Furthermore, Mn and Cu compounds both inhibited NMDA-mediated receptors individually (Guilarte and Chen 2007). From these studies, there appears to be potential for these metals to influence one another’s concentrations in the body as well as their potential effects on the brain.

This study used BKMR with hierarchical variable selection in order to guide the selection of exposure biomarkers. In analyses using this approach, saliva Mn was most strongly associated with VIQ score. Saliva Mn was imputed for a large portion of our data (44%); yet a sensitivity analysis using participants with complete data for saliva (n=330 ), identified saliva Mn as the biomarker most strongly associated with VIQ (see Figure S6). In addition, we sought to examine the influence of extreme values of saliva Mn in our full sample; however, no outliers were identified by the generalized extreme Studentized deviate (ESD) many-outlier procedure using imputed data sets (Rosner 1983). Although research using saliva as a biomarker for Mn exposure is sparse, recent studies describing analytical methodologies for measuring saliva Mn in humans have reported associations between environmental Mn exposure and saliva Mn levels (Butler et al. 2019; Lucas et al. 2015) and have quantified saliva Mn in occupationally exposed workers (Gil et al. 2011; Wang et al. 2008). Among Mn-exposed welders, studies investigating saliva Mn have reported significant correlations with serum Mn (r=0.57 ), airborne Mn (r=0.65 ), and years of work experience (r=0.40 ) (Fan et al. 2017; Wang et al. 2008; Zhou et al. 2010). In environmental studies of children, saliva Mn was significantly associated with air and soil Mn in areas near ferroalloy activity in the same cohort as the present study (Butler et al. 2019), but not with Mn in drinking water in a Canadian cohort (Bouchard et al. 2018; Ntihabose et al. 2018). This latter discrepancy may be attributed to analytical differences in saliva sample processing between studies. These findings warrant further study of saliva as a biomarker of Mn exposure.

There are limitations to this study. As with any cross-sectional analysis, we are unable to establish temporality between exposure and outcome, thus precluding our ability to evaluate causality. The temporality of metals and neurological disorders is a common concern given that some neurological disorders, such as ASD, could influence both IQ and metals concentrations. For example, metal concentrations in hair among patients with ASD could be influenced by altered metal metabolism, diet, behavior, or medication use from ASD rather than being a causal contributor to the etiology of ASD (Kalkbrenner et al. 2014). However, participants in this study were generally healthy adolescents with no diagnosed neurological disorder. Selection bias could be possible if participation in the study was based on a factor related both to exposures and outcome. For example, it is possible that subjects who perceived they were at greater risk of exposure to environmental metals or at increased risk of health effects from environmental exposures were more likely to participate in the study, although we think this unlikely. In addition, a diagnosed neurobehavioral disorder was an exclusion criterion in the study. We cannot rule out unmeasured confounding by zinc or other co-exposures, which may be associated with levels of Mn, Pb, or Cu, as well as with neurodevelopment (Adamo and Oteiza 2010; Mahaffey 1990). In addition, both prenatal and postnatal metals exposure could be associated with neurodevelopment and correlated with adolescent exposure and thereby have acted as potential confounders in our study. In a subsample of our cohort (n=195 ), Mn levels were measured in deciduous teeth as a proxy for prenatal and early postnatal metals exposure. We found that tooth Mn levels reflecting the prenatal, postnatal, and childhood exposure windows were not correlated with adolescent hair Mn levels (r=0.01 , p=0.86 ; r=0.08 , p=0.29 ; and r=0.10 , p=0.17 , respectively, for the three exposure windows). Although we lacked concurrent samples of teeth and hair, the weak correlations estimated between tooth and hair Mn reduced our concern that Mn associations can be explained entirely by prenatal and/or postnatal metals levels.

Although the use of hair as a biomarker for metal exposure has been scrutinized owing to the potential for exogenous contamination (Skröder et al. 2017), we used a conservative, validated method to extensively wash hair of exogenous metals (Eastman et al. 2013; Jursa et al. 2018). This may partly explain why hair Mn concentrations measured in our study (mean=128 ng/g ; range=11–1,130 ng/g were lower than the range of concentrations reported in other Mn studies of communities near ferromanganese industry, including among Ohio children (mean=360 ng/g , range 17–15,970) (Haynes et al. 2018) and Brazilian children (mean=5,830 ng/g , range=100–86,680 ) (Menezes-Filho et al. 2011), and U.S. children living near a Superfund site in Oklahoma (mean 471 ng/g , range 89–2,145) (Wright et al. 2006). We expect hair metals levels in our data to represent primarily endogenous levels. Hair Cu and Cr levels in our study (Cu: median=9.6μg/g , range: 1.7–191; Cr: median=0.05μg/g , range=0.006–1.81μg/g ) were in the range of global reference values in hair for children 3–15 years of age (range: 7.2–82.7μg/g ; 0.001–4.56μg/g ) (Mikulewicz et al. 2013).

There are numerous strengths to our study. Our use of BKMR allowed us to account for nonlinearity and the potential for high-dimensional interactions between metals in association with adolescent cognition. Further, our capabilities to explore the interactive, nonlinear, and joint associations of the metal mixture on IQ extended beyond information gleaned from multiple linear regression. The use of this method is particularly relevant for metals given that a) many metals are essential but neurotoxic in excess and in turn may have nonlinear dose–response curves; b) metals commonly co-occur in the environment (i.e., have the potential to be correlated); and c) the presence of some metals can influence another’s metabolism, distribution, and, possibly, mechanisms of toxicity. Although BKMR requires a larger sample size to sufficiently explore high-dimensional interactions, we employed this method using a large data set with a wealth of biomarkers. This was particularly imperative given the lack of consensus on Mn, Cu, and Cr biomarkers of environmental exposure. Use of the hierarchical variable selection procedure in secondary analyses allowed us to identify which biomarkers for Mn, Cu, and Cr were most associated with VIQ and informs biomarker prioritization for future research. This study evaluated the health effects of multiple biomarkers simultaneously in the context of a mixture and to utilize a data-driven approach to inform the selection of biomarkers. Our findings suggest that saliva Mn may be highly informative and particularly important to collect in settings where inhalation exposures are relevant.

Conclusions

Our study investigated the association between a metal mixture of Mn, Pb, Cu, and Cr and cognition in adolescence, a potentially sensitive window for neurodevelopment. Our findings suggest that joint exposure to these metals during adolescence has negative associations with concurrent cognition and that saliva Mn is strongly related to decreased VIQ. We also observed an inverted U-shaped association between ln-Cu and IQ scores, where both low and high concentrations of Cu measured in hair were associated with lower IQ scores. Future research evaluating the joint, interactive, and individual associations of Mn, Pb, and Cu on neurodevelopmental toxicity would further inform this field. Our findings are relevant for improving the health of communities, particularly those where inhalation of metals is an important exposure pathway due to historic or current industrial activity.

Acknowledgments

The research described in this paper was funded in part by National Institutes of Health/National Institute of Environmental Sciences grants F31ES029010, T32ES014562, R00 ES022986, R01 ES019222, R01ES028800, P30ES000002, P30ES023515, and R01ES013744.

References

Adamo AM, Oteiza PI . 2010. Zinc deficiency and neurodevelopment: the case of neurons. Biofactors 36(2):117–124, PMID:20333753,10.1002/biof.91 . Crossref, Medline,

Ala A, Walker AP, Ashkan K, Dooley JS, Schilsky ML . 2007. Wilson’s disease. Lancet 369(9559):397–408, PMID:17276780,10.1016/S0140-6736(07)60196-2 . Crossref, Medline,

ATSDR (Agency for Toxic Substances and Disease). 2004. Public health statement for copper. https://www.atsdr.cdc.gov/phs/phs.asp?id=204&tid=37 [accessed 16 July 2018 ].

ATSDR. 2012. Toxicological Profile for manganese. https://www.atsdr.cdc.gov/toxprofiles/tp.asp?id=102&tid=23 [accessed 22 August 2020 ].

Bauer JA, Claus Henn B, Austin C, Zoni S, Fedrighi C, Cagna G, et al. 2017. Manganese in teeth and neurobehavior: sex-specific windows of susceptibility. Environ Int 108:299–308, PMID:28941415,10.1016/j.envint.2017.08.013 . Crossref, Medline,

Behzadfar L, Abdollahi M, Sabzevari O, Hosseini R, Salimi A, Naserzadeh P, et al. 2017. Potentiating role of copper on spatial memory deficit induced by beta amyloid and evaluation of mitochondrial function markers in the hippocampus of rats. Metallomics 9(7):969–980, PMID:28644490,10.1039/C7MT00075H . Crossref, Medline,

Bertinato J, Zouzoulas A . 2009. Considerations in the development of biomarkers of copper status. J AOAC Int 92(5):1541–1550, PMID:19916391,10.1093/jaoac/92.5.1541 . Crossref, Medline,

Bica L, Liddell JR, Donnelly PS, Duncan C, Caragounis A, Volitakis I, et al. 2014. Neuroprotective copper bis(thiosemicarbazonato) complexes promote neurite elongation. PLoS One 9(2):e90070, PMID:24587210,10.1371/journal.pone.0090070 . Crossref, Medline,

Blakemore S-J, Choudhury S . 2006. Development of the adolescent brain: implications for executive function and social cognition. J Child Psychol Psychiatry 47(3–4):296–312, PMID:16492261,10.1111/j.1469-7610.2006.01611.x . Crossref, Medline,

Bobb JF, Claus Henn B, Valeri L, Coull BA . 2018. Statistical software for analyzing the health effects of multiple concurrent exposures via Bayesian kernel machine regression. Environ Health 17(1):67, PMID:30126431,10.1186/s12940-018-0413-y . Crossref, Medline,

Bobb JF, Valeri L, Claus Henn B, Christiani DC, Wright RO, Mazumdar M, et al. 2015. Bayesian kernel machine regression for estimating the health effects of multi-pollutant mixtures. Biostatistics 16(3):493–508, PMID:25532525,10.1093/biostatistics/kxu058 . Crossref, Medline,

Bouchard MF, Sauvé S, Barbeau B, Legrand M, Brodeur M-È, Bouffard T, et al. 2011. Intellectual impairment in school-age children exposed to manganese from drinking water. Environ Health Perspect 119(1):138–143, PMID:20855239,10.1289/ehp.1002321 . Link,

Bouchard MF, Surette C, Cormier P, Foucher D . 2018. Low level exposure to manganese from drinking water and cognition in school-age children. Neurotoxicology 64:110–117, PMID:28716743,10.1016/j.neuro.2017.07.024 . Crossref, Medline,

Bulcke F, Dringen R, Scheiber IF . 2017. Neurotoxicity of copper. Adv Neurobiol 18:313–343, PMID:28889275,10.1007/978-3-319-60189-2_16 . Crossref, Medline,

Burton NC, Guilarte TR . 2009. Manganese neurotoxicity: lessons learned from longitudinal studies in nonhuman primates. Environ Health Perspect 117(3):325–332, PMID:19337503,10.1289/ehp.0800035 . Link,

Butler L, Gennings C, Peli M, Borgese L, Placidi D, Zimmerman N, et al. 2019. Assessing the contributions of metals in environmental media to exposure biomarkers in a region of ferroalloy industry. J Expo Sci Environ Epidemiol 29(5):674–687, PMID:30337680,10.1038/s41370-018-0081-6 . Crossref, Medline,

Caito S, Aschner M . 2017. Developmental neurotoxicity of lead. Adv Neurobiol 18:3–12, PMID:28889260,10.1007/978-3-319-60189-2_1 . Crossref, Medline,

Carlin DJ, Rider CV, Woychik R, Birnbaum LS . 2013. Unraveling the health effects of environmental mixtures: an NIEHS priority. Environ Health Perspect 121(1):A6–A8, PMID:23409283,10.1289/ehp.1206182 . Link,

Carvalho CF, Menezes-Filho JA, de Matos VP, Bessa JR, Coelho-Santos J, Viana GFS, et al. 2014. Elevated airborne manganese and low executive function in school-aged children in Brazil. Neurotoxicology 45:301–308, PMID:24308913,10.1016/j.neuro.2013.11.006 . Crossref, Medline,

Carvalho CF, Oulhote Y, Martorelli M, Carvalho CO, Menezes-Filho JA, Argollo N, et al. 2018. Environmental manganese exposure and associations with memory, executive functions, and hyperactivity in Brazilian children. Neurotoxicology 69:253–259, PMID:29432852,10.1016/j.neuro.2018.02.002 . Crossref, Medline,

Cesana GC, Ferrario M, De Vito G, Sega R, Grieco A . 1995. Evaluation of the socioeconomic status in epidemiological surveys: hypotheses of research in the Brianza area MONICA project. Med Lav 86(1):16–26, PMID:7791660 . Medline,

Chiu Y-HM, Claus Henn B, Hsu H-HL, Pendo MP, Coull BA, Austin C, et al. 2017. Sex differences in sensitivity to prenatal and early childhood manganese exposure on neuromotor function in adolescents. Environ Res 159:458–465, PMID:28858760,10.1016/j.envres.2017.08.035 . Crossref, Medline,

Claus Henn B, Coull BA, Wright RO . 2014. Chemical mixtures and children’s health. Curr Opin Pediatr 26:223–229, PMID:24535499,10.1097/MOP.0000000000000067 . Crossref, Medline,

Claus Henn BC, Schnaas L, Ettinger AS, Schwartz J, Lamadrid-Figueroa H, Hernández-Avila M, et al. 2012. Associations of early childhood manganese and lead coexposure with neurodevelopment. Environ Health Perspect 120(1):126–131, PMID:21885384,10.1289/ehp.1003300 . Link,

Coetzee DJ, McGovern PM, Rao R, Harnack LJ, Georgieff MK, Stepanov I . 2016. Measuring the impact of manganese exposure on children’s neurodevelopment: advances and research gaps in biomarker-based approaches. Environ Health 15(1):91, PMID:27576472,10.1186/s12940-016-0174-4 . Crossref, Medline,

D’Ambrosi N, Rossi L . 2015. Copper at synapse: release, binding and modulation of neurotransmission. Neurochem Int 90:36–45, PMID:26187063,10.1016/j.neuint.2015.07.006 . Crossref, Medline,

Devick KL . 2019. BKMR plot functions for multiply imputed data. Last updated 10 June 2019. https://github.com/kdevick/bkmr_MI [accessed 22 August 2020 ].

Eastman RR, Jursa TP, Benedetti C, Lucchini RG, Smith DR . 2013. Hair as a biomarker of environmental manganese exposure. Environ Sci Technol 47(3):1629–1637, PMID:23259818,10.1021/es3035297 . Crossref, Medline,

Fan X, Luo Y, Fan Q, Zheng W . 2017. Reduced expression of PARK2 in manganese-exposed smelting workers. Neurotoxicology 62:258–264, PMID:28826884,10.1016/j.neuro.2017.08.006 . Crossref, Medline,

Fu S, O’Neal S, Hong L, Jiang W, Zheng W . 2015. Elevated adult neurogenesis in brain subventricular zone following in vivo manganese exposure: roles of copper and DMT1. Toxicol Sci 143(2):482–498, PMID:25575534,10.1093/toxsci/kfu249 . Crossref, Medline,

Giedd JN, Blumenthal J, Jeffries NO, Castellanos FX, Liu H, Zijdenbos A, et al. 1999. Brain development during childhood and adolescence: a longitudinal MRI study. Nat Neurosci 2(10):861–863, PMID:10491603,10.1038/13158 . Crossref, Medline,

Gil F, Hernández AF, Márquez C, Femia P, Olmedo P, López-Guarnido O, et al. 2011. Biomonitorization of cadmium, chromium, manganese, nickel and lead in whole blood, urine, axillary hair and saliva in an occupationally exposed population. Sci Total Environ 409(6):1172–1180, PMID:21211822,10.1016/j.scitotenv.2010.11.033 . Crossref, Medline,

Gorell JM, Johnson CC, Rybicki BA, Peterson EL, Kortsha GX, Brown GG, et al. 1997. Occupational exposures to metals as risk factors for Parkinson’s disease. Neurology 48(3):650–658, PMID:9065542,10.1212/WNL.48.3.650 . Crossref, Medline,

Guilarte TR, Burton NC, Verina T, Prabhu VV, Becker KG, Syversen T, et al. 2008. Increased APLP1 expression and neurodegeneration in the frontal cortex of manganese-exposed non-human primates. J Neurochem 105(5):1948–1959, PMID:18284614,10.1111/j.1471-4159.2008.05295.x . Crossref, Medline,

Guilarte TR, Chen M-K . 2007. Manganese inhibits NMDA receptor channel function: implications to psychiatric and cognitive effects. Neurotoxicology 28(6):1147–1152, PMID:17662456,10.1016/j.neuro.2007.06.005 . Crossref, Medline,

Guilarte TR, Chen M-T, McGlothan JL, Verina T, Wong DF, Zhou Y, et al. 2006. Nigrostriatal dopamine system dysfunction and subtle motor deficits in manganese-exposed non-human primates. Exp Neurol 202(2):381–390, PMID:16925997,10.1016/j.expneurol.2006.06.015 . Crossref, Medline,

Haynes EN, Sucharew H, Hilbert TJ, Kuhnell P, Spencer A, Newman NC, et al. 2018. Impact of air manganese on child neurodevelopment in East Liverpool, Ohio. Neurotoxicology 64:94–102, PMID:28888663,10.1016/j.neuro.2017.09.001 . Crossref, Medline,

Haynes EN, Sucharew H, Kuhnell P, Alden J, Barnas M, Wright RO, et al. 2015. Manganese exposure and neurocognitive outcomes in rural school-age children: the Communities Actively Researching Exposure Study (Ohio, USA). Environ Health Perspect 123(10):1066–1071, PMID:25902278,10.1289/ehp.1408993 . Link,

Holappa L . 2010. Towards sustainability in ferroalloy production. J S Afr Inst Min Metall 110:(12):703–710.

Hung LW, Villemagne VL, Cheng L, Sherratt NA, Ayton S, White AR, et al. 2012. The hypoxia imaging agent Cu¹¹(atsm) is neuroprotective and improves motor and cognitive functions in multiple animal models of Parkinson’s disease. J Exp Med 209(4):837–854, PMID:22473957,10.1084/jem.20112285 . Crossref, Medline,

Jursa T, Stein CR, Smith DR . 2018. Determinants of hair manganese, lead, cadmium and arsenic levels in environmentally exposed children. Toxics 6(2):19, PMID:29565296,10.3390/toxics6020019 . Crossref, Medline,

Kaler SG . 2011. ATP7A-related copper transport diseases—emerging concepts and future trends. Nat Rev Neurol 7(1):15–29, PMID:21221114,10.1038/nrneurol.2010.180 . Crossref, Medline,

Kalita J, Kumar V, Misra UK, Bora HK . 2018. Memory and learning dysfunction following copper toxicity: biochemical and immunohistochemical basis. Mol Neurobiol 55(5):3800–3811, PMID:28536976,10.1007/s12035-017-0619-y . Crossref, Medline,

Kalkbrenner AE, Schmidt RJ, Penlesky AC . 2014. Environmental chemical exposures and autism spectrum disorders: a review of the epidemiological evidence. Curr Probl Pediatr Adolesc Health Care 44(10):277–318, PMID:25199954,10.1016/j.cppeds.2014.06.001 . Crossref, Medline,

Kicinski M, Vrijens J, Vermier G, Hond ED, Schoeters G, Nelen V, et al. 2015. Neurobehavioral function and low-level metal exposure in adolescents. Int J Hyg Environ Health 218(1):139–146, PMID:25287296,10.1016/j.ijheh.2014.09.002 . Crossref, Medline,

Kumar V, Kalita J, Misra UK, Bora HK . 2015. A study of dose response and organ susceptibility of copper toxicity in a rat model. J Trace Elem Med Biol 29:269–274, PMID:25022334,10.1016/j.jtemb.2014.06.004 . Crossref, Medline,

Lakshmi Priya MD, Geetha A . 2011. Level of trace elements (copper, zinc, magnesium and selenium) and toxic elements (lead and mercury) in the hair and nail of children with autism. Biol Trace Elem Res 142(2):148–158, PMID:20625937,10.1007/s12011-010-8766-2 . Crossref, Medline,

Lidsky TI, Schneider JS . 2003. Lead neurotoxicity in children: basic mechanisms and clinical correlates. Brain 126(pt 1):5–19, PMID:12477693,10.1093/brain/awg014 . Crossref, Medline,

Lin C-C, Chen Y-C, Su F-C, Lin C-M, Liao H-F, Hwang Y-H, et al. 2013. In utero exposure to environmental lead and manganese and neurodevelopment at 2 years of age. Environ Res 123:52–57,10.1016/j.envres.2013.03.003 . Crossref, Medline,

Liu SH, Bobb JF, Claus Henn B, Gennings C, Schnaas L, Tellez-Rojo M, et al. 2018. Bayesian varying coefficient kernel machine regression to assess neurodevelopmental trajectories associated with exposure to complex mixtures. Stat Med 37(30):4680–4694, PMID:30277584,10.1002/sim.7947 . Crossref, Medline,

Lucas EL, Bertrand P, Guazzetti S, Donna F, Peli M, Jursa TP, et al. 2015. Impact of ferromanganese alloy plants on household dust manganese levels: implications for childhood exposure. Environ Res 138:279–290, PMID:25747819,10.1016/j.envres.2015.01.019 . Crossref, Medline,

Lucchini RG, Albini E, Benedetti L, Borghesi S, Coccaglio R, Malara EC, et al. 2007. High prevalence of parkinsonian disorders associated to manganese exposure in the vicinities of ferroalloy industries. Am J Ind Med 50(11):788–800, PMID:17918215,10.1002/ajim.20494 . Crossref, Medline,

Lucchini RG, Guazzetti S, Zoni S, Benedetti C, Fedrighi C, Peli M, et al. 2014. Neurofunctional dopaminergic impairment in elderly after lifetime exposure to manganese. Neurotoxicology 45:309–317, PMID:24881811,10.1016/j.neuro.2014.05.006 . Crossref, Medline,

Lucchini RG, Guazzetti S, Zoni S, Donna F, Peter S, Zacco A, et al. 2012a. Tremor, olfactory and motor changes in Italian adolescents exposed to historical ferro-manganese emission. Neurotoxicology 33(4):687–696, PMID:22322213,10.1016/j.neuro.2012.01.005 . Crossref, Medline,

Lucchini RG, Zoni S, Guazzetti S, Bontempi E, Micheletti S, Broberg K, et al. 2012b. Inverse association of intellectual function with very low blood lead but not with manganese exposure in Italian adolescents. Environ Res 118:65–71, PMID:22925625,10.1016/j.envres.2012.08.003 . Crossref, Medline,

Lukaski HC . 1999. Chromium as a supplement. Annu Rev Nutr 19(1):279–302, PMID:10448525,10.1146/annurev.nutr.19.1.279 . Crossref, Medline,

Mahaffey KR . 1990. Environmental lead toxicity: nutrition as a component of intervention. Environ Health Perspect 89:75–78, PMID:2088758,10.1289/ehp.908975 . Link,

Menezes-Filho JA, Carvalho CF, Rodrigues JLG, Araújo CFS, Dos Santos NR, Lima CS, et al. 2018. Environmental co-exposure to lead and manganese and intellectual deficit in school-aged children. Int J Environ Res Public Health 15(11):2418, PMID:30384464,10.3390/ijerph15112418 . Crossref, Medline,

Menezes-Filho JA, de Carvalho-Vivas CF, Viana GFS, Ferreira JRD, Nunes LS, Mergler D, et al. 2014. Elevated manganese exposure and school-aged children’s behavior: a gender-stratified analysis. Neurotoxicology 45:293–300, PMID:24121006,10.1016/j.neuro.2013.09.006 . Crossref, Medline,

Menezes-Filho JA, Novaes Cde O, Moreira JC, Sarcinelli PN, Mergler D . 2011. Elevated manganese and cognitive performance in school-aged children and their mothers. Environ Res 111(1):156–163, PMID:20943219,10.1016/j.envres.2010.09.006 . Crossref, Medline,

Mikulewicz M, Chojnacka K, Gedrange T, Górecki H . 2013. Reference values of elements in human hair: a systematic review. Environ Toxicol Pharmacol 36(3):1077–1086, PMID:24141206,10.1016/j.etap.2013.09.012 . Crossref, Medline,

National Longitudinal Surveys. 1979. Appendix A: HOME-SF Scales (NLSY79 Child). https://www.nlsinfo.org/content/cohorts/nlsy79-children/other-documentation/codebook-supplement/appendix-home-sf-scales/page/0/0/#AppendixA1 [accessed 15 December 2016 ].

Ntihabose R, Surette C, Foucher D, Clarisse O, Bouchard MF . 2018. Assessment of saliva, hair and toenails as biomarkers of low level exposure to manganese from drinking water in children. Neurotoxicology 64:126–133, PMID:28867366,10.1016/j.neuro.2017.08.011 . Crossref, Medline,

Obrenovich ME, Shamberger RJ, Lonsdale D . 2011. Altered heavy metals and transketolase found in autistic spectrum disorder. Biol Trace Elem Res 144(1–3):475–486, PMID:21755304,10.1007/s12011-011-9146-2 . Crossref, Medline,

Olsen SE, Tangstad M, Lindstad T . 2007. Production of Manganese Ferroalloys. Trondheim, Norway: Tapir Akademisk Forlag.

Otten JJ, Hellwig JP, Meyers LD . 2006. Dietary Reference Intakes: the Essential Guide to Nutrient Requirements. Washington, DC: National Academies Press.

Park S, Sim C-S, Lee H, Kim Y . 2013. Blood manganese concentration is elevated in infants with iron deficiency. Biol Trace Elem Res 155(2):184–189, PMID:23955423,10.1007/s12011-013-9782-9 . Crossref, Medline,

Randall JA, Gibson RS . 1989. Hair chromium as an index of chromium exposure of tannery workers. Br J Ind Med 46(3):171–175, PMID:2930727,10.1136/oem.46.3.171 . Crossref, Medline,

Riojas-Rodríguez H, Solís-Vivanco R, Schilmann A, Montes S, Rodríguez S, Ríos C, et al. 2010. Intellectual function in Mexican children living in a mining area and environmentally exposed to manganese. Environ Health Perspect 118(10):1465–1470, PMID:20936744,10.1289/ehp.0901229 . Link,

Robison G, Zakharova T, Fu S, Jiang W, Fulper R, Barrea R, et al. 2013. X-ray fluorescence imaging of the hippocampal formation after manganese exposure. Metallomics 5(11):1554–1565, PMID:23999853,10.1039/c3mt00133d . Crossref, Medline,

Rodrigues JLG, Araújo CFS, Dos Santos NR, Bandeira MJ, Anjos ALS, Carvalho CF, et al. 2018. Airborne manganese exposure and neurobehavior in school-aged children living near a ferro-manganese alloy plant. Environ Res 167:66–77, PMID:30007874,10.1016/j.envres.2018.07.007 . Crossref, Medline,

Rosner B . 1983. Percentage points for a generalized ESD many-outlier procedure. Technometrics 25(2):165–172,10.2307/1268549 . Crossref,

Rubin DB . 2004. Multiple Imputation for Nonresponse in Surveys. Hoboken, NJ: Wiley.

Sander MC, Lindenberger U, Werkle-Bergner M . 2012. Lifespan age differences in working memory: a two-component framework. Neurosci Biobehav Rev 36(9):2007–2033, PMID:22771333,10.1016/j.neubiorev.2012.06.004 . Crossref, Medline,

Scheiber IF, Mercer JFB, Dringen R . 2014. Metabolism and functions of copper in brain. Prog Neurobiol 116:33–57, PMID:24440710,10.1016/j.pneurobio.2014.01.002 . Crossref, Medline,

Skröder H, Kippler M, Nermell B, Tofail F, Levi M, Rahman SM, et al. 2017. Major limitations in using element concentrations in hair as biomarkers of exposure to toxic and essential trace elements in children. Environ Health Perspect 125(6):067021, PMID:28669939,10.1289/EHP1239 . Link,

Smith D, Gwiazda R, Bowler R, Roels H, Park R, Taicher C, et al. 2007. Biomarkers of Mn exposure in humans. Am J Ind Med 50(11):801–811, PMID:17924418,10.1002/ajim.20506 . Crossref, Medline,

Smith EA, Newland P, Bestwick KG, Ahmed N . 2013. Increased whole blood manganese concentrations observed in children with iron deficiency anaemia. J Trace Elem Med Biol 27(1):65–69, PMID:22940083,10.1016/j.jtemb.2012.07.002 . Crossref, Medline,

Sowell ER, Thompson PM, Tessner KD, Toga AW . 2001. Mapping continued brain growth and gray matter density reduction in dorsal frontal cortex: inverse relationships during postadolescent brain maturation. J Neurosci 21(22):8819–8829, PMID:11698594,10.1523/JNEUROSCI.21-22-08819.2001 . Crossref, Medline,

Taylor KW, Joubert BR, Braun JM, Dilworth C, Gennings C, Hauser R, et al. 2016. Statistical approaches for assessing health effects of environmental chemical mixtures in epidemiology: lessons from an innovative workshop. Environ Health Perspect 124(12):A227–A229, PMID:27905274,10.1289/EHP547 . Link,

Tjalkens RB, Zoran MJ, Mohl B, Barhoumi R . 2006. Manganese suppresses ATP-dependent intercellular calcium waves in astrocyte networks through alteration of mitochondrial and endoplasmic reticulum calcium dynamics. Brain Res 1113(1):210–219, PMID:16934782,10.1016/j.brainres.2006.07.053 . Crossref, Medline,

Torres-Agustín R, Rodríguez-Agudelo Y, Schilmann A, Solís-Vivanco R, Montes S, Riojas-Rodríguez H, et al. 2013. Effect of environmental manganese exposure on verbal learning and memory in Mexican children. Environ Res 121:39–44, PMID:23141434,10.1016/j.envres.2012.10.007 . Crossref, Medline,

U.S. EPA (U.S. Environmental Protection Agency). 2008. Copper Facts. EPA 738-F-06-014. https://www3.epa.gov/pesticides/chem_search/reg_actions/reregistration/fs_G-26_1-Jun-08.pdf [accessed 22 August 2020 ].

Valeri L, Mazumdar MM, Bobb JF, Claus Henn B, Rodrigues E, Sharif OIA . 2017. The joint effect of prenatal exposure to metal mixtures on neurodevelopmental outcomes at 20–24 months of age: evidence from rural Bangladesh. Environ Health Perspect 125(6):067015, PMID:28669934,10.1289/EHP614 . Link,

van Buuren S, Groothuis-Oudshoorn K . 2011. Mice: multivariate imputation by chained equations in R. J Stat Softw 45(3):1–67,10.18637/jss.v045.i03 . Crossref,

van Wendel de Joode B, Mora AM, Lindh CH, Hernández-Bonilla D, Córdoba L, Wesseling C, et al. 2016. Pesticide exposure and neurodevelopment in children aged 6–9 years from Talamanca, Costa Rica. Cortex 85:137–150, PMID:27773359,10.1016/j.cortex.2016.09.003 . Crossref, Medline,

Viana GFS, de Carvalho CF, Nunes LS, Rodrigues JLG, Ribeiro NS, de Almeida DA, et al. 2014. Noninvasive biomarkers of manganese exposure and neuropsychological effects in environmentally exposed adults in Brazil. Toxicol Lett 231(2):169–178, PMID:24992226,10.1016/j.toxlet.2014.06.018 . Crossref, Medline,

Wahlstrom D, Collins P, White T, Luciana M . 2010. Developmental changes in dopamine neurotransmission in adolescence: behavioral implications and issues in assessment. Brain Cogn 72(1):146–159, PMID:19944514,10.1016/j.bandc.2009.10.013 . Crossref, Medline,

Wang D, Du X, Zheng W . 2008. Alteration of saliva and serum concentrations of manganese, copper, zinc, cadmium and lead among career welders. Toxicol Lett 176(1):40–47, PMID:18054180,10.1016/j.toxlet.2007.10.003 . Crossref, Medline,

Wasserman GA, Liu X, Parvez F, Chen Y, Factor-Litvak P, LoIacono NJ, et al. 2018. A cross-sectional study of water arsenic exposure and intellectual function in adolescence in Araihazar, Bangladesh. Environ Int 118:304–313, PMID:29933234,10.1016/j.envint.2018.05.037 . Crossref, Medline,

Wasserman GA, Liu X, Parvez F, Factor-Litvak P, Ahsan H, Levy D, et al. 2011. Arsenic and manganese exposure and children’s intellectual function. Neurotoxicology 32(4):450–457, PMID:21453724,10.1016/j.neuro.2011.03.009 . Crossref, Medline,

Wechsler D, Orsini A, Picone L . 2006. WISC-3: Wechsler Intelligence Scale for Children. [In Italian.] 3rd ed. Florence, Italy: Giunti Pyschometrics.

White IR, Royston P, Wood AM . 2011. Multiple imputation using chained equations: issues and guidance for practice. Stat Med 30(4):377–399, PMID:21225900,10.1002/sim.4067 . Crossref, Medline,

Wright RO, Amarasiriwardena C, Woolf AD, Jim R, Bellinger DC . 2006. Neuropsychological correlates of hair arsenic, manganese, and cadmium levels in school-age children residing near a hazardous waste site. Neurotoxicology 27(2):210–216, PMID:16310252,10.1016/j.neuro.2005.10.001 . Crossref, Medline,

Zahran S, Mielke HW, Weiler S, Hempel L, Berry KJ, Gonzales CR . 2012. Associations between standardized school performance tests and mixtures of Pb, Zn, Cd, Ni, Mn, Cu, Cr, Co, and V in community soils of New Orleans. Environ Pollut 169:128–135, PMID:22705504,10.1016/j.envpol.2012.05.019 . Crossref, Medline,

Zheng W, Jiang Y-M, Zhang Y, Jiang W, Wang X, Cowan DM . 2009. Chelation therapy of manganese intoxication with para-aminosalicylic acid (PAS) in Sprague–Dawley rats. Neurotoxicology 30(2):240–248, PMID:19150464,10.1016/j.neuro.2008.12.007 . Crossref, Medline,

Zhou G, Ji X, Cui N, Cao S, Liu C, Liu J . 2015. Association between serum copper status and working memory in schoolchildren. Nutrients 7(9):7185–7196, PMID:26343713,10.3390/nu7095331 . Crossref, Medline,

Zhou YZ, Chen J, Shi XJ, Zou Y, Shen XB, Zheng W, et al. 2010. Early biological markers of manganese exposure. [In Chinese.] Chinese J Ind Hyg Occup Dis 28(9):645–647, PMID:21126474 . Medline,