1. Introduction
Coronavirus disease 2019 (COVID-19) caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has resulted in unprecedented disruptions to society, but also led to impressive innovations in the fields of diagnostics, vaccines, and therapeutics. Serology assays have been and continue to be vital for managing COVID-19 [1,2]. Neutralizing antibody titers have been widely used to assess the vaccine efficacy for immunological correlates of protection (CoPs) for investigational and licensed vaccines, including the most recent bivalent boosters [3,4]. In this context, a CoP is defined as an immune marker that can be used to reliably predict a vaccine’s efficacy in preventing a clinically significant outcome [5,6,7]. Anti-SARS-CoV-2 spike antibody levels can also be used for establishing CoPs, as supported by animal studies [8,9], natural infection cohorts [10], and vaccine trial studies [11]. The US Food and Drug Administration (FDA) and European Medicines Agency (EMA) have since accepted these two CoPs, anti-spike-binding antibody titer and neutralization antibody titer for vaccine assessment and approval [7].
Serological binding assays are known to have large variations across different laboratories due to the multi-component nature of the assays [2,12]. Among the SARS-CoV-2 serological binding assays, immobilized recombinant antigens, either the full spike protein, receptor-binding domain (RBD), subunit 1 (S1) of the spike protein, or nucleocapsid protein (N), have been utilized for detecting different isotypes of antiviral antibodies with different detection modalities [13]. A binding assay can be designed using different formats, such as bead-based and plate-based, further introducing measurement variabilities [14]. These complexities have made it difficult to evaluate assay accuracy, precision, robustness and compare results obtained from different assays. The FDA removed 27 serology tests from its Emergency Use Authorization (EUA) in May 2020 due to the lack of assay validation data and potential risks to public health. In response, the World Health Organization (WHO), in collaboration with the National Institute for Biological Standards and Control (NIBSC) initiated an effort to develop the first WHO international standard (WHO IS) and reference panel for anti-SARS-CoV-2 antibody for normalizing serological assays using a pool of plasma from 11 SARS-CoV-2 convalescent patients. Since the establishment of the WHO IS in November 2020 [15], several studies were conducted for evaluating its suitability for COVID-19 serology assay [16,17,18]. The conclusion of these investigations was that normalization using the WHO IS improved analytical and diagnostic comparability by placing results on a similar scale, but did not remove any sources of variability, including those associated with different assay reagents and platforms/formats. Importantly, the lack of uncertainty quantification hindered assay harmonization and the quantitative assessment of assay comparability. In addition, the rapid uptake of this standard led to a depletion of stock by August 2021. The establishment of the second WHO IS for anti-SARS-CoV-2 immunoglobulin and a reference panel for antibodies to SARS-CoV-2 variants of concern [19] that occurred in July 2022 and the subsequent expansion of the WHO reference panel for antibodies to SARS-CoV-2 variants of concern [20] that took place in March 2023 more directly addressed the reference standard needs for the recently emerged and widely circulated Omicron variants.
Due to biological complexities, the standardization of serology assays has been extremely challenging [21]. It is generally accepted that only matrix-matched and pooled convalescent reference standards have the breadth of epitopes necessary for standardizing various serology assays. However, to the best of our knowledge, there are no definitive data supporting this claim for SARS-CoV-2. Moreover, the curation, pooling, preparation, and testing of large volumes of convalescent patient samples is a time-consuming process, as shown by the WHO standardization effort. As SARS-CoV-2 continues to rapidly evolve [22], it becomes even more challenging to keep pace with the most current variants to obtain a pure convalescent sample from a specific viral variant serving as an assay reference standard for the variant-specific assay. In addition, pooled samples are exceedingly complex; so, bridging and/or traceability concepts are nearly impossible to implement.
Monoclonal antibodies (mAb), including anti-SARS-CoV-2 spike-neutralizing antibodies (nAb), can be generated rapidly with the current technologies [3,23]. mAbs have been used extensively as in-house reference materials for serology but have not been considered for broad use. In this paper, we conduct a small-scale interlaboratory serology study consisting of WHO IS, three anti-SARS-CoV-2 spike mAbs, and 62 human serum/plasma samples to (1) evaluate the suitability of an mAb panel to enable comparability and standardized results of serological binding and neutralization assays, (2) understand the strengths and weaknesses of different neutralization assay formats, and (3) establish more predictive methods for correlated protection for SARS-CoV-2.
2. Results
2.1. Serological Binding Assays
Figure 1a shows the ‘Day 1’ measured dilution curves for all 62 samples along with the WHO IS obtained by one lab. Figure 1b illustrates the corresponding data collapse onto the dilution curve of the WHO IS. Note that we rotated the mean fluorescence intensity (MFI)readout in Figure 1a to the horizontal axis of Figure 1b to streamline the normalization algorithm [24]. In Figure 1a, and as expressed by Equation (4), the magnitude of the translation factor on the log scale for each curve relative to that of the standard is equal in magnitude to the scaled antibody concentration This new analysis method inherently reduces many sources of variability, and surprisingly, we were able to determine values with precise uncertainty quantifications for what is generally thought to be semi-quantitative assays. We repeated this process for all combinations of standards and assays. The average log antibody concentration, determined from measured on separate days, was used in all subsequent analyses.
Figure 2 shows the results from all participants normalized and harmonized to the WHO IS using the new analysis methods described above (Equations (4)–(6)), where the samples are ordered from the lowest to largest value of log antibody consensus concentration. The average of each log concentration normalized to the WHO IS clearly shows a systematic bias for each assay relative to the others that corresponds to the systemic bias (Figure 2a). The consensus antibody concentrations cs,r determined using Equation (5) are shown as red squares along with harmonized antibody concentrations , as shown in Figure 2b. The removal of contributions from clearly improved agreements among all laboratories, with the results better coalescing around cs,r. The randomness associated with assay- and sample-dependent uncertainties and , respectively, were quantified, where differences from different laboratories/methods are clear (Figure 2c).
When the same analysis procedure was implemented using each of the three mAb as the standard, we observed the following: identical trends for , a slightly different value of cs,r due to the reference-dependent bias , and identical values for random assay- and sample-dependent uncertainties (Appendix B, Figure A1), all consistent with the predictions from the above equation and our theory [24]. Figure 3 summarizes the log consensus values for each sample using the different standards. Note that, while the consensus values for a fixed sample differ according to standard, the difference is sample-independent. In other words, the consensus values for all standards are interchangeable up to a constant, which is straightforward to determine using any sample. This ambiguity in the definition of the consensus value arises from the fact that only differences between Gibbs free energies are meaningful, not the free energies per se.
2.2. Neutralization Assays
The NT50 values were determined using the Hill equation described in the ‘Centralized Data Analysis’. Figure 4 shows the results of the Wilcoxon matched-pairs signed-rank tests [25] performed on the bead-based surrogate, pseudovirus-based neutralization assay (pvNT), and the live-virus microneutralization assay (MN). Each assay pair was significantly different. Based on a limit of detection (LOD) of 8.94 for the surrogate assay determined using the 24 negative samples, the surrogate assay identified 28 of 39 positive serum samples, including the WHO IS (Figure 4a,c). The Spearman correlation was 0.928, p < 0.000, between pvNT and the surrogate assay (Figure 4a); 0.866, p < 0.0001, between pvNT and the MN assay (Figure 4b); and 0.738, p < 0.0001, between MN and the surrogate assay.
We further compared the three different neutralization assays using receiver operating characteristic (ROC) curves to evaluate the performance of the assays (Figure 4d) [26]. The optimal threshold for pvNT was an NT50 of 2.00, yielding a sensitivity of 87.2% and a specificity of 100%. There were no false positives reported, but some false negatives. For the surrogate assay, the optimal threshold based on the ROC curve was an NT50 of 8.87, with a sensitivity of 70.3% and a specificity of 92.3%. This optimal threshold is in good agreement with the LOD of 8.94 noted above. Lastly, the ROC analysis led to a determination of an optimal NT50 threshold of 47.50, with a sensitivity of 86.6% and a specificity of 79.2%, for MN. The values of the area under curve (AUC) were 0.931 for pvNT, 0.888 for the surrogate assay, and 0.885 for MN. A higher AUC indicates that an assay is better able to identify neutralization.
2.3. Probability for CoPs
Considering pvNT as an optimal predictor of the presence of the neutralizing antibodies in this study, we constructed the probability model connecting consensus-binding concentrations to their neutralization counterparts. The left plot of Figure 5 shows the relationship expressed by Equation (8) for mAb 1 vs. the NT50 values from pvNT. It is notable that the mean relationship is not linear. The physical origins of this discrepancy are likely associated with the complexity of biological responses and point to the differing information content provided by the binding and neutralizing assays.
From a practical standpoint, however, the relationship expressed by Equation (8) and illustrated in Figure 5 (left) still provides useful information. For example, given a desired NT50 level , we can determine the minimum consensus-binding concentration that would yield a neutralization measurement with a probability of 95%. Figure 5 (right) provides a probability function where the vertical line indicates that a log consensus-binding level of ~8.1 yields a >95% chance that the neutralizing titer is greater than 40 (exp(3.68)). While not pursued in this work, it is possible to incorporate the residual uncertainty associated with the variance of to estimate the harmonized binding level from a given assay for which that corresponds to with a desired probability. Such tasks are assay-specific and will be addressed in future work. In either case, however, it is straightforward to show that changing the reference material only rescales the coefficients associated with the deterministic part of Equation (8) but does not otherwise modify the probability model associated with the plot shown in Figure 5 (left). Thus, a decision based on the correlates of protection, as we defined, is also invariant to the reference material used.
3. Discussion
3.1. Serological Binding Assays
The results shown in Figure 2 and Figure A1, together with the corresponding theories [24], clearly demonstrate that the choice of reference does not affect the harmonized antibody concentration . In other words, all binding references performed equivalently for the purposes of harmonization when using our thermodynamically derived analysis, as shown in Figure 3, successfully meeting the first objective of the interlaboratory study. This critical finding will allow a more rapid development of an mAb-based reference standard to support assay comparability, particularly during the early onset of an outbreak. As we witnessed for COVID-19, the development of pooled human serum reference materials was a lengthy effort that cannot keep pace with the rapidly evolving virus. Moreover, our analyses enable the quantitatively bridging of different reference standards, since their respective, harmonized concentrations only differ by constant biases associated with . This aspect of our approach is extremely useful if multiple standards are developed and deployed by the community, as in the case of COVID-19.
Our analysis examines the choice of reference through the lens of uncertainty quantification (UQ). Specifically, our new analysis method (Equation (5)) shows that the average log antibody concentration an experimentally determined dimensionless value, is composed of four components: consensus antibody concentration cs,r, assay- and reference-dependent (systematic) bias , and assay- and sample-dependent uncertainties and , respectively. The contributions od and are important because cs,r cannot be determined in a diagnostic setting where only one assay is used. Thus, the harmonized measurement is our best estimate of cs,r, and the variance of quantifies the extent to which sample–assay effects render the consensus unknown.
Figure 6 (left) examines the combined contribution from and in absolute values. Note that thermodynamics underpinning this led to the opposite sign of and in Equation (5); this means that their contributions can be canceled out if an improper experimental design is conducted when developing a reference. A comparison of the value determined when using the WHO IS versus an mAb provides a quantitative measure of the additional variability introduced when an mAb is used as the normalization standard (not the harmonization standard). In essence, mAbs can be used as a normalization standard, albeit with increased (~8x) variabilities as compared to WHO IS, consistent with the notion that a pooled sample serves as a better standard for normalization. However, this variation is all due to bias, which can be quantitatively removed, leaving as the only thermodynamic source of variability and therefore the identical performance of mAb and pooled convalescence standard via harmonization. Figure 6 (right) shows the per-lab estimate of the standard deviation in computed using each of the reference materials, an expansion of the orange portion of the Figure 6a that can be interpreted as the uncertainty in harmonized (i.e., bias-corrected) concentrations across the samples. Importantly, does not change (to within statistical uncertainty) according to the reference material, further supporting the notion that random bias arises from the samples and not from the reference interaction with the assay. By comparison, Labs 2, 3, and 4 demonstrated exceptional robustness/precision.
One unexpected finding is that, through the new analysis methods, we were able to generate quantitative metrics, namely, cs,r, , and the variance associated with for all assays, including those typically considered qualitative. This reflects the fact that, through the collection of a larger dataset to generate a master curve from sample dilution curves (Figure 1), not just a small linear range of sample dilution curves, a common practice in serology, we can reduce and/or remove the large bias arising from any given data point. This further supports the strengths of our method in normalizing and harmonizing a more diverse set of assay formats and predicting robust assays through uncertainty quantification.
3.2. Neutralization Assays
Our interlaboratory study includes assays spanning live-virus MN assay to surrogate the neutralization assay, enabling the assessment of the strengths and weakness of different neutralization assay formats. Live-virus MN assays are thought to provide the most biologically relevant data but must be conducted in a biosafety level 3 (BSL3) containment facility. They are also time-consuming, labor-intensive, and known to have relatively large uncertainties [27]. Hence, pvNT and non-cell based surrogate assays that can be performed in a BSL2 facility have gained enormous attraction. pvNT arguably mimics the live-virus MN assay better, whereas non-cell-based surrogate assays are inherently better controlled with an expected higher reproducibility. However, there are limited data that directly compares the performance of different neutralization assays formats [28]. Since we have results from each assay category, we can direct compare these. Note that, since neutralization studies were not performed in the same manner as serological anti-spike assays, we could not use the thermodynamic-based harmonization analysis method for centralized analysis.
The surrogate assay clearly showed the highest precision (Figure 4). Consistent with previous reports, the MN assay showed large uncertainties. By the ROC analysis, the pvNT and MN assays show similar sensitivity, 87.2% and 86.6%, respectively, indicating pvNT is a good alternative to the more complex MN assay. The faster surrogate assay showed the lowest sensitivity (70.3%). Interestingly, the specificity of pvNT was 100%, followed by the surrogate assay at 92.3%. MN showed the lowest specificity at 79.2%, indicative of the significant number of false positives detected, perhaps due to cross-reactivity. Although pvNT has the lowest detection threshold based on the ROC curve (label not visible), it is accepted that pvNT will unlikely reach 100% sensitivity because the selected COVID-19-positive (determined by PCR analysis) samples may, in fact, contain a low IgG titer. The lower sensitivity of the RBD-based surrogate assay may also be due to contribution from the N-terminal of the spike protein other than the RBD to prevent the virus’s entry into the target cells [29]. From the ROC curve analyses and AUC values, pvNT emerges as the best-performing neutralization assay for this study, although we note the choice of assay should ultimately be selected based on fit for purpose.
3.3. Correlates of Protection (CoPs)
A fundamental goal of serology testing is to understand the extent to which an individual is protected from further infection. Because neutralization assays are essential for determining the infection rate, predicted humoral protection, as well as vaccine efficacy during clinical trials and after large-scale vaccination, the different platforms of neutralization assays, including MN, pvNT, and surrogate assay, have been developed [30]. An increasing body of literature supports the neutralization assay and the clinical outcome. In this section, we make several observations and conclusions regarding the use of serological assays as CoPs on the basis of our work.
Chief among these is the recognition that assessing immunity, whether expressed in NT50 levels or otherwise, amounts to decision making under uncertainty. This is most clearly expressed by Figure 5, which shows that consensus-binding measurements are an imperfect proxy for neutralization levels. If we wish to deduce the latter from the former, we incur a penalty in the form of increased uncertainty in the NT50 value. This observation holds more generally when we replace one measurement with another, as is often the case when a gold standard is expensive to deploy but a surrogate is not. A key goal of our analysis has therefore been to estimate this uncertainty “penalty”. This can be used to make quantitative statements to the effect that a certain binding level ensures a minimum needed neutralization level with sufficient probability (e.g., 95%). It is likely that such probabilistic statements are the most practical and accessible routes to supporting decision-making processes in diagnostics.
This approach to UQ-based decision making is also well adapted to incorporating information from our harmonization analysis. As discussed in previous sections, the uncertainty quantifies the extent to which the consensus value is unknown, given the harmonized (i.e., bias-corrected) measurement associated with a specific assay. Through well-known uncertainty propagation techniques, the variance can be incorporated into the uncertainty model associated with Equation (8) and the binding–neutralization relationship. This allows for a decision to be made directly on the basis of a specific serology measurement, not just a consensus concentration, which is in general unknown; see also ref. [24] for more details. In this way, the UQ also provides a route for determining which binding assay is a best proxy for neutralization measurements. Moreover, it facilitates downstream analyses that can connect uncertainty associated with neutralization measurements to more real-world measures of protection, e.g., associated with clinical outcomes and individual risk levels.
4. Materials and Methods
4.1. Interlaboratory Study Design
The study materials consisted of WHO IS, 3 anti-SARS-CoV-2 spike mAbs, and 62 human serum/plasma samples. Two of the three mAbs were provided by Regeneron via a material transfer agreement. Another mAb and 47 convalescent serum samples consisting of 24 negative samples (collected pre-pandemic) and 23 positive samples (natural infection verified by PCR tests) were donated by Abbott Laboratories. Fifteen convalescent plasma/serum samples provided by CDC were from vaccinated individuals, and five samples from each vaccine modality: Johnson and Johnson, Moderna, and Pfizer. All convalescent sample curations were IRB-approved by the respective organizations. The study was also approved by the IRB office of the National Institute of Standards and Technology (NIST) through an individual material transfer agreement with each participating organization.
A detailed study design is provided in the Supplementary Materials. A study protocol was provided to participants to specify the requirements for preparing and handling samples and controls. A reporting template was provided to harmonize the reporting of assay information and results.
Six serological binding assays and three neutralization assays were incorporated in the centralized data analysis of this interlaboratory study (see Table 1 and Appendix A for more information).
4.2. Centralized Serology Data Analysis
Data analysis proceeds via a hierarchy of methods motivated by concepts from thermodynamics and conditional probability. A core idea of this analysis is that normalization and harmonization are distinct tasks. A complete justification is provided in our companion paper [24]. A brief explanation is provided below.
Serology assays always involve a reversible binding interaction of the form,
(1)
where Y is an antibody, B is a substrate, and is an antibody–substrate complex. When used as a measurement tool, such assays quantify not only the properties of the antibodies but also the substrate, e.g., binding for a given epitope. Stated differently, serology measures the properties of a molecular interaction, in this case, the equilibrium binding between antibody in blood serum and the antigen-bound substrate. As such, it is impossible to fully remove effects of the substrate from the process of quantifying antibody concentration. This means the measurement does not determine the reactants or antibody concentrations, even though the serology results are generally reported in terms of antibody titer.This observation motivated us to define normalization as the task of estimating the antibody quantity relative to a chosen reference for a fixed assay. The normalized antibody concentration is
(2)
where is the dimensionless, scaled antibody concentration of sample s relative to reference r using assay n (Equation (3)) and B is an arbitrarily assigned binding antibody unit (BAU) concentration, e.g., associated with the WHO IS or another standard. Here and throughout the paper, the subscripts s, n, and r are reserved for a sample, assay, and reference, respectively. is defined as(3)
where and are the absolute concentrations of bound antibodies associated with the sample and reference, respectively. Note that the reference is generally a calibrator for the specific assay. In practice, is generally calculated via the ratio of mean fluorescence intensity or optical density (OD) in the linear range of the log–log curve of serology assays.To better utilize the entire set of data from the dilution measurements that reduces random variations, we developed a novel analysis method that generalizes the slope-correction method [31] and directly estimates
(4)
where is the translation factor for each sample, such that the fluorescence values as a function of dilution all collapse onto the master curve (see Supplementary Materials). In other words, the scaling factor laterally translates the dilution curve when considering fluorescence as a function of , where is the log-scaled antibody concentration.In contrast, harmonization is intuitively defined as the process of altering the numerical values of normalized concentrations so that the (normalized) measurements of each assay become interchangeable for a fixed sample. This leads to the concept of a consensus antibody concentration for each sample, to which all harmonized measurements are sufficiently close, i.e., within a quantifiable uncertainty. More specifically, thermodynamic arguments imply that the experimentally determined, average (e.g., as measured over multiple days), log-scaled antibody concentration is comprised of four components according to
(5)
where and are Gibbs free energies associated with the reference–assay pair and sample–assay pair, respectively, and is a sample-dependent day-to-day variation associated with instrument noise, operator effects, etc. [32]. Importantly, introduces a systematic bias that depends on the choice of the reference, whereas introduces random bias, expressed as variance . Note that, although the values of depend on the reference used, they are readily interchangeable among different references.Equation (5) provides the bases for determining the harmonized antibody concentration, , via
(6)
Effectively, is the reference–assay dependent, systematic-bias-corrected, averaged log antibody concentration or the consensus antibody concentration together with the random assay–sample-dependent uncertainties. Equation (6) is key to understanding the uncertainties from a given serology assay.
4.3. Neutralization Assays
For serology measurements, diluting samples containing antiviral antibodies changes the number of antibodies bound to the assay surface, as given by the Hill equation, shown in Equation (7) [33].
(7)
where NAb is the total number of labeled Ab immobilized on surface and Nmax and Nmin are the maximum and minimum numbers of the labeled NAb, respectively. NT50 equals to the dilution at which half of the immobilized antigen molecules are filled and n is the Hill coefficient. Equation (7) is an excellent representation of Gibbs free energy function characterizing the thermodynamic interaction system between immobilized antigen molecules and antiviral antibodies in the sample. A detailed procedure for applying Equation (7) for the three neutralization assays platformed is provided in the Supplementary Materials.4.4. Probability Models for CoPs
To construct the probability models that connect binding and neutralizing assays, we used the consensus values for a fixed reference and the harmonic mean (across days) of NT50 values associated with a single neutralizing assay. It stands to reason that, on average, increasing consensus values should correspond to increasing NT50 estimates. However, given the uncertain biochemical connection between binding measurements and their neutralization counterparts, we anticipated that the relationship between them is (i) not necessarily linear and (ii) most likely partly random.
This motivated us to postulate a minimal model in which is a low-order polynomial of with a constant-variance Gaussian noise. More precisely, we posited that
(8)
where the and are unknown parameters and is a normal random variable with an unspecified mean and variance. We determined these unknown parameters using the maximum likelihood analysis.4.5. Disclaimer
To specify an experimental procedure as completely as possible, certain commercial materials, instruments, and equipment were identified in this manuscript. In no case does the identification of the manufacturer of particular equipment or materials imply a recommendation or endorsement by the National Institute of Standards and Technology or the Centers for Disease Control and Prevention, nor does it imply that the materials, instruments, and equipment identified are necessarily the best available for the purpose. The findings and conclusions in this article are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention or the Agency for Toxic Substances and Disease Registry.
5. Conclusions
The results from this interlaboratory study and accompanying new analysis methods provide a means to achieve unprecedented serological-binding assay harmonization. Importantly, our analysis methods afford a more rigorous quantitation of antibody concentrations as well as sources of uncertainties, thus allowing a better understanding of assay performance. Generally, harmonization is sought through experimental design that requires all laboratories use the same reagents to perform the identical assay. Because of logistics and supply chain issues associated with this method of assay harmonization and the existence of diverse serological assay platforms using different assay reagents during the COVID-19 pandemic, the harmonization of serological-binding assays has not been accomplished at present. In this paper, we conclusively demonstrated that a single anti-spike mAb can be used alone for assay harmonization when utilizing the new analysis method. Considering the presence of different viral variants respective to the original Wuhan-Hu-1 strain investigated in this study, a cocktail of mAbs with different binding specificities to different viral strains would be better to harmonize serological-binding assays across different viral strains. This means that mAb-based reference materials can be rapidly developed to support emerging infections. Together, and for the first time, we can establish methods to enable traceability and comparability across different assay platforms and reference materials.
This interlaboratory study also directly compared surrogate, pvNT, and MN neutralization assays and showed that the pvNT assay had the highest sensitivity and specificity compared to the other methods. Using NT50 values from the pvNT assay, we showed good correlations with binding assays used in the study, consistent with literature reports of the two markers, anti-spike antibody titer and neutralizing antibody titer, serving as CoPs for the efficacy evaluation of vaccines and COVID-19 disease management.
This approach was developed and demonstrated for COVID-19, although we expect the thermodynamics underpinning of binding assays to be generally applicable to broader serological assays, including efforts to develop new vaccines for a variety of diseases, such as RSV and pan-influenza, for seroprevalence monitoring and in the assessment of pre-existing immunity prior to the administration of therapeutics, such as emerging gene therapies.
Conceptualization, L.W. and S.L.-G.; methodology, L.W. and S.L.-G.; software, P.N.P., A.J.K. and A.K.G.; validation, J.R.I., J.C.P., H.J.K., W.T., M.K., H.X., L.T., E.B.E., E.J.K., T.K. and S.J.; formal analysis, P.N.P., A.K.G. and A.J.K.; investigation, J.R.I., J.C.P., H.J.K., W.T., M.K., H.X., L.T., E.B.E., E.J.K., T.K. and S.J.; resources, L.W., S.L.-G., J.C.P., N.T., L.C.M. and A.V.G.; data curation, P.N.P., A.J.K. and A.K.G.; writing—original draft preparation, L.W. and S.L.-G.; writing—review and editing, L.W., S.L.-G., P.N.P. and A.J.K.; visualization, P.N.P., J.R.I. and L.W.; supervision, L.W. and S.L.-G.; project administration, L.W. and P.N.P.; funding acquisition, S.L.-G. All authors have read and agreed to the published version of the manuscript.
The study was approved by the Institutional Review Board of NIST through an individual material transfer agreement with each participating institution under the organizational project MML_2019-0125.
Informed consent was obtained from all subjects involved in the study by the respective participating institution prior to the donation of human serum samples to NIST for the study.
All study raw data reported by participants are saved in a shared folder of the study kept by P.N.P. and A.J.K. at NIST and are available upon request.
The authors are indebted to Ligia Pinto at the Frederick National Laboratory for Cancer Research (FNLCR) for commenting on and support of the study; Kristen Tramaglini, Alina Baum, and Christo Kyratsous at Regeneron for providing two anti-SARS-CoV-2 spike mAbs for the study; Sumeet Poudel and Jaime Almeida at NIST Biosystems and Biomaterials Division (BBD) for the preparation of blinded study samples; and Tara Eskandari and Shaswat Koirala at BBD and the Staff at the Office of Reference Materials of NIST for ensuring the flawless shipment of the study materials to the participants. SARS-CoV-2 clinical isolates (i.e., D614G, Alpha, Beta, and Delta) were obtained through BEI Resources, NIAID, and NIH: SARS-Related Coronavirus 2.
The authors declare no conflict of interest.
Footnotes
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Figure 1. (a) Sixty-two sample dilution curves from experiment Day 1 obtained by ‘Lab 1’. (b) Shifting of 62 dilution curves from (a) with respect to the antibody level to the WHO IS yields a single normalized master curve of MFI as a function of the normalized antibody concentration. Note that the axes in (b) were switched to allow the mathematical calculation of the optimization functions.
Figure 2. (a) Log-normalized antibody concentrations ln([Forumla omitted. See PDF.]), normalized to the WHO IS for 38 positive convalescent serum samples determined by the six different serological IgG assays labeled as Labs 1–6; (b) log-harmonized antibody concentrations [Forumla omitted. See PDF.] shows a greater concordance with ln(cs,r), shown as red squares; and (c) assay- and sample-dependent uncertainties, [Forumla omitted. See PDF.], which reflect the randomness of each individual’s sample–assay interaction.
Figure 3. Log consensus antibody concentration values associated with each antibody standard. The estimates are ordered by increasing value according to the WHO IS-harmonized measurements. Note that, while the consensus values depend on the reference material, all consensus values for a given reference differ by the same constant relative to another reference across the samples.
Figure 4. Wilcoxon matched-pairs signed-rank tests were performed comparing the pseudovirus-based neutralization assay (pvNT) to surrogate neutralization assay ((a) n = 28, p < 0.0001), pvNT to live virus-based microneutralization assay (MN) ((b) n = 34, p < 0.0001), and surrogate assay to MN ((c) n = 28, p < 0.0001). The error bars are the standard deviations obtained from the sample replicates (n ≥ 3). (d) Comparison of the receiver operating characteristic (ROC) curves and the area under curve (AUC) for three different neutralization assays used in the study.
Figure 5. Left: Consensus-binding concentrations ln(cs,r) associated with mAb 1 compared to the NT50 values associated with pvNT from Lab 1. Data presented are from 37 positive samples. Right: Integral of the probability function shown in the left figure is plotted. The vertical line in the right plot is the binding level (~8.1), for which there is a >95% probability that the neutralizing level is greater than 3.68 (Ln40). Color bars adjacent to the plots indicate the numerical scales of the probabilities. See the Results and Discussion Sections for additional context.
Figure 6. Left: Contribution of bias from [Forumla omitted. See PDF.] and [Forumla omitted. See PDF.] across all assays using each reference material. Note that, although [Forumla omitted. See PDF.] and [Forumla omitted. See PDF.] carry a sign in Equation (7), these show their contributions in absolute values. Right: Average residuals of the bias-corrected antibody concentrations for each lab and reference material.
Serological and neutralization assays used in this interlaboratory study.
Participants | Serological Binding Assay(s) | Neutralization Assay(s) |
---|---|---|
NIST | SARS-CoV-2 spike IgG assay (quantitative, Wuhan-Hu-1) |
|
FDA |
|
Live virus-based microneutralization assay |
FNLCR/NCI |
|
|
Abbott | ARCHITECT i2000SR Immunoassay (quantitative, Wuhan-Hu-1, spike/RBD IgG, chemiluminescent microparticle immunoassay, FDA- and EUA-granted) | |
Roche | Elecsys anti-SARS-CoV-2 S assay on cobas e 801 immunoanalyzer |
Supplementary Materials
The supporting information can be downloaded at:
References
1. Krammer, F.; Simon, V. Serology assays to manage COVID-19. Science; 2020; 368, pp. 1060-1061. [DOI: https://dx.doi.org/10.1126/science.abc1227] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32414781]
2. Gundlapalli, A.V.; Salerno, R.M.; Brooks, J.T.; Averhoff, F.; Petersen, L.; McDonald, L.; Lademarco, M.F. SARS-CoV-2 Serologic Assay Needs for the Next Phase of the US COVID-19 Pandemic Response. Open Forum Infect. Dis.; 2020; 8, ofaa555. [DOI: https://dx.doi.org/10.1093/ofid/ofaa555] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33442555]
3. Marovich, M.; Mascola, J.R.; Cohen, M.S. Monoclonal Antibodies for Prevention and Treatment of COVID-19. JAMA; 2020; 324, pp. 131-132. [DOI: https://dx.doi.org/10.1001/jama.2020.10245] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32539093]
4. Jeyanathan, M.; Afkhami, S.; Smaill, F.; Miller, M.S.; Lichty, B.D.; Xing, Z. Immunological considerations for COVID-19 vaccine strategies. Nat. Rev. Immunol.; 2020; 20, pp. 615-632. [DOI: https://dx.doi.org/10.1038/s41577-020-00434-6]
5. Khoury, D.S.; Cromer, D.; Reynaldi, A.; Schlub, T.E.; Wheatley, A.K.; Juno, J.A.; Subbarao, K.; Kent, S.J.; Triccas, J.A.; Davenport, M.P. Neutralizing antibody levels are highly predictive of immune protection from symptomatic SARS-CoV-2 infection. Nat. Med.; 2021; 27, pp. 1205-1211. [DOI: https://dx.doi.org/10.1038/s41591-021-01377-8] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34002089]
6. Earle, K.A.; Ambrosino, D.M.; Fiore-Gartland, A.; Goldblatt, D.; Gilbert, P.B.; Siber, G.R.; Dull, P.; Plotkin, S.A. Evidence for antibody as a protective correlate for COVID-19 vaccines. Vaccine; 2021; 39, pp. 4423-4428. [DOI: https://dx.doi.org/10.1016/j.vaccine.2021.05.063]
7. Gilbert, P.B.; Donis, R.O.; Koup, R.A.; Fong, Y.; Plotkin, S.A.; Follmann, D. A COVID-19 Milestone Attained—A Correlate of Protection for Vaccines. N. Engl. J. Med.; 2022; 387, pp. 2203-2206. [DOI: https://dx.doi.org/10.1056/NEJMp2211314] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36507702]
8. McMahan, K.; Yu, J.; Mercado, N.B.; Loos, C.; Tostanoski, L.H.; Chandrashekar, A.; Liu, J.; Peter, L.; Atyeo, C.; Zhu, A. et al. Correlates of protection against SARS-CoV-2 in rhesus macaques. Nature; 2021; 590, pp. 630-634. [DOI: https://dx.doi.org/10.1038/s41586-020-03041-6]
9. Corbett, K.S.; Nason, M.C.; Flach, B.; Gagne, M.; O’Connell, S.; Johnston, T.S.; Shah, S.N.; Edara, V.V.; Floyd, K.; Lai, L. et al. Immune correlates of protection by mRNA-1273 vaccine against SARS-CoV-2 in nonhuman primates. Science; 2021; 373, eabj0299. [DOI: https://dx.doi.org/10.1126/science.abj0299]
10. Krammer, F. A correlate of protection for SARS-CoV-2 vaccines is urgently needed. Nat. Med.; 2021; 27, pp. 1147-1148. [DOI: https://dx.doi.org/10.1038/s41591-021-01432-4]
11. Wei, J.; Pouwels, K.B.; Stoesser, N.; Matthews, P.C.; Diamond, I.; Studley, R.; Rourke, E.; Cook, D.; Bell, J.I.; Newton, J.N. et al. Antibody responses and correlates of protection in the general population after two doses of the ChAdOx1 or BNT162b2 vaccines. Nat. Med.; 2022; 28, pp. 1072-1082. [DOI: https://dx.doi.org/10.1038/s41591-022-01721-6] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35165453]
12. Ozçürümez, M.K.; Ambrosch, A.; Frey, O.; Haselmann, V.; Holdenrieder, S.; Kiehntopf, M.; Neumaier, M.; Walter, M.; Wenzel, F.; Wölfel, R. et al. SARS-CoV-2 antibody testing-questions to be asked. J. Allergy Clin. Immunol.; 2020; 146, pp. 35-43. [DOI: https://dx.doi.org/10.1016/j.jaci.2020.05.020] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32479758]
13. FDA, US Food and Drug Administration. Coronavirus Disease 2019 (COVID-19) Emergency Use Authorizations for Medical Devices: In Vitro Diagnostics EUAs. 2020; Available online: https://www.fda.gov/medical-devices/coronavirus-disease-2019-covid-19-emergency-use-authorizations-medical-devices/vitro-diagnostics-euas (accessed on 27 October 2023).
14. Karger, A.B.; Brien, J.D.; Christen, J.M.; Dhakal, S.; Kemp, T.J.; Klein, S.L.; Pinto, L.A.; Premkumar, L.; Roback, J.D.; Binder, R.A. et al. The Serological Sciences Network (SeroNet) for COVID-19: Depth and Breadth of Serology Assays and Plans for Assay Harmonization. mSphere; 2022; 7, e0019322. [DOI: https://dx.doi.org/10.1128/msphere.00193-22] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35703544]
15. Mattiuzzo, G.; Bentley, E.M.; Hassall, M.; Routley, S.; Richardson, S.; Beranasconi, V.; Kristiansen, P.; Harvala, H.; Roberts, D.; Semple, M.G. et al. Establishment of the WHO International Standard and Reference Panel for Anti-SARS-CoV-2 Antibody; WHO/BS/2020.2403 World Health Organization: Hertfordshire, UK, 2020; Available online: https://www.who.int/publications/m/item/WHO-BS-2020.2403 (accessed on 27 October 2023).
16. Infantino, M.; Pieri, M.; Nuccetelli, M.; Grossi, V.; Lari, B.; Tomassetti, F.; Calugi, G.; Pancani, S.; Benucci, M.; Casprini, P. et al. The WHO International Standard for COVID-19 serological tests: Toward harmonization of anti-spike assays. Int. Immunopharmacol.; 2021; 100, 108095. [DOI: https://dx.doi.org/10.1016/j.intimp.2021.108095] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34619529]
17. Giavarina, D.; Carta, M. Improvements and limits of anti SARS-CoV-2 antibodies assays by WHO (NIBSC 20/136) standardization. Diagnosis; 2021; 9, pp. 274-279. [DOI: https://dx.doi.org/10.1515/dx-2021-0126] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34851563]
18. Zhuo, R.; Charlton, C.; Plitt, S.; Thompson, L.A.; Braun, S.; Day, J.; Osiowy, C.; Tipples, G.; Kanji, J.N. Comparison of SARS-CoV-2 spike antibody quantitative titer reporting using the World Health Organization International Standard Units by four commercial assays. J. Clin. Virol.; 2022; 156, 105292. [DOI: https://dx.doi.org/10.1016/j.jcv.2022.105292] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36108404]
19. Bentley, E.M.; Atkinson, E.; Rigsby, P.; Elsley, W.; Bernasconi, V.; Kristiansen, P.; Harvala, H.; Turtle, L.C.; Dobson, S.; Wendel, S. et al. Establishment of the 2nd WHO International Standard for Anti-SARS-CoV-2 Immunoglobulin and Reference Panel for Antibodies to SARS-CoV-2 Variants of Concern; WHO/BS/2022.2427 World Health Organization: Geneva, Switzerland, 2022; Available online: https://www.who.int/publications/m/item/who-bs-2022.2427 (accessed on 27 October 2023).
20. Bentley, E.M.; Rigsby, P.; Elsley, W.; Bernasconi, V.; Kristiansen, P.A.; Wendel, S.; Fachini, R.; Devine, J.; Shongwe, N.; Rose, N.J. et al. Expansion of WHO Reference Panel for Antibodies to SARS-CoV-2 Variants of Concern; WHO/BS/2023.2450 World Health Organization: Geneva, Switzerland, 2023; Available online: https://www.who.int/publications/m/item/WHO-BS-2023-2450 (accessed on 27 October 2023).
21. Dimech, W. The Standardization and Control of Serology and Nucleic Acid Testing for Infectious Diseases. Clin. Microbiol. Rev.; 2021; 34, e0003521. [DOI: https://dx.doi.org/10.1128/CMR.00035-21] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34319148]
22. Fernandes, Q.; Inchakalody, V.P.; Merhi, M.; Mestiri, S.; Taib, N.; El-Ella, D.M.A.; Bedhiafi, T.; Raza, A.; Al-Zaidan, L.; Mona, O. et al. Emerging COVID-19 variants and their impact on SARS-CoV-2 diagnosis, therapeutics and vaccines. Ann. Med.; 2022; 54, pp. 524-540. [DOI: https://dx.doi.org/10.1080/07853890.2022.2031274]
23. Chen, R.E.; Winkler, E.S.; Case, J.B.; Aziati, I.D.; Bricker, T.L.; Joshi, A.; Darling, T.L.; Ying, B.; Errico, J.B.; Shrihari, S. et al. In vivo monoclonal antibody efficacy against SARS-CoV-2 variant strains. Nature; 2021; 596, pp. 103-108. [DOI: https://dx.doi.org/10.1038/s41586-021-03720-y]
24. Patrone, P.N.; Wang, L.; Lin-Gibson, S.; Kearsley, A.J. Unertainty Quantification of Antibody Measurements: Physical Principles and Implications for Standardization. 2023; in review
25. Scheff, S.W. Nonparametric Statistics. Fundamental Statistical Principles for the Neurobiologist; Academic Press: Cambridge, MA, USA, 2016; pp. 157-182.
26. Li, X.; Pang, L.; Yin, Y.; Zhang, Y.; Xu, S.; Xu, D.; Shen, T. Patient Clinical Factors at Admission Affect the Levels of Neutralization Antibodies Six Months after Recovering from COVID-19. Viruses; 2022; 14, 80. [DOI: https://dx.doi.org/10.3390/v14010080]
27. Amanat, F.; White, K.M.; Miorin, L.; Strohmeier, S.; McMahon, M.; Meade, P.; Liu, W.; Albrecht, R.A.; Simon, V.; Martinez-Sobrido, L. et al. An In Vitro Microneutralization Assay for SARS-CoV-2 Serology and Drug Screening. Curr. Protoc. Microbil.; 2020; 58, e108. [DOI: https://dx.doi.org/10.1002/cpmc.108] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32585083]
28. Abe, K.T.; Li, Z.; Samson, R.; Payman, S.-T.; Valcourt, E.J.; Wood, H.; Budylowski, P.; Dupuis, A.P., 2nd; Girardin, R.C.; Rathod, B. et al. A simple protein-based surrogate neutralization assay for SARS-CoV-2. JCI Insight; 2020; 5, e142362. [DOI: https://dx.doi.org/10.1172/jci.insight.142362] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32870820]
29. Haslwanter, D.; Dieterle, M.E.; Wec, A.Z.; O’brien, C.M.; Sakharkar, M.; Florez, C.; Tong, K.; Garrett Rappazzo, C.; Lasso, G.; Vergnolle, O. et al. A Combination of Receptor-Binding Domain and N-Terminal Domain Neutralizing Antibodies Limits the Generation of SARS-CoV-2 Spike Neutralization-Escape Mutants. mBio; 2021; 12, e0247321. [DOI: https://dx.doi.org/10.1128/mBio.02473-21] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34607456]
30. Septisetyani, E.P.; Prasetyaningrum, P.W.; Anam, K.; Santoso, A. SARS-CoV-2 Antibody Neutralization Assay Platforms Based on Epitopes Sources: Live Virus, Pseudovirus, and Recombinant S Glycoprotein RBD. Immune Netw.; 2021; 21, e39. [DOI: https://dx.doi.org/10.4110/in.2021.21.e39]
31. Barrette, R.W.; Urbonas, J.; Silbart, L.K. Quantifying Specific Antibody Concentrations by Enzyme-Linked Immunosorbent Assay Using Slope Correction. Clin. Vaccine Immunol.; 2006; 13, pp. 802-805. [DOI: https://dx.doi.org/10.1128/CVI.00422-05] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/16829619]
32. Wang, L.; Bhardwaj, R.; Mostowski, H.; Patrone, P.N.; Kearsley, A.J.; Watson, J.; Lim, L.; Pichaandi, J.; Ornatsky, O.; Majonis, D. et al. Establishing CD 19 B-cell reference control materials for comparable and quantitative cytometric expression analysis. PLoS ONE; 2021; 16, e0248118.
33. Giovanni, Y.D.V.; Formari, C.; Goldlust, I.; Mills, G.; Koh, S.B.; Bramhall, J.L.; Richards, F.M.; Jodrell, D.I. An automated fitting procedure and software for dose-response curves with multiphasic features. Sci. Rep.; 2015; 5, 14701.
34. Frey, A.; Di Canzio, J.; Zurakowski, D. A statistically defined endpoint titer determination method for immunoassays. J. Immunol. Methods; 1998; 221, pp. 35-41. [DOI: https://dx.doi.org/10.1016/S0022-1759(98)00170-7]
35. Radvak, P.; Kwon, H.J.; Kosikova, M.; Ortega-Rodriguez, U.; Xiang, R.; Phue, J.N.; Shen, R.F.; Rozzelle, J.; Kapoor, N.; Rabara, T. et al. SARS-CoV-2 B.1.1.7 (alpha) and B.1.351 (beta) variants induce pathogenic patterns in K18-hACE2 transgenic mice distinct from early strains. Nat. Commun.; 2021; 12, 6559. [DOI: https://dx.doi.org/10.1038/s41467-021-26803-w]
36. Tian, L.; Elsheikh, R.B.; Patrone, P.N.; Kearsley, A.J.; Gaigalas, A.K.; Inwood, S.; Gibson, S.L.; Esposito, D.; Wang, L. Towards Quantitative and Standardized Serological and Neutralization Assays for COVID-19. Int. J. Mol. Sci.; 2021; 22, 2723. [DOI: https://dx.doi.org/10.3390/ijms22052723]
37. Izac, J.; Kwee, E.; Tian, L.; Gaigalas, A.; Elsheikh, E.; Elliott, J.T.; Wang, L. Development of a Cell-Based SARS-CoV-2 Pseudovirus Neutralization Assay Using Imaging and Flow Cytometry Analysis. Int. J. Mol. Sci.; 2023; 24, 12332. [DOI: https://dx.doi.org/10.3390/ijms241512332]
You have requested "on-the-fly" machine translation of selected content from our databases. This functionality is provided solely for your convenience and is in no way intended to replace human translation. Show full disclaimer
Neither ProQuest nor its licensors make any representations or warranties with respect to the translations. The translations are automatically generated "AS IS" and "AS AVAILABLE" and are not retained in our systems. PROQUEST AND ITS LICENSORS SPECIFICALLY DISCLAIM ANY AND ALL EXPRESS OR IMPLIED WARRANTIES, INCLUDING WITHOUT LIMITATION, ANY WARRANTIES FOR AVAILABILITY, ACCURACY, TIMELINESS, COMPLETENESS, NON-INFRINGMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Your use of the translations is subject to all use restrictions contained in your Electronic Products License Agreement and by using the translation functionality you agree to forgo any and all claims against ProQuest or its licensors for your use of the translation functionality and any output derived there from. Hide full disclaimer
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Abstract
COVID-19 has highlighted challenges in the measurement quality and comparability of serological binding and neutralization assays. Due to many different assay formats and reagents, these measurements are known to be highly variable with large uncertainties. The development of the WHO international standard (WHO IS) and other pool standards have facilitated assay comparability through normalization to a common material but does not provide assay harmonization nor uncertainty quantification. In this paper, we present the results from an interlaboratory study that led to the development of (1) a novel hierarchy of data analyses based on the thermodynamics of antibody binding and (2) a modeling framework that quantifies the probability of neutralization potential for a given binding measurement. Importantly, we introduced a precise, mathematical definition of harmonization that separates the sources of quantitative uncertainties, some of which can be corrected to enable, for the first time, assay comparability. Both the theory and experimental data confirmed that mAbs and WHO IS performed identically as a primary standard for establishing traceability and bridging across different assay platforms. The metrological anchoring of complex serological binding and neuralization assays and fast turn-around production of an mAb reference control can enable the unprecedented comparability and traceability of serological binding assay results for new variants of SARS-CoV-2 and immune responses to other viruses.
You have requested "on-the-fly" machine translation of selected content from our databases. This functionality is provided solely for your convenience and is in no way intended to replace human translation. Show full disclaimer
Neither ProQuest nor its licensors make any representations or warranties with respect to the translations. The translations are automatically generated "AS IS" and "AS AVAILABLE" and are not retained in our systems. PROQUEST AND ITS LICENSORS SPECIFICALLY DISCLAIM ANY AND ALL EXPRESS OR IMPLIED WARRANTIES, INCLUDING WITHOUT LIMITATION, ANY WARRANTIES FOR AVAILABILITY, ACCURACY, TIMELINESS, COMPLETENESS, NON-INFRINGMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Your use of the translations is subject to all use restrictions contained in your Electronic Products License Agreement and by using the translation functionality you agree to forgo any and all claims against ProQuest or its licensors for your use of the translation functionality and any output derived there from. Hide full disclaimer
Details







1 Biosystems and Biomaterials Division, National Institute of Standards and Technology (NIST), Gaithersburg, MD 20899, USA;
2 Applied and Computational Mathematics Division, National Institute of Standards and Technology (NIST), Gaithersburg, MD 20899, USA;
3 Abbott Laboratories, Abbott Park, IL 60064, USA;
4 Laboratory of Pediatric and Respiratory Viral Diseases, Office of Vaccines Research and Review, Center for Biologics Evaluation, Food and Drug Administration (FDA), Silver Spring, MD 20993, USA;
5 Vaccine, Immunity and Cancer Directorate, Frederick National Laboratory for Cancer Research (FNLCR), Frederick, MD 21702, USA;
6 Roche Diagnostics GmbH, 82377 Penzberg, Germany;
7 Centers for Disease Control and Prevention (CDC), Atlanta, GA 30329, USA;