Defining the determinants of protection against SARS-CoV-2 infection and viral control in a dose-down Ad26.CoV2.S vaccine study in nonhuman primates

Abstract

Despite the rapid creation of Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) vaccines, the precise correlates of immunity against severe Coronavirus Disease 2019 (COVID-19) are still unknown. Neutralizing antibodies represent a robust surrogate of protection in early Phase III studies, but vaccines provide protection prior to the evolution of neutralization, vaccines provide protection against variants that evade neutralization, and vaccines continue to provide protection against disease severity in the setting of waning neutralizing titers. Thus, in this study, using an Ad26.CoV2.S dose-down approach in nonhuman primates (NHPs), the role of neutralization, Fc effector function, and T-cell immunity were collectively probed against infection as well as against viral control. While dosing-down minimally impacted neutralizing and binding antibody titers, Fc receptor binding and functional antibody levels were induced in a highly dose-dependent manner. Neutralizing antibody and Fc receptor binding titers, but minimally T cells, were linked to the prevention of transmission. Conversely, Fc receptor binding/function and T cells were linked to antiviral control, with a minimal role for neutralization. These data point to dichotomous roles of neutralization and T-cell function in protection against transmission and disease severity and a continuous role for Fc effector function as a correlate of immunity key to halting and controlling SARS-CoV-2 and emerging variants.

Introduction

Since December 2019, Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) has spread globally, linked to greater than 200 million confirmed cases and over 4 million deaths [1]. Several vaccines have been granted emergency use authorization globally [2–7], affording variable levels of efficacy, particularly against emerging viral variants of concern (VOCs) [8–13]. While neutralizing antibodies represented an early robust surrogate of immunity prior to the evolution of VOCs, vaccine breakthroughs are rapidly emerging globally in the absence of concurrent levels of disease [10–12]. Moreover, vaccines afford protection against severe disease and death prior to the evolution of neutralization [14–16] as well as in the setting of waning immunity [17], suggesting that alternate immunological mechanisms may exist, which provide protection against disease, including antibody-mediated Fc effector functions [18–21] and T cells [22–26], which appear to be less affected by VOCs [9,27]. However, how neutralization, Fc effector function, and T cells collectively contribute to durable and effective protection against SARS-CoV-2 and emerging variants remains incompletely understood.

To begin to dissect the protective role of each immune mechanism, we exploited a vaccine dose-down strategy, aimed at inducing progressively lower SARS-CoV-2 immunity, to capture the immunological breaking point of immune protection upon challenge. Neutralizing antibodies, T-cell responses, and Fc effector functions [28] were captured at peak immunogenicity across all dosing groups prior to challenge. While antibody titers and neutralization were minimally affected by dosing, Fc profiles were highly dose dependent. Similarly, recognition of VOCs was also highly dose dependent. Protection against infection was robustly linked to neutralizing antibody titers and Fc effector mechanisms. Conversely, viral control following infection was associated with Fc effector functions and T cells. Thus, Fc effector function clearly synergizes with both neutralization and T cells, to block transmission or control viremia, respectively. These data point to disparate functions of neutralizing antibodies and T cells in antiviral immunity, but a consistent role of Fc effector functions across both transmission and protection against disease.

Results

Dose-dependent upper respiratory SARS-CoV-2 replication following challenge

To more deeply understand the correlates of protection, 5 independent groups of rhesus macaques (n = 5) were immunized with different doses of the Ad26.CoV2.S vaccine (1 × 1011, 5 × 1010, 1.125 × 1010, and 2 × 109 viral particles [VPs]) or a sham control (Fig 1A). On day 42, all macaques were challenged with 1.0 × 105 TCID50 (1.2 × 108 RNA copies) of SARS-CoV-2, and viral replication was assessed by analyzing subgenomic RNA (sgRNA) levels in the upper and lower respiratory tracts by way of nasopharyngeal swabs (NSs) and bronchoalveolar lavage (BAL), respectively, on days 1, 2, 4, 7, and 10 postchallenge. As previously published in [29], vaccine doses as low as 2 × 109 VPs provided robust protection in the lower respiratory tract, whereas protection in the upper respiratory tract showed a negative dose dependency (Fig 1B), in which a higher dose (1.125 × 1010 VPs) was required for protection. Additionally, as previously reported [30], higher doses generated consistently higher binding antibody titers and neutralizing titers, in addition to stimulating increased T-cell cytokine production above an observed threshold (Fig 1B) in response to both SARS-CoV-2 spike and receptor-binding domain (RBD). This illustrates the complete protection provided by high doses of the vaccine that wanes as a function of dose and highlights the need to deeply mine for correlates of protection in the upper respiratory tract.

[Figure omitted. See PDF.]

Fig 1. AD.COV2.S dose-down immunization scheme and graphical depiction of the immune response.

(A) Rhesus macaques were immunized with 5 different doses of Ad26.COV2.S, profiled for their immune response, and challenged with SARS-CoV-2. The timeline shows the timing of immunization (week 0), serum collection (week 2, 4, and 6), SARS-CoV-2 challenge (day 43), and viral titer sampling of the cohort (day 0, 1, 2, 4, 7, and 10 postinfection). The legend states the dose of Ad26.COV2.S given to each group of NHP. (B) From left to right, summary of the observed trends of viral sgRNA in the upper respiratory tract, sgRNA in the lower lower respiratory tract, binding antibody titer, neutralizing antibody titer, and T-cell IFNγ secretion as they relate to dose. Fig 1B was created with BioRender.com. BAL, bronchoalveolar lavage; IFNγ, interferon gamma; NHP, nonhuman primate; SARS-CoV-2, Severe Acute Respiratory Syndrome Coronavirus 2; sgRNA, subgenomic RNA; VP, viral particle.

https://doi.org/10.1371/journal.pbio.3001609.g001

Dosing effects on cellular and humoral immune responses

To further dissect the qualitative effects of vaccine dosing on the functional humoral immune response, systems serology was applied to the aforementioned BAL and NS samples, collected on day 43 prior to SARS-CoV-2 challenge. Similar to the neutralizing antibody titers, the 2 highest doses (I and II) induced similar levels of anti–SARS-CoV-2 IgG1, IgG2, IgG3, IgG4, and IgA antibody titers, with slightly lower levels at dose III and significantly lower levels at dose IV (Fig 2A–2F). Conversely, while the 2 highest doses also generated equivalent levels of Fcγ receptor (FcR) binding titers (Fig 2G and 2H), more significant loss of FcR binding was observed in dose group IV, with a broad spread of binding to the FcRs. These data point to a more significant impact of dosing on shaping the overall magnitude of the FcR binding compared to driving titers, neutralizing antibody responses or T cells.

[Figure omitted. See PDF.]

Fig 2. System serology profiling of anti-Spike antibodies from Ad26.COV2.S vaccinated NHPs.

(A) The humoral response against SARS-CoV-2 Spike was profiled using system serology. (B–G) The titer of IgG1 (B), IgG2 (C), IgG3 (D), IgG4 (E), IgM (F), and IgA (G) antibodies against SARS-CoV-2 were profiled using Luminex. (H, I) The titer of anti–SARS-COV-2 antibody binding to FcγRIIA-1 (H) and FcγRIII (I) were profiled using Luminex. (J–M) The graphs represent the ability of the humoral response to stimulate ADCP (J), neutrophil phagocytosis (ADNP) (K), complement deposition (ADCD) (L), or NK cell activation (NKdegran) (M) when stimulated with antigen-coated beads or plates. Bars represent the mean, and error bars represent the standard of deviation. This figure can be generated from the data found in data/figure2_data.csv of https://github.com/dzhu8/Ad26-Dose-Down. ADCD, antibody-dependent complement deposition; ADCP, antibody-dependent cellular phagocytosis; ADNP, antibody-dependent neutrophil phagocytosis; MFI, median fluorescent intensity; NHP, nonhuman primate; NK, natural killer; SARS-CoV-2, Severe Acute Respiratory Syndrome Coronavirus 2; VP, viral particle.

https://doi.org/10.1371/journal.pbio.3001609.g002

To further explore the impact of dosing on Fc effector function, we finally profiled the vaccine-induced Spike-specific response. Similar to FcR binding profiles, similar antibody effector functions were observed in the top 2 dose groups (Fig 2I–2L), with a significant decline in antibody-dependent cellular phagocytosis (ADCP) and antibody-dependent neutrophil phagocytosis (ADNP) in group III and significant decline in ADCP, ADNP, antibody-dependent complement deposition (ADCD), and antibody-mediated NK cell activating (ADNKA) activity in group IV. Together, these findings suggest that antibody FcR binding and Fc effector function are more sensitive to changes in vaccine dosing, compared to traditional metrics of immunogenicity (neutralizing antibodies, T cells, and titers) and may provide additional insights into correlates of immunity.

Distinct humoral profiles across vaccine dose regimens revealed by multivariate analysis

Given the different univariate variation across multiple SARS-CoV-2–specific immune responses, we next aimed to define the impact of dose on overall vaccine-induced immune profiles. To focus on the effect of vaccine-induced immunity, we removed all Sham samples. Striking differences were noted in the overall Spike-specific humoral immune profiles across the 4 dose groups (Fig 3A), marked by the most robust, fully immunoglobin class-switched (no IgM) titers in the animals that received the highest vaccine dose, followed by robust titers, FcR binding, T-cell responses, and Fc effector function in group II immunized animals. Conversely, decreased antibody titers and Fc effector functions were noted in group III immunized animals, with a very significant loss of FcR binding and limited T-cell immune responses. Finally, the lowest antibody titers, Fc effector functions, T-cell responses, and negligible FcR binding were noted in group IV immunized animals. Moreover, additional analysis of the overall architecture of the humoral immune response using Pearson correlations between all humoral variables highlighted the relative conservation of the humoral immune architecture across the vaccine groups (S1 Fig), with the exception of ADCP, which was least dose dependent.

[Figure omitted. See PDF.]

Fig 3. Multivariate analysis of the humoral response among each vaccine regimen.

Further analysis was performed to identify which humoral features defined the response in each dose. (A) Polar plots showing the mean percentile of each of the anti–SARS-CoV-2 antibody titers, FcR binding, and effector function activation features for NHPs, grouped by the approximate number of VPs received in the vaccine dose. Each wedge represents an individual feature, with colors representing the type of feature: orange: effector function; lavender: antibody titer; violet: FcR binding; indigo: neutralizing titer; and pink: T-cell cytokine secretion. (B) The heatmap shows the humoral response, antibody isotype, and FcR binding titers, to multiple SARS-CoV-2 proteins (Spike, S1, S2, and RBD), and NTD, as well as the functional responses to Spike. The squares represents the z-scored value for each immune feature (x-axis) for each NHP (y-axis). Z-scored was calculated across each individual column. (C) A latent variable scores plot resulting from PCA shows the degree of separation between the 4 vaccine doses that was achieved by analyzing anti–SARS-CoV-2 (anti-S1, S2, NTD, RBD, and Spike) features. Each point is an individual NHP, color coded by vaccine dose group (purple: 1011 VPs, orange: 5 × 1010 VPs, blue: 1.125 × 1010 VPs, and pink: 2 × 109 VPs). Ellipses illustrate the 95% confidence interval for dose groups. (D) Loadings plot depicting the most variant features along the first principal component (corresponding to the x-axis for the scores plot in C). This figure can be generated using the data and code deposited in the “src” folder of https://github.com/dzhu8/Ad26-Dose-Down. Instructions to do so can be found in the README file. FcR, Fcγ receptor; NHP, nonhuman primate; NTD, N-terminal domain; PCA, principal component analysis; RBD, receptor-binding domain; SARS-CoV-2, Severe Acute Respiratory Syndrome Coronavirus 2; VP, viral particle.

https://doi.org/10.1371/journal.pbio.3001609.g003

To further define the impact of vaccine dosing on the epitope-specific response across the Spike protein, we also examined the vaccine-induced humoral immune response to the RBD, S1, S2 and N-terminal domain (NTD) (Fig 3B). While heterogenous responses were seen for animals in the same dose group following vaccination, generally, the highest and broadest immune responses were noted in group 1 (1 × 1011 VP) group, with a dose-dependent decrease in titer, FcR binding, and Fc-induced immune function across the other vaccine dose groups for all regions of the SARS-CoV-2 Spike. IgM titers were the exception, with IgM responses anticorrelated with other subclasses, as a marker of mature class switching to IgA and IgG [31]. To finally define whether antibody profiles could distinguish the dose groups, a principal component analysis (PCA) was generated across the 4 vaccine groups (Fig 3C). The 2 top dose groups clustered together, whereas the 2 bottom groups clustered to the left of the PCA plot. The majority of the variance (60%) was explained by the first principal component, and a comparison of the loadings (the coefficients of the linear combination of variables used to construct principal components) along this axis (Fig 3D) revealed enrichment of S2 and Spike FcR binding features as the predominantly enhanced features in animals that received the highest vaccine doses. These data revealed the significant enrichment of FcR binding antibodies among animals that received the higher vaccine doses.

Neutralizing antibody titers and FcR binding are enriched in protected animals

To begin to define the specific immunologic markers associated with protection against viral challenge, animals were divided into 3 groups, consisting of nonhuman primates (NHPs) exhibiting total protection (no detectable viral replication), partial protection (no detectable viral replication in BAL), and no protection (viral replication in BAL and nasal swab). Polar plots revealed differences in the overall vaccine-induced immune profiles across the groups, with the strongest responses, across immunologic readouts, in animals with total protection (Fig 4A). Intermediate responses were observed in the partially protected group, and lower responses, marked by near complete loss of FcR binding in the animals that were not protected. Unsupervised multivariate analysis (PCA, as above) highlighted a clear separation in the immune profiles of animals that exhibited complete protection against SARS-CoV-2 and animals exhibiting viral load in at least one compartment (Fig 4B). To precisely define the features that tracked with protection, fold changes were computed for each immunologic marker and visualized in a volcano plot (Fig 4C), across animal groups. The most significant fold change, depicted on the y-axis, was neutralizing antibody titers (as assessed using the nonparametric Mann–Whitney U test), with all humoral features save for ACDP, NK-secreted CD107a, and IgM titer also found to be significant after multiple hypothesis correction using the Holm–Bonferroni correction. Data suggest a more mature humoral immune response, consistent with a loss of IgM observed in the higher vaccine dosed animals (Fig 3A), and a significant enrichment of the Fc functional response that is corroborated by combining all titer, FcR binding, and functional features and comparing between protected and not protected animals (S2 Fig). Conversely, FcR binding features underwent the largest average fold change, depicted on the x-axis of the volcano plot, marked by >2-fold change and broad binding across nearly all, and especially, the phagocytic FCGR2A-3 receptor in protected animals.

[Figure omitted. See PDF.]

Fig 4. Multivariate analysis of the immune response for varying degrees of protection against SARS-CoV-2 challenge identifies key determinants of immunity.

(A) Polar plots showing the mean percentile of each of the anti–SARS-CoV-2 antibody titers, FcR binding, and effector function activation features for NHPs, grouped by the degree of protection against viral challenge in the upper and lower respiratory tract; groups comprised NHPs protected in neither, one, or both compartments. Each wedge represents an individual feature, with colors representing the type of feature: orange: effector function; light purple: antibody titer; dark purple: FcR binding; blue: neutralizing antibody titer; and pink: T-cell IFNγ secretion. (B) A latent variable scores plot resulting from PCA shows the degree of separation between the 4 vaccine doses that was achieved by analyzing anti–SARS-CoV-2 spike-component (anti-S1, S2, NTD, and RBD) features. Each point is an individual NHP, color coded by protection from infection (yellow: protected in BAL and nasal swab, purple: protected in BAL only or no protection in BAL or nasal swab). Ellipses illustrate the 95% confidence interval for dose groups. (C) A volcano plot showing the average fold change for each of the anti–SARS-CoV-2 spike features between completely protected NHPs and NHPs lacking complete protection, with the significance of the difference between these groups on the y-axis, quantified using the negative log p-value and measured using 2-sided Wilcoxon rank-sum tests. Each point represents an individual feature, color coded by the degree of statistical significance on a continuous colorbar. The star characters represent statistical significance following Holm–Bonferroni correction. Note that ADNP is not explicitly labeled due to overlap with IgG2, but its fold change is also significant. (D) A latent variable scores plot resulting from PLS-DA, with the variable regressed on being the categorical assignment of “protection” or “not protected/only protected in the lower respiratory tract,” shows the degree of separation between NHPs that were completely protected and NHPs lacking complete protection that was achieved with anti–SARS-CoV-2 anti-Spike-component features. Each point is an individual NHP color coded by protection from infection (yellow: protected in BAL and nasal swab, purple: protected in BAL only or no protection in BAL or nasal swab). (E) Loadings plot depicting the most variant features along the first principal component (corresponding to the x-axis for the scores plot in D). This figure can be generated using the data and code deposited in the “src” folder of https://github.com/dzhu8/Ad26-Dose-Down. Instructions to do so can be found in the README file. ADNP, antibody-dependent neutrophil phagocytosis; BAL, bronchoalveolar lavage; FcR, Fcγ receptor; NHP, nonhuman primate; NTD, N-terminal domain; PCA, principal component analysis; PLS-DA, partial least squares discriminant analysis; RBD, receptor-binding domain; SARS-CoV-2, Severe Acute Respiratory Syndrome Coronavirus 2.

https://doi.org/10.1371/journal.pbio.3001609.g004

Given the highly correlated nature of the vaccine-induced immune response, we next aimed to identify a minimal combination of features that best discriminated peak immune profiles in animals that were protected or infected after challenge. Least absolute shrinkage and selection operator (LASSO) regularization was used to reduce the degree of multicollinearity and identify a minimal set of immune features that tracked with protective immunity. As few as 7 features were sufficient to split the protected animals to those that became infected following challenge, as visualized by partial least squares discriminant analysis (PLS-DA) (Fig 4D and 4E). The protective features did not include neutralizing antibody levels, but instead included class switched IgG levels (IgG2, 4, and 3), ADCD, FcγR2A-3 binding, and ADCP activity, likely related to the presence of high neutralizing antibody levels across all animals. IgM was the sole feature associated with susceptibility, potentially marking an incompletely mature humoral immune response.

The Fc effector profile is shaped by both RBD binding and non–RBD binding antibodies

To further dissect the relationship between neutralizing and Fc receptor binding antibodies, dose effects were examined across epitope-specific vaccine-induced antibodies. Given that the majority of neutralizing antibodies target the RBD [32], we probed the overall changes in RBD, NTD, and S2-specific titers and FcR binding across the dose-down groups (Fig 5A). A clear titer and FcR binding dose effect was observed across all 3 epitope-specific antibody classes, across both dominant neutralizing and nonneutralizing antibody targets. Differences were noted in some epitope specific populations, with a dampened NTD-specific FcγRIIa-3 response compared to NTD-specific binding to other Fc receptors, pointing to additional differences in epitope-specific evolution. Multivariate analysis of these epitope specific antibody responses demonstrated an enrichment of both RBD-targeting and S2-targeting features in protected animals with lower viral loads, supporting a protective role for both RBD-specific and non–RBD-specific antibody Fc effector functions (Fig 5B).

[Figure omitted. See PDF.]

Fig 5. Comparison of FcγR binding profiles across SARS-CoV-2 spike domains.

(A) The titer of anti–SARS-CoV-2 RBD, S2 spike subunit, and NTD spike subunit antibody binding to FcγRIIA-1, FcγRIIA-2, FcγRIIA-3, FcγRIIA-4, and FcγRIII were profiled using Luminex. (B) Scores plot (upper plot) and loadings along the first latent variable (lower plot) for a PLS-R model, regressing on the AUC of viral load as measured by BAL over a 10-day time course. Each point represents an NHP, either completely protected (orange) or lacking complete protection (purple); ellipses correspond to the 95% confidence intervals for each group. Fig 5A data can be found in the file “data/figure5_data.csv.” Fig 5B can be generated using the data and code deposited in the “src” folder of https://github.com/dzhu8/Ad26-Dose-Down. Instructions to do so can be found in the README file. AUC, area under the curve; BAL, bronchoalveolar lavage; NHP, nonhuman primate; NTD, N-terminal domain; PLS-R, partial least square regression; RBD, receptor-binding domain; SARS-CoV-2, Severe Acute Respiratory Syndrome Coronavirus 2.

https://doi.org/10.1371/journal.pbio.3001609.g005

Fc effector functions collaborate with neutralizing antibodies and T-cell immunity differently to achieve protection

Both antibodies and T cells have been linked to resolution of infection [18,22,32]. However, the precise contribution of neutralization, Fc effector function, and T-cell immunity remains unclear. Thus, to explore the contribution of each “arm” of the immune system to protection, we created composite scores of our systems serology data using the partial least squares path modeling (PLS-PM) [33] framework to generate latent variable measurements (Fig 6A). Antibody features were divided into 3 groups including (1) antibody-mediated effector functional measurements (ADCP, ADNP, ADCC, NK-MIP1b+%, NK-CD107a+%, and NK-interferon gamma (IFNγ)+%); (2) binding antibody titers (IgG1, IgG2, IgG3, IgG4, IgA, and IgM); and (3) FcR binding measurements (FcγRA2-1, FcγRA2-2, FcγRA2-3, FcγRA2-4, and FcγRA3). The enrichment of these composite metrics as well as neutralizing antibody titers and T-cell immunity were compared across animals that remained uninfected or became infected after challenge. All 3 composite metrics and neutralizing antibody titers were significantly highly enriched among protected animals (Fig 6B). Thus, to define the degree of enrichment, a variable importance (VIP) coefficient was generated, comparing each variable to outcome as assessed by viral load. A clear hierarchy emerged across the 5 variables, with antibody titers representing the strongest correlate of protection, followed by antibody functions, FcR binding, neutralization, and T cells (Fig 6C). However, to gain a clearer sense of the precise role of each of the composite variables in providing protection from infection (blocking infection in both the upper and lower respiratory tract) or in controlling viral replication (viral levels after infection), VIP coefficients were generated for each outcome. The same hierarchy of features (Fig 6C) were associated with protection from infection (Fig 6D), with protection assessed using a binary variable indicating whether breakthrough was observed or not. Conversely, the arrangement of the 5 metrics changed with viral breakthrough (Fig 6E), when examining the association of each parameter with the level of viral replication only in the breakthrough animals. Specifically, antibody titers remained the top feature, followed by antibody functions, T-cell activity, FcR binding, and neutralizing antibodies were least critical for predicting viral control. These data highlight the importance of neutralization in the blockade of infection, T cells in the control of viremia, and antibody Fc effector functions as a critical collaborator and contributor to both prevention and control of viral infection.

[Figure omitted. See PDF.]

Fig 6. Combined variable analysis identifies key combinatorial correlates of protection.

(A) Diagram depicting the process for computing latent variable coefficient scores with PLS-PM. Five PLS-PM models were computed, and the coefficients compared. Figure created with BioRender.com. (B) Violin plots showing the distribution of latent scores for the functional, antibody titer, and FcR binding features, as determined by PLS-PM, as well as for neutralizing antibody titer and T-cell IFNγ secretion. The white dot represents the median, the thicker bar the IQR, and the thinner bar 1.5 times the IQR in either direction. Thickness of the violin represents the density of a particular region. (C) Barplots showing coefficients for the PLS-PM analyses (D, E) Barplots showing coefficients for 2 PLS-PM analyses: with a binary protection variable (“fully protected” or “partially protected or not protected at all”) (D) and with the protection variable being a combination of the AUC of cumulative viral titers measured at day 10 by both nasal swab and BAL (E). This figure can be generated using the data and code deposited in the “src” folder of https://github.com/dzhu8/Ad26-Dose-Down. Instructions to do so can be found in the README file. AUC, area under the curve; BAL, bronchoalveolar lavage; IFNγ, interferon gamma; IQR, interquartile range; PLS-PM, partial least squares path modeling.

https://doi.org/10.1371/journal.pbio.3001609.g006

Antibody response to the SARS-CoV-2 mutant VOCs

As the coronavirus pandemic continues, new VOCs that are more infectious or escape the immune response are continuing to arise [34,35]. Current vaccines have been shown to be efficacious against the wild-type (WT) SARS-COV-2; however, the robustness of the variants to neutralization by either monoclonal or vaccine-induced binding antibodies [36–38] has prompted concerns. To address the impact of dose on emerging VOCs, we profiled the humoral response to multiple VOC antigens: D614G [39], N501YΔ69–70 (representing key mutations in Alpha/B.1.1.7 [40], and E484K (representing the key mutation in Beta/B.1.351 [41] and Gamma/P.1 [42], as well as being found in other variants of interest, such as Iota/B.1.526 [43]). We observed relatively stable antibody binding across the variants in the highest doses with greater reductions to N501Yd69-70 or E484K in the lowest dose group, while FcR binding to the variants had a more profound loss across all doses (Fig 7A). This loss was linked to more profound loss of ADCP against the N501Yd69-70 and E484K variants, pointing to a potential weakness in the protection against VOCs in individuals with low vaccine-induced immunity. Composite analysis of the response to the VOCs highlighted robust titers and functionality in the top 2 dosed groups, but some loss of FcR binding and function in the lower dosed animals (Fig 7B). Comparisons of composite scores composed of the variant features showed significant differences between animals that were protected or infected after WT challenge, as well as high correlations between variant features and analogous WT features (S3A–S3D Fig). These data point to reduced Fc effector function against particular VOCs, linked to declining antibody titers, which may account for evidence of global breakthroughs, but in addition detectable levels of nearly all antibody effector functions across key mutations persisted that may contribute to global protection against severe disease in collaboration with T-cell immunity.

[Figure omitted. See PDF.]

Fig 7. Humoral response to VOCs.

(A) The humoral response of vaccinated NHPs was profiled against several SARS-CoV-2 spike proteins (Spike: D614G, N501YΔ69–70, and E484K). The antibody titers and FcR binding titers were assayed by Luminex and the functional responses were analyzed using their respective assays. The bars represent the median response and error bars represent standard deviation. (B) The dot plots show the correlation of the IgG1 titer, FcRgIIA-I binding titer, ADCP, or complement deposition between D614G SARS-CoV-2 and the variants N501YΔ69–70 and E484K. D614G is plotted as an example of perfect correlation. A 2-way ANOVA with Tukey correction for multiple comparisons was used to compare antibody levels between groups. Only significant comparisons are shown. ns, not significant, *p ≤ 0.05, **p ≤ 0.01, ***p ≤ 0.001, ****p ≤ 0.0001. This figure can be generated from the data found in data/figure7_data.csv of https://github.com/dzhu8/Ad26-Dose-Down. ADCP, antibody-dependent cellular phagocytosis; FcR, Fcγ receptor; MFI, median fluorescent intensity; NHP, nonhuman primate; SARS-CoV-2, Severe Acute Respiratory Syndrome Coronavirus 2; VOC, variant of concern.

https://doi.org/10.1371/journal.pbio.3001609.g007

Discussion

Despite the development of several efficacious vaccines against Coronavirus Disease 2019 (COVID-19) that provide protection against severe disease and death [5,7,15,44–46], understanding of the correlates of immunity have raised concerns about durability of protection against the multitude of emerging highly contagious VOCs. While neutralizing antibodies have emerged as a logical correlate of immunity to SARS-CoV-2, emerging signals of protection against VOCs-that evade neutralizing antibodies as well as in the setting of waning immunity point to the importance of alternate correlates of protection against SARS-CoV-2. The identification of the key immunologic mechanisms of protection could provide the critical insights to guide vaccine design and boosting regimens to provide durable protection to end this global pandemic.

While neutralizing antibody levels were identified as a strong correlate of protection in large scale vaccine efficacy trials, antibody binding titers were more robustly correlated with protection against severe disease across vaccine platforms [47,48]. Whether binding antibodies, with alternate nonneutralizing functions, represent direct mechanistic correlates of immunity or represent markers of a robust T-cell response remains unclear. However, both antibody-mediated Fc effector function and T-cell immunity have been proposed as markers of protective immunity [49–52]. Thus, to begin to define the role of neutralization, T-cell immunity, and alternate antibody mechanisms of action, we exploited a vaccine dose-down study in NHP to begin to tease apart the role of individuals arms of the immune response in protection against infection or disease. We found that in the context of an Ad26 vaccine, while antibody titers and neutralizing antibodies were robustly induced across most vaccine doses, T-cell responses were more sensitive to vaccine dose and were only detectable in animals immunized with doses equal to or greater than current clinically tested doses (1 × 1011 or 5 × 1010 VPs). Conversely, antibody FcR binding and effector function were highly sensitive to vaccine dose, demonstrating a strong dose dependence, highlighting an unexpected disconnect in antibody titers/neutralization and T-cell and antibody functional induction. Whether these findings are generalizable to other vaccine formulations remains to be determined.

Breakthrough infections in the upper respiratory tract of animals immunized with the lower vaccine doses provided an opportunity to define immune correlates of protection. Because of the large number of variables captured using systems serology, the probability of an association with an Fc signal was larger. Thus, systems serology data were collapsed into 3 latent variables that captured antibody subclass/isotype diversification, FcR binding breadth, and Fc effector polyfunctionality. While this simplification of the data may obscure the precise correlate of immunity, this approach provides a more conservative means to compare additional antibody metrics to neutralization and T-cell immunity in the context of protective immunity. While T cells, neutralization, and Fc effector responses were all enriched in protected animals, the hierarchy of correlates of protection differed across outcomes. Specifically, antibody titers, FcR binding, antibody effector functions collaborated with neutralization in the context of complete protection from infection (blockade of infection in the upper and lower respiratory tract). Conversely, antibody titers, FcR binding and antibody effector functions collaborated with T-cell immunity in the context of viral control in animals that experienced a breakthrough infection. These data support the notion that neutralization and Fc functions are key to blocking transmission, but that T cells collaborate with antibody functions to drive viral control and clearance.

The significant enrichment of several Fc effector functions in protected animals raises additional questions as to the particular Fc mechanism that may be most critical for protection from infection and/or disease. Among the Fc functions, NK cell activation was enriched in protected animals. However, early studies examining humoral correlates of immunity in natural COVID-19 infection pointed to a significant role for opsonophagocytic, but not NK cell activating antibodies, as correlates of resolution of severe disease [24]. Given the low abundance of NK cells in the upper respiratory tract prior to infection, it is unlikely that these cells contribute dominantly to the control/clearance of infection compared to the more highly abundant resident phagocytes and circulating granulocytes [53,54]. Moreover, because viral entry and release from target cells occurs via endocytosis and assembly on the Golgi (not the cell membrane), respectively, little to no Spike is likely expressed on the surface of infected cells [55]. Thus, in the absence of antibody-mediated recognition of Spike on infected cells, the role for NK cells in eliminating infected cells remains unclear. Conversely, Fc effector mediated capture and clearance of VPs may create a critical bottleneck, rapidly eliminating incoming or newly released viruses, providing a window for T cells to activate, proliferate, recognize, and eliminate infected cells, leading to the ultimate resolution of the infection. Thus, antibodies and T cells likely collaborate intimately to resolve SARS-CoV-2 infection, warranting further dissection to fully define the mechanistic correlates of immunity against SARS-CoV-2.

These data provide critical insights in the context of the current pandemic, where emerging VOCs have the capability of breaking through vaccine-mediated protection, but vaccine-induced immunity still elicits a response against several variants and provides some level of protection against severe disease and death [17,56–58]. Whether T cells or antibody effector function are more critical remains to be determined, but the analysis here points to a collaboration between vaccine-induced antibody Fc mechanisms and T cells, potentially via rapid antibody-mediated opsonophagocytic clearance and anamnestic boosting of T-cell immune responses that collectively lead to robust control and clearance of infection. While postimmunization samples could be linked to outcomes in this study, postchallenge humoral profiles will be important to compare prechallenge and postchallenge correlates of protection, and additionally to provide potential mechanistic links between the identified correlates of protection and containment of SARS-CoV-2.

The continued emergence of several SARS-CoV-2 variants with enhanced infectivity and immune evasive capacity have further complicated the worldwide vaccination effort [39,59–62]. The duration of the pandemic has been marked by decreased vaccine efficacy against these variants. Early trials for Novavax NVX-CoV2373 and Pfizer-BioNTech BNT162b2 saw a decrease from >90% efficacy against the WT to 50% to 60% against the Beta variant [15,63–65]. An exception was seen with the Delta variant, for which vaccines continue to confer >80% protection against severe disease and death [66]. Understanding vaccine-mediated correlates of protection and the contribution of Fc effector functions and T cells is critical in the context of estimating vaccine efficacy to new and emerging VOCs. Along these lines, despite the extraordinary global surge of the highly infectious Omicron variant, which significantly evades neutralizing antibodies [67–70], striking increases in infections were observed in the absence of a significant rise in severe disease and death. These data argue that while transmission occurred due to evasion of neutralizing antibody responses, additional vaccine-induced mechanisms likely continue to contribute to the rapid control and clearance of the virus. Moreover, given the persistence of robust SARS-CoV-2 specific T cell [71–76] and Fc effector functions [77] across VOCs including Omicron, these arms of the immune response likely contribute to remains intact against the omicron strain of SARS-CoV-2. Thus, in line with the data explored here, which show a little drop in VOC binding, our results point to the potential mechanistic complementary importance of a collaboration between T cells and Fc-mediated effector functions, each playing a role in SARS-CoV-2 transmission and reduced disease. However, because our experiments used WA1/2020 antigens, it remains to be determined whether these observations will generalizable to other SARS-CoV-2 strains and vaccine inserts. Further experiments will need to be conducted to determine which Fc effector mechanism persists during Omicron infection.

The identification of correlates, and even surrogates of immunity, against the emerging VOCs is of the greatest urgency. While early studies against the original SARS-CoV-2 variant pointed to antibody titers and neutralization as key mechanisms of protection across vaccine platforms across the original SARS-CoV-2 variant, emerging data point to a central role of Fc effector functions and T-cell immunity as critical mechanistic players in antiviral control required for the prevention of severe disease and death. This study further demonstrates the key role of antibody effector functions and neutralization as mechanistic players in limiting transmission, but an alternate cooperation between Fc effector functions and T cells in limiting viral replication, a proxy of disease severity [78]. Thus, given the simplicity of measuring FcR binding antibodies, or simple opsonophagocytic functions, the latter utilized broadly for the licensure of bacterial vaccines, it is plausible that next generation assays aimed at measuring Fc biology across both existing and emerging VOCs could provide instrumental insights to guide next generation vaccine development or boosting.

Materials and methods

Cell line, viruses, and receptor

THP-1 cells (ATCC TIB-202) were maintained in Roswell Park Memorial Institute (RPMI) medium, supplemented with 10% FBS, 1% glutamine, 1% P/S, 1% HEPES, and 50 μM β-ME.

Animal study design

The study design has been described [29]. In brief, 30 outbred Indian-origin adult male (10) and female (20) rhesus macaques (Macaca mulatta) were randomly allocated to groups. Animals received a single immunization of 1 × 1011, 5 × 1010, 1.125 × 1010, or 2 × 109 vps Ad26.COV2.S (Janssen; n = 5/group) or sham (n = 10) by the intramuscular route without adjuvant at week 0. All animals were challenged with 1.0 × 105 TCID SARS-CoV-2, which was derived from USA-WA1/2020 (NR-52281; BEI Resources - Atlanta, Georgia, USA), by the intratracheal (IT) route at week 6. All serum samples were collected at week 6 postimmunization, before the challenge. All animals were housed at Bioqual (Rockville, Maryland, USA). Animal studies were conducted in compliance with all relevant local, state, and federal regulations and were approved by the Bioqual Institutional Animal Care and Use Committee (IACUC approval number 20-015P).

Subgenomic mRNA assay

The assay has been described previously [29]. In brief, RNA was isolated from macaque BAL fluid and nasal swabs using the IndiSpin QIAcuba HT Pathogen Kit (Indical Bioscience - Leipzig, Germany). RNA was reverse transcribed to cDNA using Superscript VILO (Invitrogen - Carlsbad, California, USA) and stored at 4°C until RT-PCR assays were performed. A Taqman custom gene expression assay (Thermo Fisher Scientific - Waltham, Massachusetts, USA) was designed using the sequences target the E gene sgRNA [79]. The sequences for the custom assay were as follows, forward primer: sgLead-CoV2.FWD: CGATCTCTTGTAGATCTGTTCTC, E_Sarbeco_R: ATATTGCAGCAGTACGCACACA, E_Sarbeco_P1_(probe): VIC-ACACTAGCCATCCTTACTGCGCTTCG-MGB. Reactions were performed on QuantStudio 6 and Flex Real-Time PCR systems (Applied Biosystems - Bedford, Massachusetts, USA). Analysis was performed on the QuantStudio Real-Time PCR Software (Life Technologies - Singapore). Standard curves were used to calculate sgRNA copies per mL or per swab.

Anti-spike and RBD ELISA

The assay has been described previously [29]. In brief, 96-well plates were coated with 1ug/mL SARS-CoV-2 spike (S) or RBD protein in 1× DPBS and incubated at 4°C overnight. After incubation, plates were washed once with was buffer (0.05% Tween 20 in 1x DPBS) and blocked with 350-uL Casein block/well for 2 to 3 hours at room temperature. After incubation, block solution was discarded and plates were blotted dry. Serial dilutions of heat-inactivated serum diluted in casein block were added to wells and plates were incubated for 1 hour at room temperature, prior to 3 washes and a 1-hour incubation with a 1:1,000 dilution of anti-macaque IgG HRP (NIH NHP Reagent Program) at room temperature in the dark. Plates were washed 3 times, and 100 uL of SeraCare KPL TMB Stop solution per well. The absorbance at 450 nm was recorded using a VersaMax or Omega microplate reader.

Pseudovirus neutralizing antibody assay

The assay has been described previously [29]. In brief, the packaging plasmid psPAX2 (AIDS Resource and Reagent Program), luciferase reporter plasmid pLenti-CMV Puro-Luc (Addgene), and spike protein expressing pcDNA3.1-SARS CoV-2 SDCT of variants were cotransfected into HEK293T cells by lipofectamine 2000 (Thermo Fisher Scientific). Pseudoviruses of SARS-CoV-2 variants were generated by using Wuhan prototype strain (Wuhan/WIV04/2019, GISAID accession ID: EPI_ISL_402124), D614G mutation, B.1.1.7 variant (GISAID accession ID: EPI_ISL_601443), or B.1.351 variant (GISAID accession ID: EPI_ISL_712096). The supernatants containing the pseudotype viruses were collected 48 hours posttransfection, which were purified by centrifugation and filtration with 0.45-mm filter. To determine the neutralization activity of the plasma or serum samples from participants, HEK293ThACE2 cells were seeded in 96-well tissue culture plates at a density of 1.75 × 104 cells/well overnight. Moreover, 3-fold serial dilutions of heat inactivated serum or plasma samples were prepared and mixed with 50 mL of pseudovirus. The mixture was incubated at 37°C for 1 hour before adding to HEK293T-hACE2 cells. Fourty-eight hours after infection, cells were lysed in Steady-Glo Luciferase Assay (Promega - Madison, Wisconsin, USA) according to the manufacturer’s instructions. SARS-CoV-2 neutralization titers were defined as the sample dilution at which a 50% reduction in relative light unit (RLU) was observed relative to the average of the virus control wells.

IFNγ and IL-4 T cell ELISPOT assay

The assay has been described previously [29]. In brief, ELISpot plates were coated with mouse anti-human IFNγ monoclonal antibody (BD Pharmingen - San Diego, California, USA) or anti-human IL-4 monoclonal antibody (Mabtech - Stockholm, Sweden) at a concentration of 5 ug/well overnight at 4°C. Plates were washed with DPBS containing 0.25% Tween 20, and blocked with R10 media (RPMI with 11% FBS and 1.1% penicillin-streptomycin) for 1 hour at 37°C. The Spike 1 and Spike 2 peptide pools contain 15 amino acid peptides overlapping by 11 amino acids that span the protein sequence and reflect the N-terminal and carboxyl-terminal halves of the protein, respectively. Spike 1 and Spike 2 peptide pools were prepared at a concentration of 2 mg/well, and 200,000 cells/well were added. The peptides and cells were incubated for 18 to 24 hours at 37°C. All steps following this incubation were performed at room temperature. The plates were washed with coulter buffer and incubated with Rabbit polyclonal anti-human IFN-g Biotin from U-Cytech (1 mg/mL), followed by a second wash and incubation with Streptavidin-alkaline phosphatase antibody from Southern Biotechnology (Birmingham, Alabama, USA) (1 mg/mL). The final wash was followed by the addition of Nitor-blue Tetrazolium Chloride/5-bromo-4-chloro-3-indolyl phosphate p-toludine salt (NBT/BCIP chromagen) substrate solution for 7 minutes (IFNγ) or 12 minutes (IL-4). The chromagen was discarded, and the plates were washed with water and dried in a dim place for 24 hours. Plates were scanned and counted on a Cellular Technologies Limited Immunospot Analyzer.

ADCP and ADNP

ADCP and ADNP were conducted as previously described [80,81]. Briefly, spike or RBD proteins were biotinylated using EDC (Thermo Fisher Scientific) and EZ-link Sulfo-NHS-LC-LC (Thermo Fisher Scientific) and then coupled to yellow/green and then coupled to yellow/green FluoSphere NeutrAvidin-conjugated beads (Thermo Fisher Scientific, F8776). Immune complexes were formed by incubating the bead+protein conjugates with diluted serum (ADNP 1:50 dilution, ADCP 1:100 dilution) for 2 hours at 37°C and then washed to remove unbound antibody. The immune complexes were then incubated overnight with THP-1 cells (ADCP), or for 1 hour with RBC-lyzed whole blood (ADNP). THP-1 cells were then washed and fixed in 4% PFA, while the RBC-lyzed whole blood was washed, stained for CD66b Pacific Blue (BioLegend - San Diego, California, USA), CD3-AlexaFluro700 (BD Biosciences - Miami, Florida, USA), and CD14-APC-Cy7 (BD Biosciences) to identify neutrophils (CD3- CD14- CD66b+) and then fixed in 4% PFA. Flow cytometry was performed to identify the percentage of quantity of beads phagocytosed by THP-1 cells or neutrophils, and a phagocytosis score was calculated (% cells positive × median fluorescent intensity of positive cells). Flow cytometry was performed with an LSRII (BD), and analysis was performed using FlowJo V10.7.1.

ADCD

ADCD was conducted as previously described [82]. Briefly, spike or RBD protein was biotinylated using EDC (Thermo Fisher Scientific) and EZ-link Sulfo-NHS-LC-LC (Thermo Fisher Scientific) and then coupled to red Neutravidin-conjugated microspheres (Thermo Fisher Scientific). Immune complexes were formed by incubating the bead+protein conjugates with diluted serum (ADCD 1:10 dilution) for 2 hours at 37°C and then washed to remove unbound antibody. The immune complexes were then incubated with lyophilized guinea pig complement (Cedarlane - Burlington, Ontario, Canada) and diluted in gelatin veronal buffer with calcium and magnesium (Boston BioProducts - Milford, Massachusetts, USA) for 30 minutes. C3 bound to immune complexes was detected by fluorescein-conjugated goat IgG fraction to guinea pig Complement C3 (MP Biomedicals - Irvine, California, USA). Flow cytometry was performed to identify the percentage of beads with bound C3 and a complement deposition score was calculated (% beads positive × median fluorescent intensity of positive beads). Flow cytometry was performed with an LSRII (BD) and analysis was performed using FlowJo V10.7.1.

Antibody-dependent NK cell degranulation

Antibody-dependent NK cell degranulation was conducted as previously described [83,84]. Spike or RBD protein was coated on Maxisorp ELISA plate (Thermo Fisher Scientific) and then blocked with 5% BSA. Diluted serum (1:25 dilution) was then added and incubated for 2 hours at 37°C. Human NK cells were isolated from peripheral blood by negative selection using the RosetteSep Human NK cell enrichment cocktail (STEMCELL Technologies - Vancouver, British Columbia, Canada) following the manufacturer’s instructions. Human NK cells were then added to the bound antibody and incubated for 5 hours at 37°C in the presence of RMPI+10% FBS, GolgiStop (BD), Brefeldin A (Sigma - Burlington, Massachusetts, USA), and anti-human CD107a-PE-Cy5 antibody (BD Biosciences). After incubation, cells were washed, stained with CD16-APC-Cy7, CD56-PE-Cy7, and CD3-Pacific Blue (BD Biosciences) and fixed in 4% PFA for 15 minutes. Intracellular staining was performed using the FIX/PERM Cell fixation and permeabilization kit (Thermo Fisher Scientific), and cells were stained for IFNγ-APC and macrophage inflammatory protein-1β-PE (BD Biosciences). Results were reported as percentage of NK cells positive for CD107a, IFNγ, or macrophage inflammatory protein-1β. Flow cytometry was performed with an LSRII (BD), and analysis was performed using FlowJo V10.7.1.

Isotype and FcR-binding Luminex profiling

Isotyping and FcR profiling was conducted as previously described [84–86]. Briefly, spike, RBD, or variant proteins were carboxyl coupled to magnetic Luminex microplex carboxylated beads (Luminex - Austin, Texas, USA) using NHS-ester linkages with Sulfo-NHS and EDC (Thermo Fisher Scientific) and then incubated with serum (Isotypes 1:100 dilution, FcRs 1:1,000 dilution) for 2 hours at 37°C. Isotyping was performed by incubating the immune complexes with secondary mouse-anti-rhesus antibody detectors for each isotype (IgG1, IgG2, IgG3, IgG4, and IgA) and then detected with tertiary anti-mouse-IgG antibodies conjugated to PE. FcR binding was quantified by incubating immune complexes with biotinylated FcRs (FcγR2A-1, FcγR2A-2, FcγR2A-III, FcγR2A-IV, and FcγR3A, courtesy of Duke Protein Production Facility) conjugated to Steptavidin-PE (Prozyme - Hayward, California, USA). Results were reported as median fluorescent intensity of PE staining for each antigen+bead combination. Flow cytometry was performed with an IQue (Intellicyt - Albuquerque, New Mexico, USA) and analysis was performed on IntelliCyt ForeCyt (v8.1).

Multivariate analysis

All analyses were performed after removing the Sham samples and were performed using Python version 3.8.5. PCAs, PLS-DAs, and partial least square regressions (PLS-Rs) were performed using the “scikit-learn” package [87], with all measurements (antibody titers, FcR binding, ADCD, ADNP, ADCC, NK cell cytokine secretion, and degranulation) z-scored using scikit-learn’s StandardScaler(). Samples were color coded on the PCA and PLS-DA visualizations using labels for either vaccine dose group or assignment of “protected” and “not protected/only protected BAL” based on the result of the viral load analysis. Ellipses were drawn for each of the labeled groups and extend 2 standard deviations in all directions based on the distribution of the corresponding group; they thus represent two-dimensional 95% confidence intervals. For the polar barplots, each feature is normalized by dividing by the maximum value of that feature within the dataset to obtain a percentile measurement, and the mean percentile measurement is computed for members of each group and represented on the plot. For the analyses described in Fig 6, PLS-PM [33] was performed using the “plspm” library in R version 4.1.0. Latent variables were comprised of a linear combination of all relevant WT spike-directed antibody features (with for example the “functional latent score” being a linear combination of ADCC, ADCP, ADNP, and NK cell degranulation and cytokine secretion). The value of a latent variable for any given sample was computed as a weighted mixture of these constituent features, with the weights being determined with the PLS-PM protocol, iterating until convergence. The correlations heatmap was constructed using the “pandas” package [88]. Visualization for all multivariate analyses was accomplished using a combination of the “plotly” [89] and “matplotlib” [90] packages.

Statistical analysis

Statistical analysis for the results described in Fig 3 onward was performed using Python 3.8.5. In the analyses described in Fig 6, measurements of antibody, FcR binding, and functional enrichments as well as neutralizing antibody and T-cell IFNγ secretion were represented using violin plots. The region enclosed by each box represents the interquartile range of the variable for a particular group, and the line bisecting each box is the median. For the analysis represented in S3B Fig, each functional enrichment score was calculated by taking an average of the Z-score for each measured variant spike-directed antibody feature (with the exception of B.1.1.7 and B.1.351, for which RBD-directed antibody features were used; for example, “functional enrichment” is the average z-score for ADCC, ADCP, NK cell degranulation, etc.). Mann–Whitney U tests were used to determine statistical significance of differences between groups for both the violin plots and the volcano plot, as normality could not be assumed for these features. Similar analysis was performed to generate S2 Fig, using WT spike-directed features. Two different Python packages were used to conduct these tests, “statannot” and “scipy” [91]. A p-value of ≤ 0.05 was considered significant for an individual test, with this p-value adjusted for multiple hypothesis testing using Holm–Bonferroni corrections (% beads positive × median fluorescent intensity of positive beads). Flow cytometry was performed with an LSRII (BD), and analysis was performed using FlowJo V10.7.1.

Supporting information

S1 Fig. Correlation between each anti–SARS-CoV-2 spike feature, neutralizing antibodies, and T-cell function, between each vaccine dose group.

(A–D) Correlation plots showing the r2 correlation between each immune feature among all vaccine regimens. (A) Group 1 1 × 1011 VPs, (B) Group II, 5 × 1010 VPs, (C) 1.125 × 1010 VPs, and (D) 2 × 109 VPs. This figure can be generated using the data and code deposited in the “src” folder of https://github.com/dzhu8/Ad26-Dose-Down. Instructions to do so can be found in the README file. SARS-CoV-2, Severe Acute Respiratory Syndrome Coronavirus 2; VP, viral particle.

https://doi.org/10.1371/journal.pbio.3001609.s001

(EPS)

S2 Fig. Enrichment scores for WT spike-directed features.

(A) The violin plots show the distribution of each labeled feature between completely protected NHPs (orange) and NHPs lacking complete protection (purple). Functional, antibody, and Fc enrichment were calculated by computing the average z-score of all WT spike-directed effector functions, antibody titers and Fc receptor binding measurements, respectively. This figure can be generated using the data and code deposited in the “src” folder of https://github.com/dzhu8/Ad26-Dose-Down. Instructions to do so can be found in the README file. NHP, nonhuman primate; WT, wild-type.

https://doi.org/10.1371/journal.pbio.3001609.s002

(EPS)

S3 Fig. Further profiling of the humoral response to VOCs.

(A) The humoral response was profiled against several additional variants and SARS-CoV-2 proteins (Spike: WT, D614G, N501Y, E484K, B.1.1.7, K417N; RBD: WT, N501Y, E484K, B.1.351, P.1). The squares represents the z-scored value for each immune feature (x-axis) for each NHP (y-axis). Z-score was calculated across each individual column. (B) The violin plots show the distribution of each labeled feature between completely protected NHPs (orange) and NHPs lacking complete protection (purple). Functional, antibody and Fc enrichment were calculated by computing the average z-score of all anti-variant effector functions, antibody titers and Fc receptor binding measurements, respectively. (C) Scatterplots of WT spike directed enrichment scores against variant-directed enrichment scores, with the computed Spearman correlation included in the plot title. Each point represents an NHP, either completely protected (orange) or lacking complete protection (purple). (D) The PCA scores plot and associated loadings show the degree of discrimination achieved using only the combination of functional, antibody and Fc enrichment scores created with anti-variant features. Each point represents an NHP, either completely protected (orange) or lacking complete protection (purple); ellipses correspond to the 95% confidence intervals for each group. This figure can be generated using the data deposited in the “data” folder and code deposited in the “src” folder of https://github.com/dzhu8/Ad26-Dose-Down. Instructions to do so can be found in the README file. Data for S3A Fig can be found in “data/figure7_data.” NHP, nonhuman primate; PCA, principal component analysis; SARS-CoV-2, Severe Acute Respiratory Syndrome Coronavirus 2; VOC, variant of concern; WT, wild-type.

https://doi.org/10.1371/journal.pbio.3001609.s003

(EPS)

Citation: Zhu DY, Gorman MJ, Yuan D, Yu J, Mercado NB, McMahan K, et al. (2022) Defining the determinants of protection against SARS-CoV-2 infection and viral control in a dose-down Ad26.CoV2.S vaccine study in nonhuman primates. PLoS Biol 20(5): e3001609. https://doi.org/10.1371/journal.pbio.3001609

References

1. Dong E, Du H, Gardner L. An interactive web-based dashboard to track COVID-19 in real time. Lancet Infect Dis. 2020:533–4. pmid:32087114

2. Khoury DS, Cromer D, Reynaldi A, Schlub TE, Wheatley AK, Juno JA, et al. Neutralizing antibody levels are highly predictive of immune protection from symptomatic SARS-CoV-2 infection. Nat Med. 2021;27:1205–11. pmid:34002089

3. Earle KA, Ambrosino DM, Fiore-Gartland A, Goldblatt D, Gilbert PB, Siber GR, et al. Evidence for antibody as a protective correlate for COVID-19 vaccines. Vaccine. 2021;39:4423–8. pmid:34210573

4. Jara A, Undurraga EA, González C, Paredes F, Fontecilla T, Jara G, et al. Effectiveness of an Inactivated SARS-CoV-2 Vaccine in Chile. N Engl J Med. 2021. pmid:34233097

5. Sadoff J, Gray G, Vandebosch A, Cárdenas V, Shukarev G, Grinsztejn B, et al. Safety and Efficacy of Single-Dose Ad26.COV2.S Vaccine against Covid-19. N Engl J Med. 2021. pmid:33882225

6. Tanriover MD, Doğanay HL, Akova M, Güner HR, Azap A, Akhan S, et al. Efficacy and safety of an inactivated whole-virion SARS-CoV-2 vaccine (CoronaVac): interim results of a double-blind, randomised, placebo-controlled, phase 3 trial in Turkey. Lancet. 2021;398:213–22. pmid:34246358

7. Kyriakidis NC, López-Cortés A, González EV, Grimaldos AB, Prado EO. SARS-CoV-2 vaccines strategies: a comprehensive review of phase 3 candidates. NPJ Vaccines. 2021;6:28. pmid:33619260

8. Voysey M, Costa Clemens SA, Madhi SA, Weckx LY, Folegatti PM, Aley PK, et al. Single-dose administration and the influence of the timing of the booster dose on immunogenicity and efficacy of ChAdOx1 nCoV-19 (AZD1222) vaccine: a pooled analysis of four randomised trials. Lancet. 2021;397:881–91. pmid:33617777

9. Alter G, Yu J, Liu J, Chandrashekar A, Borducchi EN, Tostanoski LH, et al. Immunogenicity of Ad26.COV2.S vaccine against SARS-CoV-2 variants in humans. Nature. 2021. pmid:34107529

10. Wang P, Nair MS, Liu L, Iketani S, Luo Y, Guo Y, et al. Antibody resistance of SARS-CoV-2 variants B.1.351 and B.1.1.7. Nature. 2021:1–6.

11. Liu Y, Liu J, Xia H, Zhang X, Fontes-Garfias CR, Swanson KA, et al. Neutralizing Activity of BNT162b2-Elicited Serum. N Engl J Med. 2021;384:1466–8. pmid:33684280

12. Planas D, Bruel T, Grzelak L, Guivel-Benhassine F, Staropoli I, Porrot F, et al. Sensitivity of infectious SARS-CoV-2 B.1.1.7 and B.1.351 variants to neutralizing antibodies. Nat Med. 2021;27:917–24. pmid:33772244

13. Planas D, Veyer D, Baidaliuk A, Staropoli I, Guivel-Benhassine F, Rajah MM, et al. Reduced sensitivity of SARS-CoV-2 variant Delta to antibody neutralization. Nature. 2021;596:276–80. pmid:34237773

14. Browne SK, Beeler JA, Roberts JN. Summary of the Vaccines and Related Biological Products Advisory Committee meeting held to consider evaluation of vaccine candidates for the prevention of respiratory syncytial virus disease in RSV-naïve infants. Vaccine. 2020;38:101–6. pmid:31706809

15. Polack FP, Thomas SJ, Kitchin N, Absalon J, Gurtman A, Lockhart S, et al. Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine. N Engl J Med. 2020;383:2603–15. pmid:33301246

16. Jackson LA, Anderson EJ, Rouphael NG, Roberts PC, Makhene M, Coler RN, et al. An mRNA Vaccine against SARS-CoV-2—Preliminary Report. N Engl J Med. 2020;383:1920–31. pmid:32663912

17. Pegu A, O’Connell S, Schmidt SD, O’Dell S, Talana CA, Lai L, et al. Durability of mRNA-1273 vaccine-induced antibodies against SARS-CoV-2 variants. Science. 2021. pmid:34385356

18. Shan C, Yao Y-F, Yang X-L, Zhou Y-W, Gao G, Peng Y, et al. Infection with novel coronavirus (SARS-CoV-2) causes pneumonia in Rhesus macaques. Cell Res. 2020;30:670–7. pmid:32636454

19. Winkler ES, Gilchuk P, Yu J, Bailey AL, Chen RE, Chong Z, et al. Human neutralizing antibodies against SARS-CoV-2 require intact Fc effector functions for optimal therapeutic protection. Cell. 2021;1804–20.e16. pmid:33691139

20. Chan CEZ, Seah SGK, Chye DH, Massey S, Torres M, Lim APC, et al. The Fc-mediated effector functions of a potent SARS-CoV-2 neutralizing antibody, SC31, isolated from an early convalescent COVID-19 patient, are essential for the optimal therapeutic efficacy of the antibody. PLoS ONE. 2021;16:e0253487. pmid:34161386

21. Ravetch J, Yamin R, Jones A, Hoffmann H-H, Kao K, Francis R, et al. Fc-engineered antibody therapeutics with improved efficacy against COVID-19. Res Sq. 2021. pmid:34075373

22. Chen G, Wu D, Guo W, Cao Y, Huang D, Wang H, et al. Clinical and immunological features of severe and moderate coronavirus disease 2019. J Clin Invest. 2020;130:2620–9. pmid:32217835

23. Dispinseri S, Secchi M, Pirillo MF, Tolazzi M, Borghi M, Brigatti C, et al. Neutralizing antibody responses to SARS-CoV-2 in symptomatic COVID-19 is persistent and critical for survival. Nat Commun. 2021;12:2670. pmid:33976165

24. Zohar T, Loos C, Fischinger S, Atyeo C, Wang C, Slein MD, et al. Compromised Humoral Functional Evolution Tracks with SARS-CoV-2 Mortality. Cell. 2020;183: 1508–19.e12. pmid:33207184

25. Grifoni A, Weiskopf D, Ramirez SI, Mateus J, Dan JM, Moderbacher CR, et al. Targets of T Cell Responses to SARS-CoV-2 Coronavirus in Humans with COVID-19 Disease and Unexposed Individuals. Cell. 2020;181: 1489–501.e15. pmid:32473127

26. Dan JM, Mateus J, Kato Y, Hastie KM, Yu ED, Faliti CE, et al. Immunological memory to SARS-CoV-2 assessed for up to eight months after infection. bioRxiv. 2020. pmid:33442687

27. Tarke A, Sidney J, Methot N, Zhang Y, Dan JM, Goodwin B, et al. Negligible impact of SARS-CoV-2 variants on CD4+ and CD8+ T cell reactivity in COVID-19 exposed donors and vaccinees. bioRxiv. 2021. p. 2021.02.27.433180. pmid:33688655

28. Chung AW, Kumar MP, Arnold KB, Yu WH, Schoen MK, Dunphy LJ, et al. Dissecting Polyclonal Vaccine-Induced Humoral Immunity against HIV Using Systems Serology. Cell. 2015;163:988–8. pmid:26544943

29. He X, Chandrashekar A, Zahn R, Wegmann F, Yu J, Mercado NB, et al. Low-dose Ad26.COV2.S protection against SARS-CoV-2 challenge in rhesus macaques. Cell. 2021;184: 3467–73.e11. pmid:34133941

30. Roozendaal R, Solforosi L, Stieh D, Serroyen J, Straetemans R, Wegmann F, et al. SARS-CoV-2 binding and neutralizing antibody levels after vaccination with Ad26.COV2.S predict durable protection in rhesus macaques. pmid:34620860

31. Stavnezer J. Antibody Class Switching. Adv Immunol. 1996:79–146. pmid:8834495

32. Schmidt ME, Varga SM. The CD8 T Cell Response to Respiratory Virus Infections. Front Immunol. 2018;9:678. pmid:29686673

33. Sanchez G. PLS path modeling with R. Berkeley: Trowchez Editions. 2013;383: 2013.

34. Gupta RK. Will SARS-CoV-2 variants of concern affect the promise of vaccines? Nat Rev Immunol. 2021. pmid:33927376

35. Walensky RP, Walke HT, Fauci AS. SARS-CoV-2 Variants of Concern in the United States—Challenges and Opportunities. JAMA. 2021:1037. pmid:33595644

36. Chen RE, Zhang X, Case JB, Winkler ES, Liu Y, VanBlargan LA, et al. Resistance of SARS-CoV-2 variants to neutralization by monoclonal and serum-derived polyclonal antibodies. Nat Med. 2021;27:717–26. pmid:33664494

37. Graham C, Seow J, Huettner I, Khan H, Kouphou N, Acors S, et al. Neutralization potency of monoclonal antibodies recognizing dominant and subdominant epitopes on SARS-CoV-2 Spike is impacted by the B.1.1.7 variant. Immunity. 2021. pmid:33836142

38. Ho D, Wang P, Liu L, Iketani S, Luo Y, Guo Y, et al. Increased Resistance of SARS-CoV-2 Variants B.1.351 and B.1.1.7 to Antibody Neutralization. bioRxiv. pmid:33532763

39. Korber B, Fischer WM, Gnanakaran S, Yoon H, Theiler J, Abfalterer W, et al. Tracking Changes in SARS-CoV-2 Spike: Evidence that D614G Increases Infectivity of the COVID-19 Virus. Cell. 2020;182: 812–27.e19. pmid:32697968

40. Volz E, Mishra S, Chand M, Barrett JC, Johnson R, Geidelberg L, et al. Transmission of SARS-CoV-2 Lineage B. 1.1. 7 in England: Insights from linking epidemiological and genetic data. medRxiv. 2021:2020–12.

41. Tegally H, Wilkinson E, Giovanetti M, Iranzadeh A, Fonseca V, Giandhari J, et al. Detection of a SARS-CoV-2 variant of concern in South Africa. Nature. 2021;592:438–43. pmid:33690265

42. Faria NR, Mellan TA, Whittaker C, Claro IM, Candido D da S, Mishra S, et al. Genomics and epidemiology of the P.1 SARS-CoV-2 lineage in Manaus, Brazil. Science. 2021;372:815–21. pmid:33853970

43. Lasek-Nesselquist E, Lapierre P, Schneider E, St. George K, Pata J. The localized rise of a B.1.526 SARS-CoV-2 variant containing an E484K mutation in New York State.

44. Baden LR, El Sahly HM, Essink B, Kotloff K, Frey S, Novak R, et al. Efficacy and Safety of the mRNA-1273 SARS-CoV-2 Vaccine. N Engl J Med. 2021;384:403–16. pmid:33378609

45. Folegatti PM, Ewer KJ, Aley PK, Angus B, Becker S, Belij-Rammerstorfer S, et al. Safety and immunogenicity of the ChAdOx1 nCoV-19 vaccine against SARS-CoV-2: a preliminary report of a phase 1/2, single-blind, randomised controlled trial. Lancet. 2020;396:467–78. pmid:32702298

46. Wu S, Huang J, Zhang Z, Wu J, Zhang J, Hu H, et al. Safety, tolerability, and immunogenicity of an aerosolised adenovirus type-5 vector-based COVID-19 vaccine (Ad5-nCoV) in adults: preliminary report of an open-label and randomised phase 1 clinical trial. Lancet Infect Dis. 2021. pmid:34324836

47. Kalimuddin S, Tham CYL, Qui M, de Alwis R, Sim JXY, Lim JME, et al. Early T cell and binding antibody responses are associated with COVID-19 RNA vaccine efficacy onset. Med (N Y). 2021;2: 682–8.e4. pmid:33851143

48. Herman JD, Wang C, Loos C, Yoon H, Rivera J, Dieterle ME, et al. Functional Antibodies in COVID-19 Convalescent Plasma. medRxiv. 2021. pmid:33758875

49. McMahan K, Yu J, Mercado NB, Loos C, Tostanoski LH, Chandrashekar A, et al. Correlates of protection against SARS-CoV-2 in rhesus macaques. Nature. 2021;590:630–4. pmid:33276369

50. Tan AT, Linster M, Tan CW, Le Bert N, Chia WN, Kunasegaran K, et al. Early induction of functional SARS-CoV-2-specific T cells associates with rapid viral clearance and mild disease in COVID-19 patients. Cell Rep. 2021;34:108728. pmid:33516277

51. Long Q-X, Liu B-Z, Deng H-J, Wu G-C, Deng K, Chen Y-K, et al. Antibody responses to SARS-CoV-2 in patients with COVID-19. Nat Med. 2020;26:845–8. pmid:32350462

52. Del Valle DM, Kim-Schulze S, Huang H-H, Beckmann ND, Nirenberg S, Wang B, et al. An inflammatory cytokine signature predicts COVID-19 severity and survival. Nat Med. 2020;26:1636–43. pmid:32839624

53. Saris A, Reijnders TDY, Nossent EJ, Schuurman AR, Verhoeff J, van Asten S, et al. Distinct cellular immune profiles in the airways and blood of critically ill patients with COVID-19. Thorax. 2021;76:1010–9. pmid:33846275

54. Liao M, Liu Y, Yuan J, Wen Y, Xu G, Zhao J, et al. Single-cell landscape of bronchoalveolar immune cells in patients with COVID-19. Nat Med. 2020;26:842–4. pmid:32398875

55. Ravindran MS, Bagchi P, Cunningham CN, Tsai B. Opportunistic intruders: how viruses orchestrate ER functions to infect cells. Nat Rev Microbiol. 2016;14:407–20. pmid:27265768

56. Wu K, Werner AP, Moliva JI, Koch M, Choi A, Stewart-Jones GBE, et al. mRNA-1273 vaccine induces neutralizing antibodies against spike mutants from global SARS-CoV-2 variants. bioRxiv. 2021. pmid:33501442

57. Abu-Raddad LJ, Chemaitelly H, Butt AA. National Study Group for COVID-19 Vaccination. Effectiveness of the BNT162b2 Covid-19 Vaccine against the B.1.1.7 and B.1.351 Variants. N Engl J Med. 2021. pmid:33951357

58. Madhi SA, Baillie V, Cutland CL, Voysey M, Koen AL, Fairlie L, et al. Efficacy of the ChAdOx1 nCoV-19 Covid-19 Vaccine against the B.1.351 Variant. N Engl J Med. 2021;384:1885–98. pmid:33725432

59. Plante JA, Liu Y, Liu J, Xia H, Johnson BA, Lokugamage KG, et al. Spike mutation D614G alters SARS-CoV-2 fitness. Nature. 2021;592:116–21. pmid:33106671

60. Volz E, Hill V, McCrone JT, Price A, Jorgensen D, O’Toole Á, et al. Evaluating the Effects of SARS-CoV-2 Spike Mutation D614G on Transmissibility and Pathogenicity. Cell. 2021;184: 64–75.e11. pmid:33275900

61. Hou YJ, Chiba S, Halfmann P, Ehre C, Kuroda M, Dinnon KH 3rd, et al. SARS-CoV-2 D614G variant exhibits efficient replication ex vivo and transmission in vivo. Science. 2020;370:1464–8. pmid:33184236

62. Linka K, Peirlinck M, Schafer A, Tikenogullari OZ, Goriely A, Kuhl E. Effects of B. 1.1. 7 and B. 1.351 on COVID-19 dynamics. A campus reopening study. medRxiv. 2021. Available from: https://www.medrxiv.org/content/10.1101/2021.04.22.21255954v1.abstract. pmid:34456557

63. Shinde V, Bhikha S, Hoosain Z, Archary M, Bhorat Q, Fairlie L, et al. Efficacy of NVX-CoV2373 Covid-19 Vaccine against the B.1.351 Variant. N Engl J Med. 2021. pmid:33951374

64. Novavax COVID-19 vaccine demonstrates 89.3% efficacy in UK phase 3 trial. [cited 7 May 2021]. Available from: https://ir.novavax.com/news-releases/news-release-details/novavax-covid-19-vaccine-demonstrates-893-efficacy-uk-phase-3.

65. Abu-Raddad LJ, Chemaitelly H, Butt AA. Effectiveness of the BNT162b2 Covid-19 Vaccine against the B.1.1.7 and B.1.351 Variants. N Engl J Med. 2021. pmid:33951357

66. Bernal JL, Andrews N, Gower C, Gallagher E, Simmons R, Thelwall S, et al. Effectiveness of Covid-19 Vaccines against the B.1.617.2 (Delta) Variant. N Engl J Med. 2021:585–94. pmid:34289274

67. Schmidt F, Muecksch F, Weisblum Y, Da Silva J, Bednarski E, Cho A, et al. Plasma Neutralization of the SARS-CoV-2 Omicron Variant. N Engl J Med. 2021. pmid:35030645

68. Yu X, Wei D, Xu W, Li Y, Li X, Zhang X, et al. Reduced sensitivity of SARS-CoV-2 Omicron variant to antibody neutralization elicited by booster vaccination. Cell Discov. 2022;8:4. pmid:35034952

69. Muik A, Lui BG, Wallisch A-K, Bacher M, Mühl J, Reinholz J, et al. Neutralization of SARS-CoV-2 Omicron by BNT162b2 mRNA vaccine-elicited human sera. Science. 2022:eabn7591. pmid:35040667

70. Garcia-Beltran WF, St Denis KJ, Hoelzemer A, Lam EC, Nitido AD, Sheehan ML, et al. mRNA-based COVID-19 vaccine boosters induce neutralizing immunity against SARS-CoV-2 Omicron variant. Cell. 2022. pmid:34995482

71. Ahmed SF, Quadeer AA, McKay MR. SARS-CoV-2 T Cell Responses Elicited by COVID-19 Vaccines or Infection Are Expected to Remain Robust against Omicron. Viruses. 2022;14. pmid:35062283

72. Ledford H. “Killer” immune cells still recognize Omicron variant. Nature. 2022;601:307. pmid:35017690

73. May DH, BER R, Dalai SC, Patel K, Shafiani S, Elyanow R, et al. Immunosequencing and epitope mapping reveal substantial preservation of the T cell immune response to Omicron generated by SARS-CoV-2 vaccines.

74. GeurtsvanKessel CH, Geers D, Schmitz KS, Mykytyn AZ, Lamers MM, Bogers S, et al. Divergent SARS CoV-2 Omicron-specific T- and B-cell responses in COVID-19 vaccine recipients. bioRxiv. 2021.

75. Keeton R, Tincho MB, Ngomti A, Baguma R, Benede N, Suzuki A, et al. SARS-CoV-2 spike T cell responses induced upon vaccination or infection remain robust against Omicron. medRxiv. 2021. Available from: https://www.medrxiv.org/content/10.1101/2021.12.26.21268380v1?s=09.

76. Liu J, Chandrashekar A, Sellers D, Barrett J, Lifton M, McMahan K, et al. Vaccines Elicit Highly Cross-Reactive Cellular Immunity to the SARS-CoV-2 Omicron Variant. medRxiv. 2022. pmid:35018387

77. Bartsch Y, Tong X, Kang J, Avendaño MJ, Serrano EF, García-Salum T, et al. Preserved Omicron Spike specific antibody binding and Fc-recognition across COVID-19 vaccine platforms. medRxiv. 2021. pmid:34981072

78. Liu Y, Yan L-M, Wan L, Xiang T-X, Le A, Liu J-M, et al. Viral dynamics in mild and severe cases of COVID-19. Lancet Infect Dis. 2020;20:656–7. pmid:32199493

79. Wölfel R, Corman VM, Guggemos W, Seilmaier M, Zange S, Müller MA, et al. Virological assessment of hospitalized patients with COVID-2019. Nature. 2020;581:465–9. pmid:32235945

80. Butler AL, Fallon JK, Alter G. A Sample-Sparing Multiplexed ADCP Assay. Front Immunol. 2019;10:1851. pmid:31456799

81. Karsten CB, Mehta N, Shin SA, Diefenbach TJ, Slein MD, Karpinski W, et al. A versatile high-throughput assay to characterize antibody-mediated neutrophil phagocytosis. J Immunol Methods. 2019;471:46–56. pmid:31132351

82. Fischinger S, Fallon JK, Michell AR, Broge T, Suscovich TJ, Streeck H, et al. A high-throughput, bead-based, antigen-specific assay to assess the ability of antibodies to induce complement activation. J Immunol Methods. 2019;473:112630. pmid:31301278

83. Gunn BM, Yu W-H, Karim MM, Brannan JM, Herbert AS, Wec AZ, et al. A Role for Fc Function in Therapeutic Monoclonal Antibody-Mediated Protection against Ebola Virus. Cell Host Microbe. 2018;24: 221–33.e5. pmid:30092199

84. Mercado NB, Zahn R, Wegmann F, Loos C, Chandrashekar A, Yu J, et al. Single-shot Ad26 vaccine protects against SARS-CoV-2 in rhesus macaques. Nature. 2020;586:583–8. pmid:32731257

85. Brown EP, Licht AF, Dugast A-S, Choi I, Bailey-Kellogg C, Alter G, et al. High-throughput, multiplexed IgG subclassing of antigen-specific antibodies from clinical samples. J Immunol Methods. 2012;386:117–23. pmid:23023091

86. Brown EP, Dowell KG, Boesch AW, Normandin E, Mahan AE, Chu T, et al. Multiplexed Fc array for evaluation of antigen-specific antibody effector profiles. J Immunol Methods. 2017;443:33–44. pmid:28163018

87. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, et al. Scikit-learn: Machine learning in Python. J Mach Learn Res. 2011;12:2825–30.

88. McKinney W, Others. Data structures for statistical computing in python. Proceedings of the 9th Python in Science Conference. Austin, TX; 2010. pp. 51–56.

89. Sievert C. Interactive Web-Based Data Visualization with R, plotly and shiny. 2020.

90. Hunter. Matplotlib: A 2D Graphics Environment. 2007;9:90–5.

91. Virtanen P, Gommers R, Oliphant TE, Haberland M, Reddy T, Cournapeau D, et al. SciPy 1.0: fundamental algorithms for scientific computing in Python. Nat Methods. 2020;17:261–72. pmid:32015543