Low agreement and frequent invalid controls in two SARS-CoV-2 T-cell assays in people with compromised immune function

Abstract

T-cell response plays an important role in SARS-CoV-2 immunogenicity. For people living with HIV (PWH) and solid organ transplant (SOT) recipients there is limited evidence on the reliability of commercially available T-cell tests. We assessed 173 blood samples from 81 participants (62 samples from 35 PWH; 111 samples from 46 SOT recipients [lung and kidney]) with two commercial SARS-CoV-2 Interferon-γ (IFN-γ) release assays (IGRA; SARS-CoV-2 IGRA by Euroimmun, and IGRA SARS-CoV-2 by Roche). The reliability between the tests was judged as low (Cohen’s kappa [κ] = 0.20; overall percent agreement [OPA] = 66%). A high proportion of tests were invalid (22% Euroimmun; 8% Roche). When excluding these invalid tests, the agreement was higher (κ  =  0.43; OPA = 90%). The low reliability between the two T-cell tests indicates that results should be interpreted with caution in SOT recipients and PWH and that SARS-CoV-2 T-cell tests need to be optimized and further validated for use in vulnerable patient populations.

Introduction

By the end of 2020, initial findings from SARS-CoV-2 vaccine trials became available, indicating that the vaccines were over 90% effective in temporarily preventing COVID-19 [1, 2]. Given uncertainties about the vaccines’ efficacy in individuals with compromised immune systems, our research group initiated the Corona VaccinE tRiAL pLatform (COVERALL) within the Swiss HIV Cohort Study (SHCS) and the Swiss Transplant Cohort Study (STCS) [3–5]. Since the vaccines were designed to stimulate an antibody response against the SARS-CoV-2 spike (S1) protein receptor binding domain, the first two COVERALL studies assessed whether people with HIV (PWH) and solid organ transplant (SOT) recipients developed adequate antibody response post vaccination [6, 7]. The emergence of highly mutable variants like Delta and Omicron compromised the neutralizing capability of antibodies [8], emphasizing the growing importance of the T-cell response [8–11].

Therefore, not only the antibody response but also T-cell response in PWH and SOT recipients (i.e. lung and kidney transplants), following a bivalent mRNA SARS-CoV-2 booster vaccination, was evaluated [12]. During this evaluation we observed differences amongst test results when using different commercially available T-cell tests. Since the reliability of the T-cell tests in people with compromised immune function is unclear, we therefore aimed to assess the agreement of two commercially available T-cell tests.

Methods

Study design

The methods and main results of the COVERALL-3 study have been published separately [12]. In brief, cohort participants from the SHCS and the STCS who had previously already revived the “basic immunization” SARS-CoV-2 vaccination (e.g. two doses of Spikevax from Moderna or two doses of Comirnaty from Pfizer-BioNtech) and who received the bivalent vaccine (mRNA-1273.214 or BA.1–adapted BNT162b2) were recruited from the 27^th of October 2022 until the 24^th of January 2023. They provided whole blood samples before vaccination (i.e. baseline), and at 4-weeks, 8-weeks, and 6-months post vaccination to measure the antibody response (full eligibility criteria in Supplement 1). A subset of participants provided 8 ml heparinized blood at baseline, 4-weeks, and 6-months to assess the T-cell response. Samples were promptly transported within 6 hours to the Institute of Medical Virology in Zürich where two commercial SARS-CoV-2 Interferon-γ (IFN-γ) release assays (IGRA) by Euroimmun, and Roche were conducted (i.e. SARS-CoV-2 IGRA by Euroimmun, and IGRA SARS-CoV-2 by Roche). The COVERALL-3 study received approval from the ethics committee Nordwest- and Zentralschweiz, Switzerland (BASEC Nr. 2022–01760), and the study protocol is accessible in a trial registry (https://clinicaltrials.gov/ct2/show/NCT04805125). Participants provided written consent for study participation and further utilization of collected biological samples.

Laboratory measurements

Heparinized blood samples were processed within 16 hours after blood withdrawal. The two following SARS-CoV-2 IGRA were used for analyzing the T-cell response following the manufacturers’ instructions: i) the quantitative SARS-CoV-2 IGRA by Euroimmun which combines the Quant-T-Cell SARS-CoV-2 kit for the T-cell stimulation and the Quant-T-Cell ELISA for measuring the released IFN-γ (Euroimmun Medizinische Labordiagnostica, Lübeck, Germany); and ii) the qualitative IGRA SARS‑CoV‑2 by Roche which combines the cobas IGRA SARS-CoV-2 tubes for the T-cell stimulation and the Elecsys IGRA SARS-CoV-2, an electrochemiluminescence immunoassay (ECLIA), for measuring the released IFN-γ (Roche Diagnostics, Rotkreuz, Switzerland).

The kits for T-cell stimulation from both manufacturers consist of three stimulation tubes per whole-blood sample: 1) no T-cell stimulation, for determination of the individual IFN-γ background; 2) specific T-cell stimulation (Euroimmun: antigens based on the S1-domain of the SARS-CoV-2 spike protein; Roche: peptides derived from structural and non-structural proteins of SARS‑CoV‑2); and 3) unspecific T-cell stimulation by means of a mitogen, for control of the stimulation ability. The Euroimmun assay uses antigens based on the S1-domain of the SARS-CoV-2 spike protein; the Roche assay uses more than 180 antigens derived from structural (spike, membrane and nucleocapsid) and non-structural proteins of SARS-CoV-2.

Data generated with the immunoassays were analysed applying the validation criteria for the negative and positive controls as defined by the manufacturers (Euroimmun: positive control ≥400 mIU/ml, negative control ≤400 mIU/ml; Roche: positive control ≥1 IU/ml, negative control ≤0.3 IU/ml). Samples with a non-valid or indeterminate negative or positive stimulation control were labelled “invalid”. Results of the specific stimulation were classified according to the manufacturers’ recommendations either as positive (>200 mIU/ml), negative (<100 mIU/ml), borderline (≥100 mIU/ml to ≤200 mIU/ml), or invalid for the Euroimmun assay, and as positive (≥0.013IU/ml), negative (<0.013 IU/ml), or invalid for the Roche assay (for comparability, non-reactive and reactive results with the Roche test were labelled negative and positive, respectively).

Data analysis

We used Cohen’s kappa to assess the agreement between the two T-cell tests. Cohen’s kappa takes also into account the agreement by chance which can be high when the proportion of true negative or true positive is high (see references for further details [13, 14]). In addition, we also present the simple overall percentage agreement (OPA) which does not adjust for chance agreement. The overall percentage agreement simply assesses the number of agreeing test results for both diagnostic tests divided by all test results. For the interpretation of the kappa we used the following scale as proposed by McHugh [13]: κ  =  0–0.20 indicates no agreement; κ  =  0.21–0.39 suggests minimal agreement; κ  =  0.40–0.59 indicates weak agreement; κ  =  0.60–0.70 signifies moderate agreement; κ  =  0.80–0.90 implies strong agreement; and κ>0.90 denotes almost perfect agreement. In the main analysis, we created a 3x3 table (comprising positive T-cell response, negative T-cell response, and invalid tests) in which borderline results from the Euroimmun test were categorized as positive results. In contrast to other studies [15, 16] we included invalid in our analyses as also these results need to be communicated to clinicians and patients in routine practice. However, we conducted two sensitivity analyses, in which we (i) excluded samples classified as invalid by either test (while treating borderline results from the Euroimmun test as positive), and (ii) treated borderline results from the Euroimmun test as negative. Furthermore, we explored patient characteristics associated with invalid test results, conducted analyses including only PWH or SOT recipients, and assessed each time point separately.

Results

Between October 2022 and September 2023, 81 of the 174 COVERALL-3 [12] study participants provided blood samples to assess the T-cell response. Among these 81 participants, 173 samples were available for testing with both T-cell tests (baseline [n = 40], 4-weeks follow-up [n = 74] and 6-months follow-up [n = 59]). Of these 173 samples, 62 were from PWH (20 with CD4 cell counts <350 cells/μl; 42 with CD4 cell counts ≥350 cells/μl) and 111 from SOT recipients (80 lung transplants; 31 kidney transplants). Baseline characteristics of the 81 participants are presented in S1 Table in S1 File.

Using the Euroimmun T-cell test, 105 of the 173 (60.7%) samples exhibited a positive T-cell response, 12 (6.9%) were borderline, 18 (10.4%) showed no response, and 38 (22.0%) were invalid (Table 1). Using the Roche T-cell test, 139 (80.3%) samples had a positive response, 20 (11.6%) showed no response, and 14 (8.1%) were invalid (Table 1). Among PWH, 2 (3.2%) had an invalid result using the Euroimmun test and 8 (12.9%) with the Roche test. Among SOT recipients, 36 (32.4%) had an invalid result with the Euroimmun test and 6 (5.4%) with the Roche test (Table 1).

[Figure omitted. See PDF.]

Classifying borderline results from the Euroimmun test as positive, kappa was minimal (κ  =  0.20) and the OPA was 66% (Table 1; S2 Table in S1 File). When stratifying by PWH (κ  =  0.14; OPA: 85%) and SOT recipients (κ  =  0.18; OPA: 54%), higher OPA for PWH and low kappas were observed (Table 1). The overall kappa decreased further by treating borderline results from the Euroimmun test as negative (κ  =  0.14; OPA = 59%; S3 Table in S1 File). The agreement was similarly low when separately assessing samples from baseline, 4-weeks, and 6-months (S4 Table in S1 File). Excluding all invalid results from either test, a high OPA (90%), but a weak agreement based on kappa was observed (Table 2).

[Figure omitted. See PDF.]

For both tests, IFN-γ concentrations of the positive controls were lower for SOT recipients compared to PWH (Fig 1). For two SOT recipients, all samples from all three timepoints were invalid using the Euroimmun test (S1 Fig in S1 File). Detailed analyses showed a median of 426 days since organ transplantation (Interquartile Range [IQR] 351–500) for these SOT recipients, in contrast to 2,822 days (IQR: 1,072–3,315) for SOT recipients with no invalid Euroimmun test (S5 Table in S1 File). Positive controls that were invalid with the Euroimmun test had low positive control values with the Roche test (S2 Fig in S1 File). A description of all invalid test results is provided in S6 Table in S1 File. Especially, most invalid results of the Euroimmun test were invalid positive controls in SOT recipients.

[Figure omitted. See PDF.]

Red lines indicate the thresholds for positivity (and borderline) for patients samples and for validity of the positive and negative controls, respectively. Abbreviations: IGRA = Interferon-γ release assays; PWH = People with HIV; SOT = Solid organ transplant.

Discussion

Ideally two commercially available tests measuring the SARS-CoV-2 T-cell response in the same blood sample should come to the same result in order to be trustworthy and so that clinicians can make an informed decision together with patients. Our study showed low agreement based on kappa when assessing the T-cell response in individuals with compromised immune function with the two commercially available SARS-CoV-2 IGRA T-cell tests by Euroimmun and Roche. As case distribution between positive and negative was uneven (especially amongst PWH), a low kappa did not necessarily mean low OPA (kappa paradox [17]). However, the high OPA was also strongly driven by the high expected agreement (i.e. agreement by chance), and the low kappa confirmed that the tests had difficulties in distinguishing the few negative and the invalid results. In agreement with our results, Carretero and colleagues found a low kappa (κ  =  0.29) for the T-cell tests by Euroimmun and Roche amongst 50 kidney patients (n = 24 kidney transplant recipients; n = 26 chronic kidney disease) [15]. In contrast to these results, a study involving 89 healthcare workers found substantial agreement among four T-cell assays (two ELISA-based IGRA and two IFN-γ ELISPOT assays) [18]. The authors observed a stronger correlation between assays within the same platform (IGRA or ELISPOT) than between different platforms. The measurement of released IFN-γ either in supernatant (ELISA-based IGRA), or immobilized (IFN-γ ELISPOT) could contribute to this difference. Of note, the Euroimmun IGRA and the QuantiFERON IGRA (Qiagen) are both ELISAs, while the IGRA by Roche used in our study is an electrochemiluminescence immunoassay. Additionally, the discrepancy between the Euroimmun and Roche IGRAs may also stem from differences in the type of antigen utilized. Natural SARS-CoV-2 infection induces cellular immunity not only towards spike (the Euroimmun antigen is solely based on the S1-domain of the spike protein), but also towards other viral proteins (the Roche antigens are peptides derived from several structural and non-structural proteins).

The potential for invalid results due to impaired T-cell activity or low T-cell numbers, among patients with different levels of immunosuppression, represents an additional challenge for T-cell response assessment. A large proportion (22%) of Euroimmun SARS-CoV-2 IGRA tests, mainly from participants having recently received a SOT, were invalid due to an invalid positive control. This might be a result of the more immunosuppressed state of this patient group compared to PWH, in combination with the antigen used in the Euroimmun test. This finding is in line with a study conducted by Saad-Albichr et al., where 20% of kidney transplant recipients had invalid Euroimmun T-cell results [16]. Conversely, when using the IGRA SARS-CoV-2 by Roche to assess the T-cell response, the proportion of invalid results in our study was 8% (14/173), mainly due to an invalid negative control (79%; 11/14). The difference in the proportion of invalid positive controls between the two tests may stem from differences in the type and concentration of the mitogens/stimulants in the positive controls, details of which are undisclosed by the manufacturers. Meanwhile, discrepancies in the proportion of invalid negative controls are likely due to varying cut-offs established by the manufacturers. These data underscore the necessity for optimizing T-cell assays for patients with different levels of immunosuppression.

The following limitations are worth mentioning: Firstly, we found low agreement between the two T-cell tests. However, due to the absence of a gold standard we cannot make any statement about the true performance of the individual tests. While some assessments showed high sensitivity and specificity of the SARS-CoV-2 IGRA by Euroimmun in the general population [19, 20], we are aware of only one published assessment of the IGRA SARS-CoV-2 by Roche which was conducted in immunosuppressed patients and excluded invalid results from the analyses [15]). Secondly, with 173 samples from 81 participants our sample size is limited. Hence, especially the sub-group analyses should be interpreted with caution.

In conclusion, our results indicate low agreement between two commercial SARS-CoV-2 T-cell tests in PWH and SOT recipients, raising concerns regarding the reliability of these tests in individuals with compromised immune function. Especially the Euroimmun test showed a high proportion of invalid test results in SOT recipients due to an insufficient positive control. Hence, optimizing and validating SARS-CoV-2 T-cell tests in vulnerable populations, for whom such tests are of particular importance, is crucial.

Supporting information

S1 Dataset.

https://doi.org/10.1371/journal.pone.0317965.s001

(XLSX)

S1 File.

https://doi.org/10.1371/journal.pone.0317965.s002

(DOCX)

Acknowledgments

We are grateful to all involved study staff at the local centers in Basel (Maria Pascarella, Louise Seiler), Zürich (Daniela Gsell, Katia Dettling, Laura Tschuor, Christina Grube, Flurina Brunschweiler, Christine Schneider, Andrea Wallensteiner, Daniel Götsch, Andrea Macedo, Esther Göldi, Alina Imoli), and the Institute of Medical Virology, University of Zürich. Most importantly, we thank all study participants.

Members of the Swiss HIV Cohort Study

Abela I, Aebi-Popp K, Anagnostopoulos A, Battegay M, Bernasconi E, Braun DL, Bucher HC, Calmy A, Cavassini M, Ciuffi A, Dollenmaier G, Egger M, Elzi L, Fehr J, Fellay J, Furrer H, Fux CA, Günthard HF (President of the SHCS), Hachfeld A, Haerry D (deputy of "Positive Council"), Hasse B, Hirsch HH, Hoffmann M, Hösli I, Huber M, Jackson-Perry D (patient representatives), Kahlert CR (Chairman of the Mother & Child Substudy), Kaiser L, Keiser O, Klimkait T, Kouyos RD, Kovari H, Kusejko K (Head of Data Centre), Labhardt N, Leuzinger K, Martinez de Tejada B, Marzolini C, Metzner KJ, Müller N, Nemeth J, Nicca D, Notter J, Paioni P, Pantaleo G, Perreau M, Rauch A (Chairman of the Scientific Board), Salazar-Vizcaya L, Schmid P, Speck R, Stöckle M (Chairman of the Clinical and Laboratory Committee), Tarr P, Trkola A, Wandeler G, Weisser M, Yerly S.

Members of the Swiss Transplant Cohort Study

The members of the Swiss Transplant Cohort Study: Patrizia Amico, John-David Aubert, Vanessa Banz, Sonja Beckmann, Guido Beldi, Christoph Berger, Ekaterine Berishvili, Annalisa Berzigotti, Isabelle Binet, Pierre-Yves Bochud, Sanda Branca, Heiner C. Bucher, Emmanuelle Catana, Anne Cairoli, Yves Chalandon, Sabina De Geest, Olivier De Rougemont, Sophie De Seigneux, Michael Dickenmann, Joëlle Lynn Dreifuss, Michel Duchosal, Thomas Fehr, Sylvie Ferrari-Lacraz, Christian Garzoni, Déla Golshayan, Nicolas Goossens, Fadi Haidar, Jörg Halter, Dominik Heim, Christoph Hess, Sven Hillinger, Hans H Hirsch, Patricia Hirt, Linard Hoessly, Günther Hofbauer, Uyen Huynh-Do, Franz Immer, Michael Koller, Bettina Laesser, Frédéric Lamoth, Roger Lehmann, Alexander Leichtle, Oriol Manuel, Hans-Peter Marti, Michele Martinelli, Valérie McLin, Katell Mellac, Aurélia Merçay, Karin Mettler, Nicolas J Mueller, Ulrike Müller-Arndt, Beat Müllhaupt, Mirjam Nägeli, Graziano Oldani, Manuel Pascual, Jakob Passweg, Rosemarie Pazeller, Klara Posfay-Barbe, Juliane Rick, Anne Rosselet, Simona Rossi, Silvia Rothlin, Frank Ruschitzka, Thomas Schachtner, Stefan Schaub, Alexandra Scherrer, Aurelia Schnyder, Macé Schuurmans, Simon Schwab, Thierry Sengstag, Federico Simonetta, Susanne Stampf, Jürg Steiger, Guido Stirnimann, Ueli Stürzinger, Christian Van Delden, Jean-Pierre Venetz, Jean Villard, Julien Vionnet, Madeleine Wick, Markus Wilhelm, Patrick Yerly.

References

1. 1. Polack FP, Thomas SJ, Kitchin N, Absalon J, Gurtman A, Lockhart S, et al. Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine. N Engl J Med. 2020;383(27):2603–15. Epub 2020/12/11. pmid:33301246; PubMed Central PMCID: PMC7745181.

* View Article

* PubMed/NCBI

* Google Scholar

2. 2. Baden LR, El Sahly HM, Essink B, Kotloff K, Frey S, Novak R, et al. Efficacy and Safety of the mRNA-1273 SARS-CoV-2 Vaccine. N Engl J Med. 2021;384(5):403–16. Epub 2020/12/31. pmid:33378609; PubMed Central PMCID: PMC7787219.

* View Article

* PubMed/NCBI

* Google Scholar

3. 3. Speich B, Chammartin F, Smith D, Stoeckle MP, Amico P, Eichenberger AL, et al. A trial platform to assess approved SARS-CoV-2 vaccines in immunocompromised patients: first sub-protocol for a pilot trial comparing the mRNA vaccines Comirnaty(R) and COVID-19 mRNA Vaccine Moderna(R). Trials. 2021;22(1):724. Epub 2021/10/23. pmid:34674742; PubMed Central PMCID: PMC8529365.

* View Article

* PubMed/NCBI

* Google Scholar

4. 4. Scherrer AU, Traytel A, Braun DL, Calmy A, Battegay M, Cavassini M, et al. Cohort Profile Update: The Swiss HIV Cohort Study (SHCS). Int J Epidemiol. 2022;51(1):33–4j. Epub 2021/08/08. pmid:34363666.

* View Article

* PubMed/NCBI

* Google Scholar

5. 5. Koller MT, van Delden C, Muller NJ, Baumann P, Lovis C, Marti HP, et al. Design and methodology of the Swiss Transplant Cohort Study (STCS): a comprehensive prospective nationwide long-term follow-up cohort. Eur J Epidemiol. 2013;28(4):347–55. Epub 2013/04/03. pmid:23546766; PubMed Central PMCID: PMC3653038.

* View Article

* PubMed/NCBI

* Google Scholar

6. 6. Speich B, Chammartin F, Abela IA, Amico P, Stoeckle MP, Eichenberger AL, et al. Antibody response in immunocompromised patients after the administration of SARS-CoV-2 vaccine BNT162b2 or mRNA-1273: A randomised controlled trial. Clin Infect Dis. 2022. Epub 2022/03/03. pmid:35234868; PubMed Central PMCID: PMC8903480.

* View Article

* PubMed/NCBI

* Google Scholar

7. 7. Griessbach A, Chammartin F, Abela IA, Amico P, Stoeckle MP, Eichenberger AL, et al. Antibody response after the third SARS-CoV-2 vaccine in solid organ transplant recipients and people living with HIV (COVERALL-2). Open Forum Infectious Diseases. 2023. pmid:38023564

* View Article

* PubMed/NCBI

* Google Scholar

8. 8. Moss PA-O. The T cell immune response against SARS-CoV-2. Nat Immunol. 2022;(1529–2916).

* View Article

* Google Scholar

9. 9. Hermens JM, Kesmir CA-O. Role of T cells in severe COVID-19 disease, protection, and long term immunity. Immunogenetics. 2023;(1432–1211).

* View Article

* Google Scholar

10. 10. Wherry EJ, Barouch DH. T cell immunity to COVID-19 vaccines. Science. 2022;377(6608):821–2. Epub 2022/08/19. pmid:35981045.

* View Article

* PubMed/NCBI

* Google Scholar

11. 11. Almendro-Vazquez P, Laguna-Goya R, Paz-Artal E. Defending against SARS-CoV-2: The T cell perspective. Front Immunol. 2023;14:1107803. Epub 2023/02/14. pmid:36776863; PubMed Central PMCID: PMC9911802.

* View Article

* PubMed/NCBI

* Google Scholar

12. 12. Amstutz A, Chammartin F, Audige A, Eichenberger AL, Braun DL, Amico P, et al. Antibody and T-cell response to bivalent booster SARS-CoV-2 vaccines in people with compromised immune function (COVERALL-3). J Infect Dis. 2024. Epub 2024/06/07. pmid:38848312.

* View Article

* PubMed/NCBI

* Google Scholar

13. 13. McHugh ML. Interrater reliability: the kappa statistic. 2012;(1330–0962 (Print)).

* View Article

* Google Scholar

14. 14. Tang W, Hu J, Zhang H, Wu P, He H. Kappa coefficient: a popular measure of rater agreement. Shanghai Arch Psychiatry. 2015;27(1):62–7. Epub 2015/04/09. pmid:25852260; PubMed Central PMCID: PMC4372765.

* View Article

* PubMed/NCBI

* Google Scholar

15. 15. Carretero D, Gimenez E, Albert E, Colomer E, Montomoli M, Hernani R, et al. Performance of the Roche Elecsys(R) IGRA SARS-CoV-2 test for the detection and quantification of virus-reactive T cells in COVID-19-vaccinated immunosuppressed patients and healthy subjects. Eur J Clin Microbiol Infect Dis. 2024. Epub 2024/05/23. pmid:38777919.

* View Article

* PubMed/NCBI

* Google Scholar

16. 16. Saad Albichr I, Mzougui S, Devresse A, Georgery H, Goffin E, Kanaan N, et al. Evaluation of a commercial interferon-gamma release assay for the detection of SARS-CoV-2 T-cell response after vaccination. Heliyon. 2023;9(6):e17186. Epub 2023/06/16. pmid:37325456; PubMed Central PMCID: PMC10256590.

* View Article

* PubMed/NCBI

* Google Scholar

17. 17. Derksen BM, Bruinsma W, Goslings JC, Schep NWL. The Kappa Paradox Explained. J Hand Surg Am. 2024;49(5):482–5. Epub 2024/02/19. pmid:38372689.

* View Article

* PubMed/NCBI

* Google Scholar

18. 18. Seo YJ, Oh I, Nam M, Shin S, Roh EY, Song EY. Comparison of Four T-cell Assays and Two Binding Antibody Assays in SARS-CoV-2 Vaccinees With or Without Omicron Breakthrough Infection. Ann Lab Med. 2023;43(6):596–604. Epub 2023/06/30. pmid:37387492; PubMed Central PMCID: PMC10345181.

* View Article

* PubMed/NCBI

* Google Scholar

19. 19. Huzly D, Panning M, Smely F, Enders M, Komp J, Falcone V, et al. Accuracy and real life performance of a novel interferon-γ release assay for the detection of SARS-CoV2 specific T cell response. J Clin Virol. 2022;148:105098. Epub 2022/02/09. pmid:35134681; PubMed Central PMCID: PMC8806708.

* View Article

* PubMed/NCBI

* Google Scholar

20. 20. Fernández-González M, Agulló V, Padilla S, García JA, García-Abellán J, Botella Á, et al. Clinical Performance of a Standardized Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) Interferon-γ Release Assay for Simple Detection of T-Cell Responses After Infection or Vaccination. Clin Infect Dis. 2022;75(1):e338–e46. Epub 2021/12/12. pmid:34893816; PubMed Central PMCID: PMC8689778.

* View Article

* PubMed/NCBI

* Google Scholar

Citation: Audigé A, Amstutz A, Schuurmans MM, Amico P, Braun DL, Stoeckle MP, et al. (2025) Low agreement and frequent invalid controls in two SARS-CoV-2 T-cell assays in people with compromised immune function. PLoS ONE 20(1): e0317965. https://doi.org/10.1371/journal.pone.0317965

About the Authors:

Annette Audigé

† Deceased.

Roles: Conceptualization, Data curation, Resources, Writing – original draft

Affiliation: Institute of Medical Virology, University of Zurich, Zurich, Switzerland

Alain Amstutz

Roles: Conceptualization, Methodology, Writing – review & editing

Affiliations: Division of Clinical Epidemiology, Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland, Oslo Center for Biostatistics and Epidemiology (OCBE), Oslo University Hospital, University of Oslo, Oslo, Norway, Bristol Medical School, University of Bristol, Bristol, United Kingdom

Macé M. Schuurmans

Roles: Data curation, Writing – review & editing

Affiliation: Division of Pulmonology, University Hospital Zurich, Zurich, Switzerland

Patrizia Amico

Roles: Data curation, Writing – review & editing

Affiliation: Clinic for Transplantation Immunology and Nephrology, University Hospital Basel, Basel, Switzerland

Dominique L. Braun

Roles: Data curation, Writing – review & editing

Affiliations: Institute of Medical Virology, University of Zurich, Zurich, Switzerland, Department of Infectious Diseases and Hospital Epidemiology, University Hospital Zurich, Zurich, Switzerland

Marcel P. Stoeckle

Roles: Data curation, Writing – review & editing

Affiliation: Division of Infectious Diseases and Hospital Epidemiology, University Hospital Basel, University of Basel, Basel, Switzerland

Barbara Hasse

Roles: Data curation, Writing – review & editing

Affiliation: Department of Infectious Diseases and Hospital Epidemiology, University Hospital Zurich, Zurich, Switzerland

René Hage

Roles: Data curation, Writing – review & editing

Affiliation: Division of Pulmonology, University Hospital Zurich, Zurich, Switzerland

ORICD: https://orcid.org/0000-0002-1994-7443

Dominik Damm

Roles: Data curation, Writing – review & editing

Affiliation: Division of Pulmonology, University Hospital Zurich, Zurich, Switzerland

Michael Tamm

Roles: Data curation, Writing – review & editing

Affiliation: Clinic of Respiratory Medicine and Pulmonary Cell Research, University Hospital Basel, Basel, Switzerland

Nicolas J. Mueller

Roles: Data curation, Writing – review & editing

Affiliation: Department of Infectious Diseases and Hospital Epidemiology, University Hospital Zurich, Zurich, Switzerland

ORICD: https://orcid.org/0000-0002-1059-3191

Huldrych F. Günthard

Roles: Data curation, Writing – review & editing

Affiliations: Institute of Medical Virology, University of Zurich, Zurich, Switzerland, Department of Infectious Diseases and Hospital Epidemiology, University Hospital Zurich, Zurich, Switzerland

ORICD: https://orcid.org/0000-0002-1142-6723

Michael T. Koller

Roles: Data curation, Writing – review & editing

Affiliations: Clinic for Transplantation Immunology and Nephrology, University Hospital Basel, Basel, Switzerland, Swiss Transplant Cohort Study, University Hospital Basel, Basel, Switzerland

Christof M. Schönenberger

Roles: Conceptualization, Visualization, Writing – review & editing

Affiliation: Division of Clinical Epidemiology, Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland

Alexandra Griessbach

Roles: Conceptualization, Project administration, Writing – review & editing

Affiliation: Division of Clinical Epidemiology, Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland

Niklaus D. Labhardt

Roles: Conceptualization, Writing – review & editing

Affiliation: Division of Clinical Epidemiology, Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland

Roger D. Kouyos

Roles: Project administration, Writing – review & editing

Affiliations: Institute of Medical Virology, University of Zurich, Zurich, Switzerland, Department of Infectious Diseases and Hospital Epidemiology, University Hospital Zurich, Zurich, Switzerland

Alexandra Trkola

Roles: Visualization, Writing – review & editing

Affiliation: Institute of Medical Virology, University of Zurich, Zurich, Switzerland

Michael Huber

Roles: Formal analysis, Visualization, Writing – review & editing

Affiliation: Institute of Medical Virology, University of Zurich, Zurich, Switzerland

ORICD: https://orcid.org/0000-0002-0384-0000

Katharina Kusejko

Roles: Project administration, Resources, Software, Writing – review & editing

Affiliations: Institute of Medical Virology, University of Zurich, Zurich, Switzerland, Department of Infectious Diseases and Hospital Epidemiology, University Hospital Zurich, Zurich, Switzerland

Heiner C. Bucher

Roles: Conceptualization, Writing – review & editing

Affiliation: Division of Clinical Epidemiology, Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland

Irene A. Abela

Roles: Conceptualization, Validation, Visualization, Writing – review & editing

Affiliations: Institute of Medical Virology, University of Zurich, Zurich, Switzerland, Department of Infectious Diseases and Hospital Epidemiology, University Hospital Zurich, Zurich, Switzerland

Matthias Briel

Roles: Conceptualization, Supervision, Writing – review & editing

Affiliations: Division of Clinical Epidemiology, Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland, Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Canada

ORICD: https://orcid.org/0000-0002-2070-5230

Frédérique Chammartin

Roles: Conceptualization, Formal analysis, Validation, Visualization, Writing – review & editing

Affiliation: Division of Clinical Epidemiology, Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland

Benjamin Speich

Roles: Conceptualization, Formal analysis, Methodology, Project administration, Visualization, Writing – original draft

E-mail: [email protected]

Affiliation: Division of Clinical Epidemiology, Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland

ORICD: https://orcid.org/0000-0002-3301-8085

the Swiss HIV Cohort Study, and the Swiss Transplant Cohort Study

¶Membership of The Swiss HIV Cohort Study, and the Swiss Transplant Cohort Study is provided in the Acknowledgments.

References

1. Polack FP, Thomas SJ, Kitchin N, Absalon J, Gurtman A, Lockhart S, et al. Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine. N Engl J Med. 2020;383(27):2603–15. Epub 2020/12/11. pmid:33301246; PubMed Central PMCID: PMC7745181.

2. Baden LR, El Sahly HM, Essink B, Kotloff K, Frey S, Novak R, et al. Efficacy and Safety of the mRNA-1273 SARS-CoV-2 Vaccine. N Engl J Med. 2021;384(5):403–16. Epub 2020/12/31. pmid:33378609; PubMed Central PMCID: PMC7787219.

3. Speich B, Chammartin F, Smith D, Stoeckle MP, Amico P, Eichenberger AL, et al. A trial platform to assess approved SARS-CoV-2 vaccines in immunocompromised patients: first sub-protocol for a pilot trial comparing the mRNA vaccines Comirnaty(R) and COVID-19 mRNA Vaccine Moderna(R). Trials. 2021;22(1):724. Epub 2021/10/23. pmid:34674742; PubMed Central PMCID: PMC8529365.

4. Scherrer AU, Traytel A, Braun DL, Calmy A, Battegay M, Cavassini M, et al. Cohort Profile Update: The Swiss HIV Cohort Study (SHCS). Int J Epidemiol. 2022;51(1):33–4j. Epub 2021/08/08. pmid:34363666.

5. Koller MT, van Delden C, Muller NJ, Baumann P, Lovis C, Marti HP, et al. Design and methodology of the Swiss Transplant Cohort Study (STCS): a comprehensive prospective nationwide long-term follow-up cohort. Eur J Epidemiol. 2013;28(4):347–55. Epub 2013/04/03. pmid:23546766; PubMed Central PMCID: PMC3653038.

6. Speich B, Chammartin F, Abela IA, Amico P, Stoeckle MP, Eichenberger AL, et al. Antibody response in immunocompromised patients after the administration of SARS-CoV-2 vaccine BNT162b2 or mRNA-1273: A randomised controlled trial. Clin Infect Dis. 2022. Epub 2022/03/03. pmid:35234868; PubMed Central PMCID: PMC8903480.

7. Griessbach A, Chammartin F, Abela IA, Amico P, Stoeckle MP, Eichenberger AL, et al. Antibody response after the third SARS-CoV-2 vaccine in solid organ transplant recipients and people living with HIV (COVERALL-2). Open Forum Infectious Diseases. 2023. pmid:38023564

8. Moss PA-O. The T cell immune response against SARS-CoV-2. Nat Immunol. 2022;(1529–2916).

9. Hermens JM, Kesmir CA-O. Role of T cells in severe COVID-19 disease, protection, and long term immunity. Immunogenetics. 2023;(1432–1211).

10. Wherry EJ, Barouch DH. T cell immunity to COVID-19 vaccines. Science. 2022;377(6608):821–2. Epub 2022/08/19. pmid:35981045.

11. Almendro-Vazquez P, Laguna-Goya R, Paz-Artal E. Defending against SARS-CoV-2: The T cell perspective. Front Immunol. 2023;14:1107803. Epub 2023/02/14. pmid:36776863; PubMed Central PMCID: PMC9911802.

12. Amstutz A, Chammartin F, Audige A, Eichenberger AL, Braun DL, Amico P, et al. Antibody and T-cell response to bivalent booster SARS-CoV-2 vaccines in people with compromised immune function (COVERALL-3). J Infect Dis. 2024. Epub 2024/06/07. pmid:38848312.

13. McHugh ML. Interrater reliability: the kappa statistic. 2012;(1330–0962 (Print)).

14. Tang W, Hu J, Zhang H, Wu P, He H. Kappa coefficient: a popular measure of rater agreement. Shanghai Arch Psychiatry. 2015;27(1):62–7. Epub 2015/04/09. pmid:25852260; PubMed Central PMCID: PMC4372765.

15. Carretero D, Gimenez E, Albert E, Colomer E, Montomoli M, Hernani R, et al. Performance of the Roche Elecsys(R) IGRA SARS-CoV-2 test for the detection and quantification of virus-reactive T cells in COVID-19-vaccinated immunosuppressed patients and healthy subjects. Eur J Clin Microbiol Infect Dis. 2024. Epub 2024/05/23. pmid:38777919.

16. Saad Albichr I, Mzougui S, Devresse A, Georgery H, Goffin E, Kanaan N, et al. Evaluation of a commercial interferon-gamma release assay for the detection of SARS-CoV-2 T-cell response after vaccination. Heliyon. 2023;9(6):e17186. Epub 2023/06/16. pmid:37325456; PubMed Central PMCID: PMC10256590.

17. Derksen BM, Bruinsma W, Goslings JC, Schep NWL. The Kappa Paradox Explained. J Hand Surg Am. 2024;49(5):482–5. Epub 2024/02/19. pmid:38372689.

18. Seo YJ, Oh I, Nam M, Shin S, Roh EY, Song EY. Comparison of Four T-cell Assays and Two Binding Antibody Assays in SARS-CoV-2 Vaccinees With or Without Omicron Breakthrough Infection. Ann Lab Med. 2023;43(6):596–604. Epub 2023/06/30. pmid:37387492; PubMed Central PMCID: PMC10345181.

19. Huzly D, Panning M, Smely F, Enders M, Komp J, Falcone V, et al. Accuracy and real life performance of a novel interferon-γ release assay for the detection of SARS-CoV2 specific T cell response. J Clin Virol. 2022;148:105098. Epub 2022/02/09. pmid:35134681; PubMed Central PMCID: PMC8806708.

20. Fernández-González M, Agulló V, Padilla S, García JA, García-Abellán J, Botella Á, et al. Clinical Performance of a Standardized Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) Interferon-γ Release Assay for Simple Detection of T-Cell Responses After Infection or Vaccination. Clin Infect Dis. 2022;75(1):e338–e46. Epub 2021/12/12. pmid:34893816; PubMed Central PMCID: PMC8689778.