Strategies for method comparison when changes in the immunogenicity method are needed within a clinical program

Abstract

Aim: To present the reader with different approaches used to compare immunogenicity methods when changes are needed during a clinical program. Results: Five case studies are presented, in the first two case studies, the approach utilized a small sample size for the comparison. In the third case, all samples from a study were analyzed by both methods. In the fourth case, the intended use of noncomparable assays in an integrated summary drove design of experiments to establish the expected limits of pooling data. In the fifth case, a selectivity approach was used as an alternate to use of incurred samples. Conclusion: When data pooling across methods is needed, it is important to define the limits of comparability.

During the drug development program of a biotherapeutic, there may be several reasons to justify a change in the bioanalytical methods used to support the program. In terms of immunogenicity methods, the reasons may include: the technology platform was discontinued, the technologies available at the time when the original method was developed, did not provide adequate sensitivity or drug tolerance, or the original method is not adequate to support new patient populations. Anti-drug antibody (ADA) methods must be sensitive, drug tolerant and reproducible in order to ensure adequate monitoring of patient safety and to determine if ADAs are impacting PK and drug efficacy. Therefore, within the context of the program's immunogenicity risk assessment, the strategy may justify the development of a new method or modifications to the current method for improvements. According to the FDA's guidance from 2019 (1), the sponsor should establish that the data is comparable between laboratories used to support bioanalysis; however, there is no mention of comparability needed across methods. The EMA guidance only discusses the comparability in terms of immunogenicity studies focus on assessments for biosimilars and in cases of manufacturing changes (2). Both guidances recommend the presentation of an integrated summary of immunogenicity as part of the sponsor's marketing authorization and/or biologics license applications, and sponsors should consider information on method comparability or lack thereof when preparing the summaries. Also, there may be cases where the sponsor is interested in pooling immunogenicity data across studies to help assess the overall impact of immunogenicity on PK, PD and safety and ultimately appropriate drug product label reporting (3). In addition, there may be an interest in comparing immunogenicity across patient populations, age groups and other factors, and although some of these comparisons may be done within the study design of a clinical protocol to seek a particular label approval, there may be larger datasets from other studies that could provide additional insights regarding factors that impact immunogenicity. When interested in analyzing data pooled across studies, which utilized different immunogenicity methods, sponsors should devise a bioanalytical strategy to assess if the methods are comparable or if they need to be analyzed as separate populations. One may categorize these strategies into different buckets:

• Rigorous comparison: driven by the immunogenicity risk assessment of the program or health authority feedback. Requires the use of a large number of samples for the comparison. Case #3 is a good example of this category, as well as the case previously presented by Qiu and colleagues (4).
• Simple/abbreviated comparison: immunogenicity poses a low risk to the patient population. Data may be summarized across studies, but pooling of datasets generated with different methods is not needed.
• Alternative comparison: comparisons using incurred samples may not be operationally feasible or needed.

Here, we present case studies for each of the above categories. Each of the methods in this manuscript was developed specifically for the drug and validated according to current best practices. This manuscript focuses on the comparability of results between fully validated methods depending on how important it was to pool data from the integrated summary versus keeping datasets separate.

Each sponsor took into consideration the immunogenicity risk assessment of the program (5,6), health authority (HA) feedback and operational constraints when planning the way in which to conduct the assessment.

Table1 provides a high-level overview of all case studies and the strategies utilized for the comparison. In the introduction, we elaborate on the background information for each biotherapeutic, while keeping the identity blinded as required by the respective contributing companies. A more detailed description of the methods and the comparison strategy is included in the experimental section. It is out of the scope of this manuscript to present the immunogenicity risk assessment for each program, instead Table1, highlights the main parameters: therapeutic type, target population and ADA impact on PK/PD/safety which influence the strategy for method comparison (immunogenicity incidence was intentionally excluded). It is worth noting that all individual incurred and spiked samples were assigned new IDs to blind the analyst for the purpose of the method comparison.

Table1. Summary of case studies and their method comparison strategy.

Case study	Biotherapeutic type and target population	ADA impact on PK/PD/safety	Method	Strategy for comparison	Same lab	New method comparable?	Strategy for sample and data analysis
1	Humanized monoclonal, multiple myeloma (drug A)	Confounded effect on PK but incidence<20%	ADA	16 incurred samples	No	100% agreement for previously reported (+)	New studies would utilize the new method. CDT summarized IMG data across studies (also per study)
2	Humanized monoclonal, multiple myeloma (drug A)	None on safety, no clear association with clearance	NAb	30 incurred samples	No	≥90% agreement for previously reported (+)	Data are not compared across studies. Use assay as a subgroup in integrated analysis
3	Fusion protein, immunology (drug B)	None	ADA	743 incurred samples. Driven by a health authority request	Yes	No, comparability was not expected due to format change	Complete ongoing analysis with original method. New studies would use new method
4	Fully human monoclonal, oncology (drug C)	None	ADA	Two sets: for titer (164) and concordance (723) evaluation	No	N/A comparability not expected due to format change	Use assay to define analysis sets rather than pooling in integrated analysis
5	Fully human monoclonal, oncology (drug C)	None	ADA	Incurred sample testing not feasible. Spiked PCs (12) and mock individual samples (50) were tested as alternate	No	Partially	Pooled analysis dataset when assessing positive/negative, separate subsets for all statistical analyses using titer values

ADA: Anti-drug antibody; NAb: Neutralizing antibody.

Case studies with simple/abbreviated comparison

Case #1 and #2 are about a humanized monoclonal therapeutic approved for the treatment of multiple myeloma, with overall low immunogenicity incidence and low risk to the patient population in terms of safety. The original anti-drug antibody (ADA) method and the neutralizing antibody (NAb) method (METHOD-1 and METHOD-3, respectively) had a Bead Extraction with Acid Dissociation (BEAD) sample pretreatment to achieve the needed drug tolerance based on the expected drug levels at the assigned collection time points (trough) for the samples. The validation of methods utilized pretreatment samples from multiple myeloma patients to determine the plate cut point. During validation, it was noticed that approximately 20% of the samples gelatinized during the BEAD step, rendering them not evaluable in the method. This prompted the need for the development of a new sample pretreatment that used heat instead of acid for the ADA-drug dissociation in the screening and confirmatory methods (METHOD-2) as well as for the neutralizing cell-based method (METHOD-4).

Case study with rigorous comparison

Case #3 is a fusion protein that is currently approved for an immunology indication. The method comparison was done prior to approval. The drug consists of the extracellular domain of a human protein (further referred to as the ‘drug tip’) fused with the hinge and CH2-CH3 domains of human immunoglobulin (Ig). During the early stages of the development of the drug, two direct-format enzyme-linked immunosorbent assays (ELISAs) were developed. The first assay, METHOD-5, measured the whole molecule, in other words, both the drug tip and Ig portion. The second assay, METHOD-6, measured the antibody response to the drug tip only. As the immunogenicity field evolved and new technologies became available, a more sensitive, drug-tolerant bridging Meso Scale Discovery Electrochemiluminescence (MSD-ECL) assay (METHOD-7) was developed. The goal was to replace the original two assays with the MSD-ECL assay, as the new assay was capable of identifying immune response to the various components of the drug in a single assay with superior drug tolerance. Due to a health authority request, study samples were analyzed in all assays to demonstrate the superiority of METHOD-7.

Case studies with alternative comparison

Case #4 is a fully human monoclonal therapeutic in development for oncology. The observed C_trough in early trials was higher than predicted, so additional drug tolerance was preferred along with improved ruggedness. The assay format implemented for the Phase III studies switched from a step-wise bridging format to a homogenous bridging format. Although validation characteristics were similar using the surrogate positive control, comparability was not expected due to anticipated improved drug tolerance and selectivity of lower affinity responses. To proactively set the caveats for cross-study comparisons, positive results from one study were titered in the new assay to establish boundaries of combining data. Subsequently, the sponsor proactively retested incurred samples from a small population, to enable direct comparison to the main dataset. After the completion of this work, the health authority requested retesting from two additional small populations for the same reason (data not shown).

Case #5 is the same human monoclonal therapeutic as Case #4, with a focus on a direct transfer of the Phase III ADA assay to China. The clinical development program includes clinical sites in China, both in dedicated China studies and global studies. Due to shifting regulations for sample export from China, all samples collected in China were tested in China and all other countries were tested at the US facility. Due to recent changes in regulations, it was not feasible to retrieve samples shipped into China for incurred sample reanalysis, a modified cross-validation approach was taken using selectivity samples rather than incurred samples. Although limited by the positive control, this approach for a straight method transfer provided additional confidence in the method transfer, above pooled PC experiments alone.

Platform comparisons and evaluations (7-9) in the context of method development and investigation of nonclinical and clinical outcomes are an important part of the overall bioanalytical strategy, however, not the scope of this manuscript.

Experimental

Case #1

For Case #1, the ADA method was updated to eliminate the issue with some samples not performing well during treatment with acid and having to be reported as nonevaluable. Table2 provides a summary on both immunogenicity methods. Additional details are provided below.

Table2. Summary of bioanalytical parameters for the immunogenicity methods case #1.

Validated method	METHOD-1	METHOD-2
Positive control	Anti-drugA monkey IgG	Anti-drugA monkey IgG
Sensitivity	6.19ng/ml	5.86ng/ml
Drug tolerance (250ng/ml anti-drug A antibody PC)	400μg/ml	800μg/ml

Original immunogenicity method (METHOD-1)

This method utilizes MSD-ECL technology in which carbon electrodes integrated into the bottom of an assay plate excite a ruthenium label, emitting light at 620nm which is then read by the MSD Sector Imager. Samples are pretreated with acid and an excess of biotinylated drug, facilitating the dissociation of circulating drug from specific anti-drug antibodies. A base is then added to neutralize the solution, allowing for re-association of these anti-drug antibodies to the now overabundant biotinylated drug, aiding in their extraction. The samples are mixed with pre-blocked streptavidin-coated magnetic beads and exposed to a magnetic field to extract the biotin-drug-antibody complexes from the samples. After washing the beads, the samples are then treated with acid to release the antibody from the biotinylated drug. The extracted sample is incubated with a master mix containing both biotinylated- and ruthenylated drug, allowing any anti-drug antibodies present in the sample to bind both labeled forms of the drug in a bridging complex. The complex is then captured on streptavidin-coated MSD microtiter plates via the biotinylated portion of the complex. Upon addition of MSD read buffer and excitation of the ruthenium label, light is emitted relative to the amount of anti-drug antibody present in the sample.

Second-generation immunogenicity method (METHOD-2)

This method utilizes the same MSD-ECL technology from the original method. Instead of the BEAD pretreatment used to improve drug tolerance, a pretreatment step was performed with the KingFisher™ Flex using heat instead of acid to dissociate drug bound to anti-drug antibodies. Samples are first incubated with biotinylated drugA (biotin-drugA) at 37°C followed by an incubation at room temperature. ADAs, dissociated from drugA during the 37°C incubation, bind to biotin-drug A. DrugA preblocked magnetic streptavidin beads are then added in the mixture to capture this complex and then washed on the KingFisher Flex. The remaining steps are the same as in the original method (see above).

Method comparison strategy

A total of 16 incurred samples (6 high titer and 10 medium titer), previously analyzed by the original method (METHOD-1) were selected for the method comparison. The confirmed positive and titered samples were grouped into four quantiles based on their titers, and the 16 medium and high titer samples were randomly chosen from the second to the forth quantiles. Samples with low titers from the first quantile were not selected for comparison purpose, since their ADA results are statistically likely to fluctuate near the cut point. These samples were analyzed along with PCs (with and without drug). At the time, standard operating procedures did not cover ADA method comparisons. Therefore, the number of incurred samples analyzed was based on discussions with a statistician and considering the fact that only the sample pre-treatment had been changed.

Case #2

As previously mentioned, Case #2 covers the same program as Case#1, but for the neutralizing antibody assay, for which the original assay had the same issues with samples turning to gel during treatment with acid. Table3 provides a summary on both NAb methods. Additional details are provided below.

Table3. Summary of bioanalytical parameters for the neutralizing antibody methods case #2.

Validated method	METHOD-3	METHOD-4
Positive control	Monkey polyclonal anti-drug A	Monkey polyclonal anti-drug A
Sensitivity (μg/ml)	3.04	3.12
Drug tolerance	NAb PC at 15μg/ml can tolerate 424μg/ml of drug	NAb PC at 5μg/ml can tolerate 400μg/ml of drug

NAb: Neutralizing antibody.

Two-cell system neutralizing antibody assay

A JurkaT-cell line (T cell) was engineered to overexpress CD16a receptor and transfected with a reporter plasmid. The reporter plasmid contains a synthetic nuclear factor of activated T cells (NFAT) promoter that controls the expression of the firefly luciferase gene. NFAT is a key transcription factor and downstream target of CD16a receptor signaling. When drug was added to the target cell, which expresses the target of the drug and the effector cell, the drug induced a dose-dependent T-cell activation and thus an increase in luciferase production. If testing samples contained NAbs against the drug that either blocked binding to its target on target cells or blocked effector function, the drug could no longer elicit a response in this cell-based functional assay.

Table4. Summary of bioanalytical parameters for the anti-drug antibody methods case #3.

Validated method	METHOD-5	METHOD-6	METHOD-7
Positive control	Monkey polyclonal anti-drug B	Monkey polyclonal anti-drugB	Monkey polyclonal anti-drugB
Sample dilution	400-fold	25-fold	10-fold
Confirmation	Whole drug, drug tip, nonrelated Ig, ovalbumin	Whole drug, drug tip, ovalbumin	Whole drug, drug tip
Sensitivity	∼20μg/ml	275ng/ml	12ng/ml
Drug tolerance	Significant drug interference	∼500ng/ml Ab PC at ∼1µg/ml of drug 975ng/ml of Ab PC at 5µg/ml of drug	250ng/ml of Ab PC at 40μg/ml of drug

Ab: Antibody; Ig: Immunoglobin.

Original neutralizing antibody functional cell-based method (METHOD-3)

To reduce potential drug interference, a sample pre-treatment step was performed. Human serum samples and controls were first incubated with acid (1:1), followed by an incubation at room temperature (RT) with biotinylated drug prepared in base to neutralize the solution. ADAs, dissociated from the drug bind to biotin-drug and were then immobilized on streptavidin-coated magnetic beads. Bead complexes were captured using a magnet and washed. Bound ADAs were released from the bead complex by a second acid treatment. The acid solution, containing ADAs, was neutralized. This ADA containing elute was then incubated with drug and analyzed in a bioassay that utilized the two-cell system mentioned above.

Second generation neutralizing antibody functional cell based method (METHOD-4)

During method validation, which used predosed multiple myeloma (MM) samples to establish the cut point, it was noticed that about 20% of MM samples turned into a gel after the bead addition and could not be evaluated for NAb. During the BEAD step, following overnight incubation of streptavidin-coated magnetic beads, no clear pellet of the Sera-Mag SpeedBeads was visually formed when the plate was placed on a magnetic block. To resolve this nonevaluable sample issue, a heat dissociation step was incorporated into the BEAD extraction. The BEAD extraction was performed with the KingFisher™ Flex. Positive controls and samples were first incubated with biotinylated drug (biotin-drugA) at 37°C followed by an incubation at room temperature. ADAs, dissociated from the bound drug, bind to biotin-drug and preblocked magnetic streptavidin beads are added in the mixture. Bead complexes were then separated and washed. Bound ADAs are released from the bead complex by an acid treatment. The acid solution, containing ADAs, was then neutralized and analyzed in the two-cell bioassay system mentioned above.

Method comparison strategy

Twenty-eight (28) blinded samples consisting of negative and positive anti-drugA neutralizing antibodies were randomly selected and then evaluated. The data generated for the comparison was from 1 run. See case #1 for rationale on the number of samples.

Case #3

Original immunogenicity methods: (METHOD-5 and METHOD-6)

Two direct-format ELISAs were previously validated. METHOD-5 measured the antibody response to the whole molecule, in other words, both the drug tip and Ig portion of the drug. For this method, the wells of a 96-well microtiter plate were coated with drug to capture drug-specific antibodies in the sample. These were then detected using a horseradish peroxidase conjugated anti-human IgG kappa and lambda light chain specific antibody. If the samples screened positive, confirmation was performed using the following: the whole drug, drug tip, unrelated Ig and ovalbumin. The second method, METHOD-6, measured the antibody response against the tip of the drug. For this method, the wells of a 96-well microtiter plate were coated with the drug tip to capture antibodies specific to the drug tip. The antibodies were then detected using a horseradish peroxidase conjugated goat anti-human IgG heavy and light chain specific antibody. If the samples screened positive, confirmation was performed using the following: whole drug, drug tip and ovalbumin.

Second-generation immunogenicity method (METHOD-7)

The newly developed MSD-ECL assay, METHOD-7, was capable of detecting antibodies directed against both the drug tip and the Ig portion of the drug in the same assay. In this method, samples were pretreated with acid to allow the complexes to dissociate, the samples were then neutralized and added to a mixture of biotin-labeled drug and ruthenium-labeled drug, subsequently forming a bridge. The mixture was then added to a streptavidin-coated MSD plate. If the samples screened positive, confirmation was performed using the following: whole drug and the drug tip. Table4 provides a summary of the main parameters for all methods.

Method comparison strategy

Due to a health authority request, incurred samples were analyzed in all three assays to assess comparability. Serum samples from 249 disease state subjects taken during the first year of treatment were used. In the ELISAs, METHOD-5 and Assay B METHOD-6, 795 and 797 samples were analyzed, respectively. In the MSD-ECL assay, METHOD-7, 743 samples were analyzed.

Case #4

For Case #4, the ADA method was updated to improve the method ruggedness. Table5 provides a summary on both immunogenicity methods, which were considered essentially equivalent based solely on validation data. Since low and high affinity clones are typically under-represented in the affinity-purified surrogate positive control, the change from a step-wise bridging format to a homogenous format was hypothesized to further improve detection of low or high affinity antibodies in actual patient samples.

Table5. Key validation parameters for the immunogenicity methods in case study4.

Validated method	METHOD-8	METHOD-9
Laboratory	Vendor A	Vendor B
Analyte	Anti-drugC antibodies	Anti-drugC antibodies
Assay format	Step-wise bridging	Homogeneous bridging
Positive control	Affinity-purified rabbit polyclonal	Affinity-purified rabbit polyclonal
Sensitivity(ng/ml)	5.12	2.3
Drug tolerance (250ng/ml anti-drugC antibody PC)	125μg/ml†	100μg/ml†

†Note that while numerically different, these lower bounds of drug tolerance are essentially the same as the difference is due to the starting point of the two-fold dilution scheme. Experiments were not completed to assess more precisely.

Original immunogenicity method: (METHOD-8)

METHOD-8 utilizes MSD-ECL technology in a step-wise format. Biotinylated drugC is captured on streptavidin-coated plates. Samples are pretreated with acid, facilitating the dissociation of circulating drugC from specific anti-drugC antibodies. A base is then added to neutralize the solution, and then incubated with the coated plates to capture the anti-drugC antibodies. Ruthenium-labeled drugC is incubated to bind the captured analyte, and then quantified by ECL detection.

Second generation immunogenicity method (METHOD-9)

METHOD-9 utilizes MSD-ECL technology in a homogenous format. Samples are pretreated with acid, facilitating the dissociation of circulating drugC from specific anti-drugC antibodies. A base is then added to neutralize the solution, and samples are mixed with biotinylated drugC and ruthenium-labeled drugC. The mixtures are then incubated in streptavidin-coated plates, where the complexes formed in solution between anti-drugC antibodies and the labeled drugC reagents are captured and then quantified by ECL detection.

Method comparison strategy

In the first experiment with Study 1, a total of 176 incurred samples previously confirmed positive by METHOD-8 were selected for the titer comparison. These samples were titered in METHOD-9.

In the second experiment with Study 2, a total of 299 incurred samples previously tested in METHOD-8 and sufficient volume remaining were selected for the concordance evaluation. These samples were tested in METHOD-9 de novo following the prespecified three-tiered testing schematic. Please note that this study required a study-specific cut point factor in METHOD-8 while the validation cut point factor was acceptable for METHOD-9.

Case #5

For Case #5, the homogeneous bridging assay METHOD-9 from Case #4 was transferred to a second laboratory in China. Table6 provides a summary on both immunogenicity methods.

Table6. Key validation parameters for the immunogenicity methods in case study5.

Validated method	METHOD-9	METHOD-10
Laboratory	Vendor A (US)	Vendor B (China)
Analyte	Anti-drugC antibodies	Anti-drugC antibodies
Assay format	Homogeneous bridging	Homogeneous bridging
Positive control	Affinity-purified rabbit polyclonal	Affinity-purified rabbit polyclonal
Sensitivity(ng/ml)	2.3	10.0
Drug tolerance (250ng/ml anti-drugC antibody PC)	At least 100μg/ml of drug	At least 100μg/ml of drug

Method comparison strategy

Approximately 12 spiked PC samples were prepared in pooled matrix at four PC concentrations spanning the method range (3 samples each concentration). In addition, 50 mock samples were prepared in ten individual patient matrices at five PC concentrations spanning the expected study range. Samples were split into two aliquots, one set were analyzed at Vendor A and the other set at Vendor B. These samples were analyzed as if they were study samples.

Results and discussion

Case #1

In case #1, the method comparison was done to confirm that the improvements to the immunogenicity method would not negatively impact immunogenicity assessments and to justify pooling of data across studies. In order to assess method comparability, 16 samples of medium and high ADA titers were selected for the analysis. The results of this method comparison are summarized in Table7 (see Supplementary Table 1 for additional details) and verify that all previously reported ADA-positive samples with METHOD-1 were confirmed positive by METHOD-2. Since METHOD-2 has an improved drug tolerance compared with METHOD-1 (Table2), differences between methods may be observed for borderline ADA negative and positive samples, therefore negative samples were intentionally excluded for the method comparison. In hindsight, we recommend to include negatives. In fact, the ADA incidence (19/53=35.8%) increased in a study supported with Method-2 compared with a previous study in the same patient population (45/299=15.1%). The higher incidence may reflect the improvement in drug tolerance with METHOD-2 but it may also be a result of variability in the sample size (number of subjects).

Table7. Concordance of confirmed results between METHOD-1 and METHOD-2.

	METHOD-1 positive	METHOD-1 negative
METHOD-2 positive	16	None tested
METHOD-2 negative	0	None tested

METHOD-2 was validated at a different laboratory than METHOD-1, with plans to use METHOD-2 to support future studies.

Case #2

In case #2, method comparison results, shown in Table8 (detailed data in Supplementary Table 2), demonstrated that 92% (26/28) of the samples which tested NAb positive by the original method (METHOD-3), tested positive for Nab in METHOD-4. There are no available data at this time where the two different methods would have been used to support analysis in studies in the same patient population.

Table8. Concordance of confirmed results between METHOD-3 and METHOD-4.

	METHOD-3 positive	METHOD-3 negative
METHOD-4 positive	26	0
METHOD-4 negative	2	10

Case #3

The results of the method comparison are detailed in Table9. Out of the 249 subjects tested in METHOD-5, five samples (3 subjects) tested positive with specificity against the Ig. In METHOD-6, two samples (1 subject) tested positive with specificity against the drug tip. In METHOD-7, 24 samples (16 subjects) tested positive. Out of these, 17 samples (11 subjects) had antibodies against the drug tip and possibly Ig portions of the drug, and seven samples (6 subjects) had antibodies against the Ig and/or junction. Since METHOD-7 exhibited better sensitivity as well as drug tolerance, the higher incidence of positive responses seen in METHOD-7 (6.4%) versus METHOD-5 and METHOD-6 (1.6% combined) (Table 9) was expected. The comparison of the methods was submitted to and deemed acceptable by the requesting health authority agency. To support future studies, the more sensitive and more drug tolerant MSD-ECL assay (METHOD-7) was implemented.

Table9. Results from immunogenicity samples tested for case #3 method comparison.

Assay	Total positive subjects/subjects analyzed (incidence)	Subject specificity	Total positive (post-dose) samples	Sample specificity
METHOD-5 and METHOD 6	4/249 (1.6%)	3 - Ig 1 - drug tip	7	5 - Ig 2 - drug tip
METHOD-7	16/249 (6.4%)	11 - drug tip and possibly Ig 6 - Ig and/or junction	24	17 - drug tip and possibly Ig 7 - Ig and/or junction

Ig: Immunoglobin.

Case #4

Case Study 4 proactively compared results of two ADA assays for the same drug. METHOD-8 was a step-wise sandwich ELISA and METHOD-9 was a homogenous bridging method, and therefore comparability is not expected. METHOD-9 was developed to improve method ruggedness, and theoretically increase detection of low affinity antibodies and improve drug tolerance. The first experiment was designed to evaluate if there was a simple conversion of titer in order to pool data from the two methods. While no conversion was expected, these data were generated proactively to address potential questions from internal stakeholders or health authorities.

For the 176 positive samples from METHOD-8, the observed titer range in METHOD-9 generally increased in alignment with the titer value from METHOD-8 and vice versa (see Figure 1). Of the 164samples with a numeric titer value in both assays, 41samples (25%) were at the minimum titer value in both.

Of the 12 samples negative in METHOD-9 while positive in METHOD-8, most had the minimum reported titer in METHOD-8. From this experiment, it cannot be determined whether each of the 12incurred samples are false positives in METHOD-8 or false negatives in METHOD-9 since neither assay can be considered the true result in this comparison. Since no negative samples from METHOD-8 were tested in METHOD-9, this experiment does not provide concordance nor information whether the goal of increased drug tolerance for low-affinity antibodies was achieved.

The overall pattern of titer values between METHOD-8 and METHOD-9 matches expectations between two fully validated assays with different formats. The range of titers in one assay for a fixed titer in the other assay most likely represent differential sensitivity to different epitopes or affinities across the two assays. As expected, no simple relationship to convert titer values from one assay to the other is feasible.

Data from the Phase III studies supported the hypothesis of increased sensitivity or drug tolerance because incidence increased approximately 50% in similar disease populations as the Phase 1 cohorts. Regardless of the method used, there was no discernable impact of ADA seroconversion on other clinical outcomes.

Upon confirming the incidence increase when new samples were tested in METHOD-9, the sponsor proactively decided to retest a small study with a rare population not replicated in global trials. The new data from METHOD-9 would be the primary ADA data as it enabled data pooling with the global study and direct population comparisons.

Since full data were available in both methods, concordance was formally evaluated in this dataset with pre-established acceptance criteria. Results for 299 samples tested in both METHOD-8 and METHOD-9 are shown in Table10. As expected, the vast majority of samples were negative in both assays. The overall concordance was 98.0% (293/299). Of the six samples were confirmed positive in METHOD-9 that were negative in METHOD-8, only three had screened positive in METHOD-8. This is supportive of but not proof of the hypothesis that METHOD-8 may be picking up lower affinity positive samples.

Table10. Concordance of confirmed results between METHOD-8 and METHOD-9.

	METHOD-8 positive	METHOD-8 negative
METHOD-9 positive	6	6
METHOD-9 negative	0	287

To complement the concordance assay for confirmed results, the false-positive samples (screen positive but not confirmed) were compared across the two methods to compare whether the same samples were repeatedly borderline (Table11). The samples near the cut point factor were mostly different between the two methods, suggesting that these false-positive results were correctly assigned.

Table11. Concordance of false positive samples between METHOD-8 and METHOD-9.

	METHOD-8 screen positive	METHOD-8 screen negative
METHOD-9 screen positive	3	22
METHOD-9 screen negative	16	N/A

Limited to the samples that screened positive in at least one assay and confirmed positive in neither. Since samples that screen negative in both assays are not false positives, the T cell is not applicable (N/A).

Case #5

METHOD-9 from case study #4 was transferred to another laboratory in China (METHOD-10). To support all studies conducted in China, including Chinese sites in a global trial. The PC performance from the two independent assay validations was consistent (Table6), with full concordance and positive and negative results across PCs and selectivity samples (Table 12). This information provided confidence that incidence data between the two methods can be pooled for integrated analyses.

Table12. Concordance of confirmed results between METHOD-9 and METHOD-10.

	METHOD-9 positive	METHOD-9 negative
METHOD-10 positive	40	0
METHOD-10 negative	0	22

More details can be found in Supplementary Tables 3 and 4.

While the mock samples showed similar values for screening and confirmatory assays and technically met the acceptance criterion of within two- to threefold dilution factors (83%; 33/40), a systemic bias in the reported titers was noted (Figure 2). Since subjects are analyzed at only one laboratory, no impact is expected for calculating boosted titer subjected with pre-existing antibodies.

Approaches to reduce, eliminate, or characterize the source of the systemic bias are underway including re-establishing a titer cut point with larger data sets for both laboratories (data not shown). The goal of this ongoing work is to determine if larger sample size for cut point factors across the two testing sites could remove the systemic bias and allow pooled titer results in future integrated analyses.

As of the writing of this manuscript, however, any analyses that depend on titer values will need to be performed as separate analysis sets by laboratory. That includes both Chinese subjects within a global study and China-only studies.

The comparable results from screening and confirmatory results support that incidence data between the two methods can be pooled for integrated analyses. Any analyses that include titer values, however, will need to be performed separately using the laboratory as a variable.

Approaches to reduce or eliminate the systemic bias in titer reporting are underway including re-establishing a titer cut point with larger datasets in both laboratories (data not shown). The goal of this ongoing work is to enable pooled titer analyses across the two testing sites in future integrated analyses.

Conclusion and practical tips

As programs progress through the stages of drug development, method changes may be planned or needed due to unforeseen operational events. Procedural method updates, platform changes or changes in critical reagents may affect bioanalytical parameters previously established during method validation. These parameters include cut point, sensitivity and/or the drug tolerance and consequently, the changes may impact immunogenicity results. As part of the bioanalytical strategy, the sponsor should evaluate the impact of method changes on ADA incidence and how it may affect data interpretation across studies.

Here, we presented contrasting approaches for the comparison of fully validated immunogenicity methods. In all cases presented, the sponsor has submitted data to health authorities using the new/updated methods with no resulting queries and in one case with a clear endorsement to continue any future analysis with the new method. Table1 summarizes the different approaches utilized and additional decisions taken for the bioanalytical strategy. The reader may be able to learn from these to implement in new situations within the context of the program's immunogenicity risk assessment.

Case study 4 further demonstrates the value of using incurred samples even when the assay validation is comparable for the surrogate positive control. Based on the assay design process, METHOD-9 was hypothesized to have better drug tolerance for low affinity antibodies. Since affinity-purification generally eliminates low affinity clones, even the equivalent validation data did not alter the scientific determination that the methods were different. The incurred sample data support this recommendation.

Case study 5 illustrates that the best practice of using incurred samples is not always technically feasible and presents an alternate approach of selectivity samples as an intermediate approach. When the assay format is the same, this approach can be used to establish comparability within the same study or between studies. It is not recommended when the laboratory change is also accompanied by method changes. In this case study, the incidence data are comparable, however, there are limitations in pooling subanalyses by titer value. Even without incurred samples, this cross-validation approach provided useful information on inter-laboratory comparisons. Cases where no comparison was done must note the differences when data are presented.

If the method change occurs mid-study, the sponsor should consider repeating samples previously analyzed with the original method version and use only the results obtained with the new method version, this may be accomplished by clarifying, in the statistical analysis plan, which dataset should be used. In situations where the sponsor may decide to reanalyze any samples reported by the original method, previous data may then be reported as an appendix to the final bioanalytical report for transparency. Additionally, the method changes and justification should be discussed in the immunogenicity bioanalytical study report. Of course, this is part of the overall risk assessment and the impact of analyzing the two methods as separate datasets. Case study #4 presents an alternative solution, implemented for legacy studies with seropositive subjects continuing past a laboratory closure. In order to ensure that the mid-study assay change did not affect stratification by titer, samples confirmed in METHOD-8 were titered in METHOD-9. This enabled METHOD-9 titer results across the study both pre- and post-lab closure while the overall incidence remained reflective of METHOD-8 and remained a separate analysis set from studies that had been analyzed exclusively in METHOD-9. In case study #5, it is not possible to execute all testing at one laboratory, so the cross-validation between METHOD-9 and METHOD-10 provided context to the interpretation and demonstrated that analyses using titer should be done as separate datasets.

However, if repeat analysis cannot be done, due to sample volume and/or availability, and the method requires a change mid-study then statistical justification comparing results from both method formats may be warranted. It is recommended to discuss with regulators what criteria would be necessary for justification of pooling results from two methods in the mid-study change. A variation on cross-validation analysis or in study cut point comparison, may provide an initial path for pooled statistical evaluation. Sponsors may also develop their own SOPs or best practices for comparison across and within studies.

In several other cases, a development program was terminated or put on hold, with revival many years later. Restarting the original assay was not feasible for various reasons (e.g., no reagents remained from the prior instance, the prior laboratory was no longer in business). In this situation, risk assessments were used to evaluate an appropriate course of action.

For one previously discontinued but revived molecule not included in this manuscript, the previous sampling intervals also deviated from best practices enough that it was determined that the earlier studies could not be expected to accurately report incidence. In this scenario, the upcoming Phase III studies were designed to be stand-alone packages for immunogenicity in the label. Since the data from prior studies were already not being pooled with the new trials, a new method up to current standards was developed and validated rather than maintaining the prior method.

For other molecules, the prior assay format was revived based on the method procedures and validated with current best practices. While the data from the current method was similar to prior data, the revived method cannot be formally crossvalidated to the prior assay performance for comparison. In these cases, data from both assays are intended for pooling in the same integrated analysis, with the original and revived assay used as a subsets for many analyses to test that hypothesis. The proposed label language depends entirely on what the totality of the data show and discussion with the health authorities.

Each case study presented here is unique and some of the strategies were developed in reaction to unexpected operational situations. We hope that they are found useful to apply in new situations and would like to reiterate best practices that would allow for a smooth implementation:

• Ensure ample supply of critical reagents. If this is not possible to cover all on-going studies, new reagent campaigns must be triggered with appropriate assignment to a study (e.g., studies in new indications would use the new lots as data may not be able to be pooled with previous studies).
• Retain study samples for at least 3 months post finalization of clinical study reports. Longer term storage may also allow the sponsor to address queries from health authorities.
• A plan to monitor the performance of the assay over time when new samples are not being actively tested, may be a good risk mitigation strategy to address sudden requests for analysis.
• Include laboratory stability in your overall vendor management plan.
• When testing must be split between laboratories, be prepared for the potential requirement to treat the two datasets separately rather than pooled analysis.

Future perspective

Immunogenicity method changes may be unavoidable and therefore recommendations laid out here can be used to justify various methods of sample reporting, particularly if it is a pivotal study. Additionally, alternate methods for establishing control limits and trending may aid in monitoring assay performance over time. It is important to consider that the validation parameters used to characterize the methods utilized positive controls that may not be representative of study samples in terms of their avidity/affinity (8), therefore incurred analysis of study samples should remain the preferred method, when feasible, for assessing comparability across assays. Protocols should include allowance for samples to be used in development and validation and back-up samples drawn and stored to ensure sufficient volume. The authors foresee assessments of comparability that incorporate clinical relevance.

Upcoming recommendations from the ADAH (ADA Harmonization) committee, which includes members from Europe, FDA and industry, will help to define various validation and in-study parameters across industry. The proposed harmonized method report format, including method summary and validation summary tables, will help to define method changes and updates, which method version was used in individual clinical studies and how they impact the data. The committee's recommendations highlight the importance to include a summary of the method throughout the entire life cycle of the therapeutic, as it aids health authorities in the review of regulatory submissions and market applications, as well as helps the sponsor clearly define their bioanalytical strategy in the context of an immunogenicity risk assessment and its evolution through each major drug development phase, from preclinical to postmarketing.

Executive summary

• Method comparison approaches may vary according to the immunogenicity the risk assessment of the biotherapeutic and operational circumstances.
• High concordance (90-100%) between methods assessed using incurred sample may be used as justification to pool data across studies.
• Depending on the biotherapeutic mechanism of action and the target patient population, health authorities may request reanalysis of high sample numbers to justify method changes.

References

1.. US FDA. Immunogenicity Testing of Therapeutic Protein Products – Developing and Validating Assays for Anti-Drug Antibody Detection (2019). https://www.fda.gov/media/119788/download

2.. Committee for Medicinal Products for Human Use (CHMP). Guideline on Immunogenicity Assessment of Therapeutic Proteins (EMEA/CHMP/BMWP/14327/2006 Rev 1) (2017). https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-immunogenicity-assessment-therapeutic-proteins-revision-1_en.pdf

3.. Shankar G, Arkin S, Devanarayan V et al. The quintessence of immunogenicity reporting for biotherapeutics. Nat. Biotechnol. 33(4), 334–336 (2015).

4.. Qiu ZJ, Ying Y, Lewin-Koh SC et al. Strategies to compare clinical antitherapeutic antibody data when changing assay platforms: a case study. Bioanalysis 7(14), 1775–1783 (2015).

5.. Chamberlain P. Effective presentation of immunogenicity risk assessments and related data in regulatory dossiers. Bioanalysis 11(17), 1581–1592 (2019).

6.. Koren E, Smith HW, Shores E et al. Recommendations on risk-based strategies for detection and characterization of antibodies against biotechnology products. J. Immunol. Methods 333(1-2), 1–9 (2008).

7.. Peng K, Siradze K, Quarmby V, Fischer SK. Clinical immunogenicity specificity assessments: a platform evaluation. J. Pharm. Biomed. Anal. 54(3), 629–635 (2011).

8.. Lofgren JA, Dhandapani S, Pennucci JJ et al. Comparing ELISA and surface plasmon resonance for assessing clinical immunogenicity of panitumumab. J. Immunol. 178(11), 7467–7472 (2007).

9.. Kim JS, Kim SH, Kwon B, Hong S. Comparison of immunogenicity test methods used in clinical studies of infliximab and its biosimilar (CT-P13). Expert Rev. Clin. Immunol. 11(Suppl. 1), S33–S41 (2015).