Mass spectrometry based biomarker discovery, verification, and validation — Quality assurance and control of protein biomarker assays

Abstract

In its early years, mass spectrometry (MS)-based proteomics focused on the cataloging of proteins found in different species or different tissues. By 2005, proteomics was being used for protein quantitation, typically based on “proteotypic” peptides which act as surrogates for the parent proteins. Biomarker discovery is usually done by non-targeted “shotgun” proteomics, using relative quantitation methods to determine protein expression changes that correlate with disease (output given as “up-or-down regulation” or “fold-increases”). MS-based techniques can also perform “absolute” quantitation which is required for clinical applications (output given as protein concentrations). Here we describe the differences between these methods, factors that affect the precision and accuracy of the results, and some examples of recent studies using MS-based proteomics to verify cancer-related biomarkers.

1 The nature of the problem

Although great progress has been made in cancer treatment and diagnosis, as witnessed by longer survival rates (Manrow and Chasan, 2013), cancer continues to be a serious global health issue, causing 8 million deaths world-wide in 2010. Estimates for 2013 predict 1.6 million new cancer cases and 580,350 cancer deaths in the US (Siegel et al., 2013). Cancer biomarkers are needed for disease classification, prediction of therapeutic response, treatment, monitoring, and – perhaps most importantly – for early detection (ideally before a tumor would be detectible by diagnostic imaging). It was recognized quite early that “one-protein one-disease” was not going to be the general rule, particularly for a disease as complex as cancer. In fact, in 2002, Anderson predicted the need for biomarker panels to detect multiple proteins (Anderson and Anderson, 2002). And these predictions have proven to be correct – the “troponin model” is, in fact, a rare occurrence.

The biomarker pipeline is commonly viewed as a series of preclinical phases – biomarker discovery, and verification – before the final clinical evaluation (Paulovich et al., 2008; Rifai et al., 2006; Rodriguez et al., 2010a; Surinova et al., 2011) The inverse relationship between the number of samples analyzed and the number of proteins quantitated is illustrated in Figure 1. Different mass spectrometric methods are used for the different phases of the biomarker pipeline: a typical proteomics discovery experiment uses a non-targeted approach (shotgun proteomics) for the relative quantitation of thousands of proteins in a small number of samples. The comparative analysis results in a list of hundreds of proteins that are differentially-expressed between healthy and diseased samples. Often, after the discovery phase, these potential biomarker proteins are “filtered” by performing studies on additional patients or at more time points, and/or by using higher-specificity mass spectrometry in a “qualification” step. Then, these potential biomarkers are “verified” on a set of 10–50 patient samples. Finally a smaller number of biomakers is “validated” on 100-500 samples. The actual clinical validation of the final biomarkers is done by quantitating a small number of proteins on 500–1000 s of samples. The different mass spectrometric techniques used for the different stages of the pipeline are described in the following section.

Schematic of the various stages of the biomarker pipeline. Modified from A) Rifai, et al. (Rifai et al., 2006) and B) Surinova, et al. (Surinova et al., 2011), and used with permission. Note that the definitions of the different stages are not always the same. We will be using the Surinova definitions in this review.

2 Mass spectrometric methods 2.1 Biomarker discovery

During the 15 or so years since its inception, mass spectrometry (MS) -based proteomics has led to the discovery and identification of thousands of potential biomarkers for cancer and other diseases. Usually, biomarker discovery is done using non-targeted relative quantitation techniques – with output in terms or “up-or-down regulation” of proteins, or fold changes. These techniques can involve a variety of differential protein expression techniques, from 1D or 2D gels with MS-based identification of the proteins of interest, to entirely LC-MS-based relative quantitation methods. Unlike gel-based approaches, MS-based relative quantitation techniques usually rely on an initial digestion of the protein, and the subsequent protein quantitation is actually based on the quantitation of proteotypic peptides that act as surrogates for the protein of interest (the “bottom-up” method). A variety of isotopic labeling techniques can be used for relative quantitation, including isobaric tagging (e.g., iTRAQ (Ross et al., 2004) or TMT-tagging (Thermo TMT, 2008)) or non-isobaric tagging (e.g., mTraq (Applied Biosystems, 2009), or acetylation), and label-free quantitation techniques such as spectral counting are also utilized (Asara et al., 2008). In these non-targeted (“shotgun”) “bottom-up” quantitation techniques, relative quantitation of a protein can be derived from either the intensities of peptides from the protein, or indirectly from the number of MS/MS scans triggered during an LC/MS/MS analysis (which can also be correlated with the protein concentration). For reviews of various MS-based quantitation techniques, see (Elliott et al., 2009; Liebler and Zimmerman, 2013; Schulze and Usadel, 2010).

There is no shortage of potential cancer biomarker proteins – even in 2006, Polanski and Anderson reported that 1261 potential biomarker proteins had been discovered (Polanski and Anderson, 2007), and more recent cancer biomarker discovery projects have added additional proteins (for recent reviews, see (Chambers et al., 2014; Craven et al., 2013; de Wit et al., 2013; Goo and Goodlett, 2010; Hanash et al., 2011; Lin et al., 2012)). However, to date, only a few plasma/serum biomarkers for cancer have been approved by the US FDA (Table 1). Biomarker discovery is not really the problem – the problem lies in the preclinical verification and validation of these putative biomarkers (Makawita and Diamandis, 2010; Rifai et al., 2006; Surinova et al., 2011).

List of FDA-approved protein tumor markers currently used in clinical practicea

[Table omitted. See PDF]

Reprinted from Fuzery, A.K., Levin, J., Chan, M.M., Chan, D.W., 2013. Translation of proteomic biomarkers into FDA-approved cancer diagnostics: issues and challenges. Reprinted from Clinical Proteomics 10, 13 [Epub ahead of print], (Fuzery et al., 2013), with permission. While hCG is commonly used as a tumor marker, it has not been cleared/approved for this application by the FDA. AFP is a Class III analyte because of its non-cancer intended use (aid in prenatal diagnosis of birth defects). 2.2 Biomarker verification and validation

Enzyme linked immunosorbent assays (ELISAs) are best suited for project where only a few biomarkers need to be verified or validated on a large number of samples. However, ELISAs require antibodies against each targeted protein or peptide, and high-quality ELISA assays are often unavailable for the targeted proteins (Haab et al., 2006). The development of an ELISA assay is an expensive (>$100,000 per antibody) and time-consuming process, involving development times of 1–2 years (Wang et al., 2009). ELISA assays also have limited multiplexing capabilities (Krastins et al., 2013), and can exhibit cross-reactivity (Hoofnagle and Wener, 2009). While ELISAs are useful for the final clinical validation assays, i.e., assays where there are fewer protein targets, ELISA technology is not well-suited for quantitating a large number of candidate biomarker proteins, and this is what is needed in the verification phase. The lack of a technique to verify the 100 s–1000 s of potential biomarkers has been termed the “bottleneck” in the biomarker pipeline (Parker et al., 2010; Paulovich et al., 2008). With large numbers of biomarkers waiting to be verified, a different analytical method was needed, and a mass spectrometric technique called multiple reaction monitoring (MRM) or selected ion monitoring (SRM), the Nature Methods “Method of the Year” for 2012 has emerged as the method of choice for performing this verification (Editor Nature Methods, 2013; Evanko, 2012).

MRM is a tandem mass spectrometric technique (MS/MS) which is performed on triple-quadrupole mass spectrometers (Figure 2). This technique involves selection of a precursor ion (in this case, a peptide which acts as a surrogate for the protein of interest). This selection is done by the first quadrupole. The precursor ion (in this case, the protonated intact peptide) is then fragmented in the second quadrupole, and one of the fragments is selected by the third quadrupole and reaches the detector. This precursor/product ion pair is referred to as a “transition”, and the quantitation of the protein is based on the signal that reaches the detector. This fragmentation occurs on the millisecond time scale, and, in an LC/MRM-MS experiment, different transitions can be selected as a function of retention time. This type of analysis is called “scheduled MRM”, and allows the quantitation of hundreds of peptides (and by extrapolation, quantitation of their parent proteins) in a single LC/MRM-MS analysis. For a review of MRM technology, including assay design of and bioinformatic resources, see (Boja and Rodriguez, 2012; Calvo et al., 2011; Gallien et al., 2011; Kuzyk et al., 2013; Makawita and Diamandis, 2010; Meng and Veenstra, 2011; Percy et al., 2013a; Picotti and Aebersold, 2012; Surinova et al., 2011).

Schematic of a “bottom-up” 1D or 2D LC/MRM-MS workflow using SIS peptides or SIS proteins. Reproduced from Figure 1 of (Percy et al., 2013f), Bioanalysis 5(22), 2837–2856 (2013), with permission of Future Science Ltd.

When ¹³C/¹⁵N stable-isotope-labeled standard (SIS) peptide analogs of the endogenous target peptides are used, the reproducibility of MRM assays is similar to that of an ELISA with coefficients of variation (CVs) usually below 10% (Abbatiello et al., 2013; Addona et al., 2009; Percy et al., 2013d). MRM with SIS peptides is considered to be the “gold standard” MS-based quantitation method (Ong and Mann, 2005), and high reproducibility within and across laboratories and instrument platforms has been demonstrated for this quantitation method (Addona et al., 2009).

In a recent MRM experiment, 312 peptides could be quantitated in a 45-min analysis, which is the equivalent of 9 s per assay (Percy et al., 2013d). The cost of a mass spectrometer capable of performing MRM-MS is $500,000, approximately the same as the development of 5 ELISA assays. After this initial investment, the cost of developing an MRM protein assay with high-purity SIS peptides is ∼$1000 per protein if antibodies are not required. MRM assay development on a set of peptides can be performed at rapidly (Picotti et al., 2010) and a highly-multiplexed assay, including standard curves to determine the linear range of the assay, can be completed within a few weeks (Percy et al., 2013d). These features make MRM well-suited for verification of large numbers of proposed cancer biomarker proteins and for the discovery of panels of biomarkers. As has been pointed out by Tan et al. (Tan et al., 2012), for a disease as complex and as varied as cancer, it would be highly unlikely that a single “one-gene-one-protein” biomarker will be sufficient. As the authors also pointed out, this is where proteomics has an advantage. In our experience, MS-based proteomics (and MRM in particular) is the ideal method for finding and quantitating a set of proteins which form a biomarker panel.

2.3 MRM as a biomarker discovery tool

We have already discussed non-targeted mass spectrometry for the biomarker discovery based on differential proteomics (i.e., relative quantitation). It should be pointed out that if you have a highly-multiplexed MRM assay, capable of quantitating several hundred proteins in a single LC/MRM-MS analysis, this assay can be used as a biomarker discovery platform, similar to protein microarrays. In fact, we have used this approach for the study of potential biomarkers of bladder cancer (Chen et al., 2012, 2013) and for biomarkers of CVD (Cohen Freue and Borchers, 2012).

3 Current challenges 3.1 Sources of variability

Although we began the previous section with “biomarker discovery”, we probably should have started with “first, collect a good sample”. There are two main categories of variability that can affect the precision and thus the validity of both the relative quantitation approaches and the “absolute” quantitation approaches. These sources of variability fall into two broad classes, preanalytical and analytical variability: preanalytical variability includes various parameters that need to be controlled during sample collection, storage, and sample preparation (including the denaturation, the digestion (usually with trypsin), that are a key steps in most “bottom-up” proteomics methods). Variables during collection include not only the ones normally considered during study design (such as age and gender), but also the type and manufacturer of the collection tubes, the type of phlebotomy device (including the diameter of the needle), whether the patient is seated or standing, the time of day, etc. For a more complete list of these factors, see (Gelfand and Omenn, 2011; Lista et al., 2013; Lundblad, 2005; Percy et al., 2013f; Yi et al., 2011). Preanalytical variables need to be controlled in order to avoid compromising the study at the outset – high variability can mask true differences due to the disease or the treatment, as well as create artificial differences (Mannello, 2008). Ultimately, these various sources of variability will affect how many samples need to be analyzed to produce a statistically significant result (the power calculation) and whether the results of the study are statistically significant or not.

Analytical variability also needs to be controlled, in both discovery and verification experiments, and includes instrument parameters such as mass resolution and collision energy. The sources of variation and importance of controlling them has been demonstrated in several interlaboratory HUPO and NCI-initiated studies (Adkins et al., 2005; Boja et al., 2011; Boja and Rodriguez, 2012; HUPO, 2010; Omenn, 2004a, b; 2007; Omenn et al., 2005; Paulovich et al., 2010; Rai et al., 2005; Rodriguez et al., 2010a, 2010b; Rudnick et al., 2010), and these sources of variability (for plasma in particular) have recently been reviewed (Gelfand and Omenn, 2011; Lista et al., 2013; Lundblad, 2005; Percy et al., 2013f; Rai and Vitzthum, 2006; Yi et al., 2011; Zhao et al., 2012). To address the issue of analytical variability, we and others have begun the development of SOPs for sample preparation and for MRM analysis (Ohlund et al., 2011; Percy et al., 2013b, 2013e; Tuck et al., 2009), and we have recently developed several “kits” to help ensure proper instrument performance (Percy et al., 2013b, 2013e). One of these kits is designed to test the instrument parameters; the other is designed to evaluate the entire workflow, from sample denaturation and digestion through the MRM data acquisition.

3.2 Problems with plasma

For the discovery of protein biomarkers of disease, plasma has continued to be the “biofluid of choice”. Not only is it relatively easily obtained, but it has long been recognized to contain proteins that reflect a variety of human disease states (Anderson and Anderson, 2002; Farrah et al., 2011; Hu et al., 2006; Thadikkaran et al., 2005) and references cited therein. However, plasma is also extremely complex. It contains proteins covering a 10-order-of magnitude range of protein concentration, from human serum albumin at the upper end (55% by weight), to potentially-important tissue leakage products in the mid-range, with cytokines and interleukins in the low pg/mL range (Anderson and Anderson, 2002) (Figure 3). The wide dynamic range of plasma proteins, as well as the heterogeneity and complexity of plasma, has led to its being termed the “most difficult protein-containing sample to characterize” (Anderson and Anderson, 2002).

Concentration range of proteins in plasma and in urine. A) Concentration range and detectability of cancer-associated proteins (CAPs) in plasma and urine. A) The plotted concentration range shows detected CAPs (blue) and CAPs that could not be detected (gray) in depleted plasma. Estimated protein concentrations for the CAPs in plasma were extracted from Human Plasma PeptideAtlas (Farrah et al., 2011). B) Proteins detected by SRM were compared to proteins previously observed by large-scale proteomic experiments derived from Human Plasma PA (including measurements in unfractionated, crude, and depleted plasma). C) The plotted concentration range shows detected CAPs (blue) and CAPs that could not be detected (gray) in urine. Estimated protein concentrations for the CAPs in urine were extracted from Urine PeptideAtlas (Farrah et al., 2011) D) Proteins detected by SRM were compared to proteins previously observed by large-scale proteomic experiments derived from Urine PA combined with protein observations from Adachi et al. (Adachi et al., 2006). Figure and figure legend reprinted from (Huttenhain et al., 2012), with permission. B) Concentration ranges and functions for 70 plasma proteins. Reprinted from (Anderson and Anderson, 2002), with permission.

3.3 Plasma or serum?

Although sometimes used almost interchangeably, plasma is preferred over serum by the Human Proteome Organization (Omenn, 2004a, 2007; Omenn et al., 2005), because it has been found to be a more reproducible sample. Because blood is an “active” sample, EDTA or other protease inhibitors must be to the collection tubes prior to blood collection (Aguilar-Mahecha et al., 2012; Hulmes et al., 2004). Even though serum is arguably a “simpler” matrix, HUPO does not recommend serum for new proteomics studies because of the variable nature of the coagulation process. However, it has been shown in some recent (Ostroff et al., 2010; Randall et al., 2012; Zimmerman et al., 2012) and older studies (Aziz et al., 1999) that some proteins in serum may be quite stable and, for biobanked samples, serum might be the only type of sample available (Ito et al., 2005). To reduce variability, however, plasma is preferred if this is a possible choice. That being said, clinicians seem to prefer serum, so many of the large-scale studies described below use serum. However, it is impossible to say whether or not the CVs and the statistics would have been better if the authors had used plasma instead.

3.4 The depletion dilemma

To accurately quantitate extremely low-level proteins, such as those shed into plasma by tumors, researchers often use depletion of the most abundant plasma proteins. This does somewhat simplify the mixture, but because plasma contains a 10¹⁰ range of concentrations, even 90% reduction of a major component such as human serum albumin still leaves a 10⁹ range of concentrations in the sample. In addition, depletion of a targeted protein may lead to the inadvertent removal of non-specifically-bound proteins (Bellei et al., 2011; Patel et al., 2012). In our laboratory, to avoid this problem, we have decided to use non-depleted plasma for our MRM studies, relying on the resolving power of HPLC and the mass spectrometer. Based on the papers by Anderson et al. (Anderson and Anderson, 2002), Huttenhain et al. (Huttenhain et al., 2012) and Percy et al. (Percy et al., 2013c), cancer-associated proteins whose concentrations fall within the top 5 orders of magnitude should still be able to be quantitated in non-depleted plasma (Figure 3). We have previously described the performance of a 142 protein panel containing 52 cancer-associated proteins, analyzed in undepleted plasma (Percy et al., 2013d). A table showing the performance metrics and disease associations has been reprinted here in the Supporting Material. The depletion issue, however, is still unresolved – Tu et al. (Tu et al., 2010) reported in 2010 that they needed depletion to see low abundance plasma proteins, while other researchers (Bellei et al., 2011; Patel et al., 2012) confirmed that depletion removed non-targeted proteins. In their study of cancer-related proteins, Huttenhain et al. used Agilent's MARS HU14 depletion system to remove the 14-most abundant proteins from the plasma samples, but did not need depletion for urine, which has a “narrower range of protein concentrations” (Huttenhain et al., 2012)}. Even with depletion, they still used SIS peptides for the plasma analyses. Most likely, whether the deletion step can be eliminated or not will probably have to be experimentally determined based on the specific protein targets and their concentrations in plasma.

3.5 Enrichment – the other side of the depletion coin

Another way to simplify the highly complex plasma matrix is through the use of enrichment. This enrichment can occur either before or after thy enzymatic digestion. In protein-capture-based immuno-MRM (IP-MRM) and mass spectrometric immunoassay (MSIA), the proteins are captured and then eluted and digested, and analyzed by LC/MS/MS (Berna et al., 2008, 2007) or by LC/MRM-MS (Krastins et al., 2013; Nicol et al., 2008). Although enrichment techniques typically cannot be effectively multiplexed, in the approach described by Krastins (Krastins et al., 2013), several target proteins can be captured in different MSIA tips, and the eluates can be combined before the digestion and subsequent MRM-MS analysis, thereby taking advantage of the multiplexing capability of MRM. Alternatively, the protein can be digested before affinity capture. In this case, an anti-peptide antibody is needed against the peptide that represents the target biomarker protein. This antibody is immobilized (usually on agarose beads), and the target is captured on the antibody, after which it is either eluted and analyzed by LC/ESI-MRM-MS (as in SISCAPA (Anderson et al., 2011, 2004)), or the beads can be placed directly on a matrix assisted laser desoprtion/ionization (MALDI) target (immunoMALDI, iMALDI (Camenzind et al., 2013; Jiang et al., 2007a, 2007b; Shah and Borchers, 2009)). In iMALDI, and the matrix solvent elutes the peptides during the matrix spotting procedure. Because of the speed of the actual MALDI analysis (seconds per sample), a recent SISCAPA paper (Razavi et al., 2011) used elution of the released analytes onto a MALDI plate followed by MALDI-MS analysis, instead of LC/MS. While neither approach has the multiplexing capabilities of MRM, they have the advantages of speed and selectivity for a small number of target analytes, which are highly desirable features in a clinical assay. The Bruker Biotyper and the BioMerieux are becoming more and more widely used for bacterial identification in hospital clinics; because these instruments are, in fact, benchtop MALDI instruments, diagnostics based on SISCAPA, iMALDI, and SISCAPA with MALDI detection may soon be used in clinical settings.

4 Bioinformatics tools for MRM method development

The proteins targeted for MRM analysis proteins can be based on “shotgun” biomarker discovery experiments, or can be based on studies found in the literature. In either case, databases must be used to ensure that the precursor peptides and corresponding fragment ion pairs (“transitions”) are unique to the target protein within the proteome (this is usually done by BLAST searching). These transitions should also be sensitive, and should not contain known post-translational modification sites, missed tryptic cleavage sites, or easily oxidized amino acids (such as methionine). In addition, they should be between 8 and 20 amino acid residues in length to provide good sensitivity and MS/MS fragmentation, and they should be sufficiently hydrophilic for good HPLC separation but hydrophobic enough to be retained on the column. These peptide selection “rules” are described in greater detail in Kuzyk et al. (2013).

Several software packages which automatically apply these rules to libraries of peptides, and libraries of transitions based on experimental data are currently available to assist in development of multiplexed assays. A survey of available software packages which can assist in the selection of peptides and transitions is given in the recent review by Boja and Rodriguez (Boja and Rodriguez, 2012). Sometimes, the relative peptide sensitivities are inferred from the number of times it has been detected or reported LC/MS/MS experiments, so these software packages fall into two broad categories – those which compile experimental data from different instruments (often with contributions from users, e.g., PeptideAtlas) (PeptideAtlas, 2010), and those which predict “sensitive” peptides and transitions on the basis of the peptide's sequences (e.g., PepFly (PepFly, 2013; Sanders et al., 2007)).

There are several available libraries of MRM-related information for potential cancer biomarkers. One of these (reported in 2009) includes a downloadable excel file containing data for 9677 peptides representing ∼1572 proteins from the human MCF-7 breast cancer cell line, along with information on the quality of the MS/MS data (Yang and Lazar, 2009). Data in this library was obtained from 2D fractionation of cell extracts (SCX and RPLC). Due to the increased sensitivity of the MRM-MS technique, the authors reported that they were able to quantitate proteins that were previously not detectable in their unfractionated original sample. Recently, Huttenhain et al., from the Aebersold group, compiled a library of more than 1000 cancer-associated peptides and transitions from an extensive study of plasma and urine (Huttenhain et al., 2012), and have deposited this information into the PeptideAtlas SRM Experiment Library (Farrah et al., 2012; PASSEL, 2012). A different type of library, designed specifically for cancer-associated pathways, is being developed for cancer research and for determining appropriate therapies (Remily-Wood et al., 2011). This library already contains MRM assays targeting 218 cancer-related proteins in specific pathways. Based on a library of 876 transitions, different subsets of peptides can be selected to examine these pathways in patient or research samples.

If you choose to determine suitable peptides yourself by performing manual searching of Uniprot and PeptideAtlas, approximately 8 proteins can be processed in a single day. However, our laboratory has recently developed an automated software package (PeptidePicker, available as a free download) to speed up this process (Mohammed et al., 2014). This newly-developed software can automatically find and use peptide sensitivity and post-translational modification (PTM) data from UniProt (UniProt Consortium, 2009), The Global Proteome Machine (The Global Proteome Machine Organization, 2004–2011), NCBI's dbSNP (Sherry et al., 1999), and PeptideAtlas (PeptideAtlas, 2010). This increases the speed and the accuracy of this process, eliminates human error, and allows the selection of proteotypic peptides for 50 proteins in 1 h.

Peptide elution time is another important factor in developing an assay. The Skyline software package for MRM assay generation (Skyline SRM/MRM Builder, 2011; update v0.7) uses indexed Retention Time (iRT)-based prediction as part of this development (iTR, (Escher et al., 2012)). The iRT value is a “dimensionless peptide-specific number” that allows prediction of retention time for various LC platforms and chromatographic conditions. The iRT software is based on empirical data and uses retention times related to a standard set of peptides. Use of the iRT software package allowed the prediction of retention times within 2 min, which the authors claim was 4 times narrower than with retention times based solely on the basis of the sequence.

5 Quality control and assay optimization

It is probably worth mentioning that we have found that empirical tuning of the collision energy and the cone voltage (using the SIS peptide analog of the target peptide) leads to an 11.4-fold improvement in sensitivities compared to automatically-predicted tuning conditions (Kuzyk et al., 2009). The availability and use of SIS peptides as internal standards not only increases the accuracy of the quantitation, but also greatly simplifies the tuning of the instrument for optimal peptide detection. It also allows accurate interference testing – which is done by comparing the peak shapes of the NAT (natural) and SIS forms of the peptide, as well as the relative ratios of their various transitions (Figure 4). Unlike other types of internal standards, ¹³C/¹⁵N-labeled internal standard peptides also have exactly the same retention times as the naturally-occurring peptides so there is no question as to which peak corresponds to the target analyte. With a standard flow LC system, we have demonstrated reproducible retention times with CV's of 0.06% and average peak widths of 5.7 s (FWHM; 20-sec at baseline) for a set of potential biomarker proteins (Percy et al., 2012). These reproducibly narrow elution windows not only improve the accuracy of interference testing, but also reduce interferences and allow tighter MRM scheduling, thus increasing the multiplexing capability of the assay. For comparison, the Biognosys website (Biognosys, 2014) claims a 22-min peptide elution window for a 90-min gradient for in silico retention time prediction, and a 5-min retention time window using iRT.

Peptide interference-screening in control and patient plasma samples. A) In the control samples, 3 transitions per peptide are monitored in buffer and plasma (n = 2 for each sample type). The average relative ratios of the Q1/Q3 MRM ion pairs for the SIS peptide in buffer, the SIS peptide in plasma, and the NAT peptide in plasma) are determined, and the variability and assessment of peak shape, symmetry, and retention time are performed. B) Interferences can be detected through peptide relative response correlation plots for each protein. For the samples that deviate from linearity (see the sample marked with an arrow), the extracted ion chromatograms for each peptide need to be inspected to determine which peptide contains the interference. Figure and figure legend reprinted for Part A are reprinted, with permission, from (Percy et al., 2013b) and the figure and figure legend reprinted for part B are reprinted, with permission, from (Percy et al., 2013d).

5.1 Interference testing

As mentioned above, while various software packages may suggest commonly-detected precursor and product ions from the target protein, it is absolutely critical to ensure that the transitions on which the quantitation is based are free of interferences from other components in the matrix, particularly in a matrix as complex as plasma. (Abbatiello et al., 2010; Domanski et al., 2012; Huttenhain et al., 2012; Kuzyk et al., 2013; Percy et al., 2013a, 2013b, 2013d; Sherman et al., 2009) Details of how interference testing is performed in our laboratory can be found in (Domanski et al., 2012; Kuzyk et al., 2013; Percy et al., 2013a, 2013d), and is shown graphically in Figure 4. Briefly, the SIS and NAT signals must co-chromatograph, exhibit similar peak shapes, and there cannot be any non-specific co-eluting ions. This type of accurate interference testing cannot be done without the use of SIS peptides. In Figure 4A, peptide VGYVSGWGR is interference-free, while peptide YWGVASFLQK has significant interference from the plasma background in the trace for the endogenous peptide.

A second test involves calculating the average relative ratio of the MRM transitions for the SIS in buffer, SIS in plasma, and NAT in plasma, which must exhibit <20% variability to qualify as being interference-free. The use of these relative response measurements on interference-free MRM transitions during protein quantitation adds an additional dimension of specificity to the dynamic MRM assay, with the other dimensions being the precursor mass-to-charge ratio, the product ion mass-to-charge ratio, and the retention time. The use of multiple peptides for each protein also allows the detection of patient-specific interferences, based on the relative ratios of the different transitions (Figure 4B).

5.2 Concentration-balanced SIS peptides

For the most accurate quantitation, assays should be developed using a concentration-balanced, rather than an equimolar, mixture of SIS peptides. This ensures that the concentrations of the SIS peptide internal standards are as close as possible to the concentrations of the endogenous peptides in the digest of the biological sample. Although balancing these ratios does increase the assay development time, it minimizes the analytical variation between analyses, improves the quantitation accuracy, and increases the linear range of the calibration curve (Figure 5).

Reproducibility of MRM analyses without SIS peptides, with an equimolar mixture of SIS peptides, and with a concentration-balanced mixture of SIS peptides.

6 Larger-scale studies – moving towards validation

While MRM is still most often used for biomarker verification involving the analysis of 10–50 samples, it is beginning to be used for biomarker validation which involves the analysis of 100–1000 samples (Table 2). We have recently written a review focused on recent applications of MRM to the analysis of plasma and serum from cancer patients for biomarker verification (Chambers et al., 2014), so some of the studies described in that publication will only be briefly mentioned here.

Recent applications of MRM to cancer biomarker determination in plasma and serum.

[Table omitted. See PDF]

The number of proteins quantified by MRM. Reprinted from (Chambers et al., 2014), with permission.

In one of the examples described in this earlier review article, 7 high abundance serum proteins were evaluated in 431 patients for predicting the probability of colorectal cancer (Brock et al., 2012). All 7 proteins were differentially-expressed in the cancer patients (p < 0.05) and a 6-protein panel provided an AUC value of 0.900. For the set of high abundance targets studied in this paper, it was possible to use very short (4.5 min) gradients, but usually the presence of co-eluting components makes longer gradients necessary.

Another example of a relatively large verification study was performed by Liu et al., in 2012 for the diagnosis of non-small cell lung cancer (NSCLC) (Liu et al., 2012). Using multidimensional chromatography (SCX fractionation followed by HPLC) and label-free quantitation (spectral counting) on samples from 18 patients, 101 serum proteins were found whose expression levels correlated with NSCLC. Two of these potential biomarker proteins, glycoproteins alpha-1B-glycoprotein (A1BG) and leucine-rich alpha-2-glycoprotein (LRG1), were validated on a set of serum samples from 101 patients. The MRM assay (targeting two non-glycosylated peptides for each protein) gave an AUC value for 0.816 for A1BG and 0.880 for LRG1 separately, and 0.909 for the combined biomarker panel (Figure 6).

Serum concentrations of A1BG, LRG1 as determined by MRM-MS, and their associations with NSCLC and the effect of using both proteins together as biomarkers. A–B) MRM Intensities of selected reference peptides of A1BG and LRG1 both showed good linear correlation with on-column abundance. The x-axis represents base-3 logarithm of ratios of spiked light and heavy isotopic peptides, with the y-axis corresponding to the observed peak area ratios in base-3 logarithmic scale. Red triangles suggest the limit of linear quantification (LOQ) of each peptide. Note that these two peptides were used to report the absolute concentration, with comparison to another two less-optimal peptides shown in Figure S4. C–D) The chromatography peaks of the best transitions of two peptides for A1BG and LRG1. Note that the Signal-to-Noise ratios are sufficient for quantification. E) The distribution of serum levels of A1BG and LRG1 between cancer and control groups. F) ROC analysis suggested over-expressed A1BG and LRG1 were associated with NSCLC and their combined panel could provide the better discriminative performance than each of the protein. Figure and figure legend are reprinted from (Liu et al., 2012), with permission.

6.1 Differentiating between protein isoforms

MRM can be used to quantitate specific protein isoforms as well as specific proteins. While 2D gel-based methods often have the advantage of being able to keeping protein isoforms separate, in a typical “bottom-up” MS-based method the protein is digested into peptides and protein isoform stoichiometry information can be lost when the different isoforms are “regrouped” into a single isoform by the database search engine (Hoofnagle and Wener, 2009). With MRM, the researcher has a choice of whether to select a specific isoform that differentiates between the variants, or one that is characteristic of all of the isoforms. This advantage of the MRM approach was recognized quite early by Yocum et al, and they used peptides that were specific to certain protein isoforms in order to obtain “more accurate measurements of [the] differential expression” of cytoskeletal keratin type II isoform 8 (Yocum et al., 2008). Another example of choosing an isoform-specific peptide is in the paper by Sung et al., where the diagnostic value of two protein isoforms of serum amyloid A (SAA), potential biomarkers for lung cancer which differ by only 10 amino acids, were separately assessed (Sung et al., 2012). Crude serum samples from 99 healthy controls and 100 lung cancer patients were analyzed and the concentration of both SAA isoforms was significantly higher in cancer patients. Approximately equal AUC values (0.702 for SAA1 and 0.708 for SAA2) were found for the two isoforms. By examining the serum from 18 late-stage cancer patients and 15 healthy controls, Tang et al. (Tang et al., 2013) studied multiple isoforms of chloride intracellular channel protein (CLIC1 and CLIC4) and 4 tropomyosin isoforms (TPM1-4) in serum as potential biomarkers for ovarian cancer. After depletion of the top 20 serum proteins, a GeLC-MRM experiment was performed without the use of internal standards. The CLIC1/CLIC4 ratio varied by more than 2-fold in some samples, with AUC values for CLIC1 and CLIC4 of 0.86 and 0.79, respectively. The AUC values for TPM were dependent on the particular isoform, with 0.72 for TPM3 and 0.81 for TPM4.

Two proline-hydroxylated sites (position 530 and 565) in α-fibrinogen were examined as potential biomarkers for pancreatic cancer (Yoneyama et al., 2013). Although the total α-fibrinogen concentration between the 27 healthy controls and 70 cancer patients stayed constant, the absolute concentrations of these specific hydroxylated peptides and the percentage hydroxylation both increased in cancer patients (p < 0.05). The hydroxylated α-fibrinogen peptides had AUC values of 0.650 and 0.668, respectively, both of which were below the AUC value (0.903) for the FDA-approved pancreatic biomarker, CA19-9.

Phosphorylated or glycosylated isoforms can also be specifically targeted, although this is usually done after enrichment. Cima et al. in a study of prostate cancer (Cima et al., 2011) and Kalin et al. in a study on metastatic castration-resistant prostate cancer (mCRPC) (Kalin et al., 2011) specifically enriched the samples in glycosylated peptides, then removed the glycosylation with PNGase-F, and used MRM to quantitate the “bare” peptides. Ahn et al. (Ahn et al., 2012) used a similar method to enrich the sample in fucosylated protein glycoforms, the proteins were eluted, and, after digestion, the glycosylated proteins were quantitated by MRM after digestion. The plasma concentrations of alpha-1-antichymotrypsin and alpha-1-antitrypsin increased in HCC patients and the 11-protein biomarker panel produced an AUC of 0.963 for hepatocellular carcinoma.

There is one caveat with these enrichment methods: if only modified proteins or peptides are enriched, one loses the information about the unmodified forms. To address this issue, in a recent paper in Analyst (Ahn et al., 2013), the same research group used MRM to measure both the glycosylated protein after lectin capture, and the entire concentration of the protein in plasma. In this way, they were able to determine whether differences were in the proportion of the protein that was glycosylated or whether the total glycoprotein concentrations were different between hepatocellular carcinoma plasmas and hepatitis B virus plasmas. From this study, they were able to determine that the change that occurred was in the abundance of the fucosylated versions of the biomarker candidate A1AT and FETUA proteins, rather than a change in the plasma concentrations of these proteins.

Another way around this problem is to indirectly determine the concentration of a modified peptide by determining the change in concentration of the unmodified peptide, before and after the removal of the PTM (Domanski et al., 2010). This technique was developed for phosphorylated peptides, and is called the phosphatase-based phosphopeptide quantitation (PPQ) method (Figure 7). This method avoids the problem of differential capture or release of multiply-phosphorylated peptides from the affinity medium, and also avoids problems due to potential differences in sensitivity of the variously modified forms. In this paper, the PPQ-MRM method was used to determine the phosphorylation stoichiometry and the absolute concentrations of specific phosphopeptides from cell lines containing phosphorylated ERα, HER2, RAF, and ERK1.

Schematic of a phosphatase-based phosphopeptide quantitation (PPQ) experiment. Proteins are digested with trypsin and isotope-labeled standard peptides (non-phosphorylated forms of target) are added (light gray) to increase accuracy of quantitation and specificity of detection. The sample is split into two and buffer-only or phosphatase treated. Two LC/MRM-MS analyses are then performed. Phosphatase treatment will increase the signal of the natural peak (dark) if phosphorylation is present. Peak area ratios are then determined from the area of the natural peak (dark) versus the area of the internal standard (light gray). Phosphorylation stoichiometry can then be determined by comparing the Peak area ratios from the untreated and phosphatase treated samples as indicated in the boxed formula (assuming 100% dephosphorylation efficiency). Figure and figure legend are reprinted from (Domanski et al., 2010), with permission.

7 Correlation between MRM and ELISA

Because the final assay for large-scale clinical evaluation has typically been an ELISA, the correlation between MRM assays and ELISAs is important. In the study by Kalin et al. mentioned above, the ELISA data indicated a significant correlation of the concentration of carcinoembryonic antigen-related cell adhesion molecule 1 with patient survival with a (p-value 0.018, log-rank with Bonferroni correction) while the MRM data showed no significant difference (p-value 0.153). Because there are 11 isoforms of this protein, it is not known if both analysis methods targeted the same specific isoforms.

In a study by Pan et al. (Pan et al., 2012), 5 proteins from plasma that had been previously found to be over-expressed in pancreatic cancer tissue or pancreatic cell culture secretome were quantitated by ELISA and MRM. The AUC values for TIMP1 by MRM assay (0.87) were better than those from the ELISA (0.77) assay for discriminating pancreatic cancer between the diseased patients and the healthy controls. In this case, the actual concentrations measured by MRM were higher than those determined by ELISA, so MRM may have measured the total TIMP1 present in the sample while ELISA may have measured only free TIMP1 protein.

A comparison of ELISA and IP-MRM was performed by Nicol et al. (Nicol et al., 2008) for potential biomarkers of lung cancer in serum. Both ELISA and IP-MRM rely on anti-protein antibodies, and linear responses of down to the fairly low ng/mL range were obtained by IP-MRM, with a For those proteins where ELISAs were available, there was a good correlation of the MRM results with ELISA – secretory leukocyte peptidase inhibitor (SLPI, correlation coefficient r = 0.83), tissue factor pathway inhibitor (TFPI = 0.70), and metalloproteinase inhibitor 1 (TIMP1, r = 0.95). Lin et al. recently performed a comparison of IP-MRM with ELISA for the quantitation of 6 colon cancer biomarker candidates in plasma (Lin et al., 2013). CVs of 2.3–19% were reported, and the correlation coefficients were very protein-dependent, but a fairly good correlation was obtained for all of the biomarkers except for endoglin (ENG). Correlation coefficients for 4 potential biomarkers are shown in Figure 8 (metalloproteinase inhibitor 1, r = 0.90; cartilage oligomeric matrix protein, r = 0.97; thrombospondin-2, r = 0.75; endoglin, r = −0.17; mesothelin r = 0.67; and matrix metalloproteinase-9, r = 0.88).

A comparison of IP-MRM with ELISA for the quantitation of 6 colon cancer biomarker candidates. Response curves for IP-MRM analyses of recombinant TIMP1, COMP, MMP9, THBS2, MSLN and ENG proteins. Proteins were spiked at 10L640 ng/mL in a background matrix of 60 mg/mL BSA in DPBS and analyzed by IP-MRM as described in Experimental Procedures. Values plotted are mean ± standard deviation (n = 3). Figure and figure legend reprinted from (Lin et al., 2013), with permission.

Végvári et al. (2013) recently reported the detection of a previously-undetected PSA variant coded by SNP-L132I (rs 2003783). The authors used a specially-designed MRM assay targeting specific peptides from PSA on in 72 plasma and seminal fluid samples. Nine samples were positive for this SNP, and the results correlated well with a commercial immunoassay. Even though the actual concentration values from the MRM assay were 34–60% lower than those observed with the immunoassay, the coefficient of determination (R²) between the two methods were 0.82–0.85 for seminal fluid, and >0.99 in plasma.

MRM performed at least as well as ELISA in several other non-cancer-related studies, including studies on C-reactive protein for the diagnosis of rheumatoid arthritis (Kuhn et al., 2004), 13 potential biomarkers in amniotic fluid as biomarkers for Down syndrome (Choa et al., 2011), and biomarkers of acute rejection in heart transplantation (Cohen Freue et al., 2013). In a study of hordein (gluten) in beer, ELISA gave false negative results, possibly because the antibody used for the ELISA assay missed some of the hordein isoforms (Tanner et al., 2013).

8 MRM and the biomarker pipeline

The complementary role of “shotgun” proteomics for biomarker discovery and MRM for biomarker verification was proposed in one of the first applications of MRM to cancer-related biomarker discovery (Whiteaker et al., 2007) – in this case for the discovery and confirmation of biomarkers found in a mouse model of breast cancer, using a SISCAPA approach. The combination of MS techniques proposed in this study has become the general approach to biomarker discovery and verification.

One important distinction to keep in mind is that there is a difference between biomarkers that fail during the verification process, and those that cannot even be evaluated, most often due to the lack of antibody reagents. There is a significant difference between these two situations, although the phrase “the biomarker could not be verified” is sometimes used for both cases. In the Whiteaker study (Whiteaker et al., 2007), 60 biomarker candidates were evaluated by semi-quantitative MRM (i.e., MRM without SIS peptides) in one month, and approximately 80% of these biomarkers were tentatively confirmed as tumor biomarkers, reflecting the accuracy of the MS-based discovery part of the experiment. To reduce the cost, the authors state that “analyte-specific reagents” (such as SIS peptides and peptide-specific antibodies) were not generated until after this tentative verification.

So what are the costs? A short but revealing recent paper by Anderson (Anderson, 2012) discusses the challenges and costs of performing a “complete” discovery-to-verification study. His detailed examination of a study on human (Addona et al., 2011) and on mouse plasma (Whiteaker et al., 2011) is summarized here. In the Addona study on human heart muscle damage, Anderson shows the progression from a large number of proteins (1000) potential biomarkers to a smaller number of “qualified” and then an even smaller number of “verified” biomarkers. Out of the original 121 proteins found to be differentially-expressed in the original discovery stage (3 patients, 2 time points), 82 proteins were detected at more than 1 time point in peripheral plasma (qualification stage #1), and 52 of these could be detected by “accurate inclusion list” MS/MS in qualification stage #2 (10 patients, 3 time points, giving 3 pooled samples). Only 20 of these 52 proteins were able to be evaluated in a verification study (6 by ELISA, 7 by Western blot, and 7 by MRM with SIS peptides). Thus, only 40% of the potential candidate proteins could actually be quantitated. Ultimately, 6 proteins were analyzed by ELISA in 100 patients (verification step), and this $2.2 million study did reveal important findings about plasma biomarkers of both heart muscle damage and biomarkers of the catheterization process itself (which also demonstrates the importance of control experiments).

In a similar “pipeline” (Whiteaker et al., 2011), also discussed by Anderson (Anderson, 2012), an entirely MS-based approach was used. A list of 1144 candidate mouse biomarker proteins for breast cancer (found in the discovery stage on tumor tissue) was reduced first to 118 proteins by “accurate inclusion list” MS/MS (qualification stage #1), and then to 91 proteins on the basis of the number of transitions per peptide that could confidently be detected and because of their ROC characteristics. Eighty-eight MRM assays for 87 mouse proteins were developed – 57 peptides (representing 57 proteins) by direct MRM and 31 peptides (representing 30 proteins) by SISCAPA. Eighty samples were analyzed in triplicate.

Based on these two studies, Anderson predicts a cost of $1.5 million and 1 year to generate MRM assays for 50-100 good candidate biomarker proteins, and a cost of $4 million and 4 years for “the full pipeline” (i.e., up to the clinical validation stage). From these calculations, Anderson concludes that these costs are “reasonable” for the verification of 50–100 good protein biomarker candidates, “especially considering the much larger amounts of money that have been spent on inconclusive biomarker ‘omics studies to date” (Anderson, 2012).

Financial & competing interests disclosure

The authors acknowledge funding from Genome Canada, Genome BC, and the Western Economic Diversification of Canada. CHB is the CSO of MRM Proteomics, which is co-marketing the standardization kits. The authors have no financial conflict with materials discussed in the manuscript apart from those disclosed, and this manuscript was written without assistance.

Supplementary data

Supplementary data related to this article can be found at http://dx.doi.org/10.1016/j.molonc.2014.03.006