Quantifying Tumor Vascular Heterogeneity with Dynamic Contrast-Enhanced Magnetic Resonance Imaging: A Review

Abstract

Tumor microvasculature possesses a high degree of heterogeneity in its structure and function. These features have been demonstrated to be important for disease diagnosis, response assessment, and treatment planning. The exploratory efforts of quantifying tumor vascular heterogeneity with DCE-MRI have led to promising results in a number of studies. However, the methodological implementation in those studies has been highly variable, leading to multiple challenges in data quality and comparability. This paper reviews several heterogeneity quantification methods, with an emphasis on their applications on DCE-MRI pharmacokinetic parametric maps. Important methodological and technological issues in experimental design, data acquisition, and analysis are also discussed, with the current opportunities and efforts for standardization highlighted.

Recommended by Ana Maria Tari

Department of Radiology and the Wright Center of Innovation in Biomedical Imaging, The Ohio State University, Room 430, 395 West 12th Avenue, Columbus, OH 43210, USA

Received 15 October 2010; Revised 14 January 2011; Accepted 23 February 2011

1. Introduction

Development of novel antiangiogenic agents has posed a new challenge to medical imaging technologies. Unlike the systemic cytocidal effect of traditional cytotoxic drugs, these new agents are selectively targeted to the endothelial cells of tumor neovasculature, inducing a rapid shutdown of tumor blood supply without substantially affecting any morphologic metric [1]. Their cytostatic nature makes the conventional method of tumor size measurement inadequate for response evaluation. Several molecular and functional imaging techniques, namely, dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) [2, 3], positron emission tomography (PET) [2, 4], and dynamic contrast enhanced computed tomography (DCE-CT) [2, 5], have been developed in response to this challenge by monitoring pathophysiologic changes in various aspects of tumor vascular structure and functionality.

Among these imaging techniques, DCE-MRI has the unique advantage of not using ionizing radiation. Consequently, its repeated use in response assessment is not limited by patient dose considerations [2, 6]. In fact, daily or weekly repeats of DCE-MRI examinations are common in preclinical and clinical studies, allowing researchers to capture rapid change in tumor vasculature during the short "normalization window" after administration of an antiangiogenic agent [7]. Moreover, the wide availability of MR scanners makes this technique easily accessible to both clinicians and patients. As of September, 2010, American College of Radiology (ACR) has accredited 5498 MR facilities in the United States, compared to only 963 ACR-accredited PET facilities. All these advantages make DCE-MRI a convenient yet flexible modality for noninvasive evaluation of tumor microvasculature and angiogenesis. It has been applied to many preclinical studies and early phase clinical trials of various antiangiogenic and antivascular agents, as well as in nontrial settings for tumor grading and clinical outcome prediction [8, 9]. Quantitative imaging biomarkers derived from pharmacokinetic modeling of DCE-MRI data have been demonstrated to be correlated with various physiologic and pathophysiologic parameters in human and animal models, and found to be predictive of response rate and progression in patients accepting PTK/ZK treatment [10, 11].

Although preclinical and clinical results have shown that DCE-MRI is a powerful technique for the detection of subtle changes in tumor vascular function, many improvements are still needed for this technique to reach its maximum potential. Particularly, the important issue of tumor vascular heterogeneity needs to be addressed. Heterogeneous distribution of neovasculature has been reported in almost all types of solid tumor [12-14]. This phenomenon reflects the more fundamental heterogeneity in the clones of different tumor cell genomic subtypes and the expression of various pro- and antiangiogenic genes [15-17]. Experimental evidence shows that tumor vascular heterogeneity is associated with disease progression and malignancy [18]. Recent studies suggest that cancer relapse and metastasis are likely to be governed by a small population of cancer stem cells in aberrant vascular niches [19-21]. This new understanding strongly implies tumor vascular heterogeneity may have key importance for the understanding of disease mechanism and the discovery of novel treatment strategies. However, so far this critical information was simply ignored or only briefly mentioned as qualitative description in most preclinical and clinical studies. Even in the limited number of cases where quantification efforts have been made, substantial discrepancies exist in quantification method, parameter choice, and result interpretation. This lack of consensus has greatly hindered the investigation of tumor vascular heterogeneity and its influence on treatment response with DCE-MRI. In this paper, we critically review different quantification methods for heterogeneity currently available in the literature, and discuss several methodological and technological issues that are important in assessment of tumor vascular heterogeneity. Although these methods can be applied to any DCE-MRI image or parametric map, we emphasize their application on pharmacokinetic parametric maps as pharmacokinetic modeling is the most advanced DCE-MRI data analysis approach that provides pathophysiologically meaningful quantitative interpretation of DCE-MRI data.

2. Imaging Tumor Vascular Heterogeneity with DCE-MRI

In most solid tumors, the neovasculature developed through angiogenesis has an abnormal structure, with defects on walls that make them more permeable than normal vessels. As blood carrying a low molecular weight contrast agent (such as Gadopentetate Dimeglumine) flows through tumor vasculature, the contrast agent molecule leaks into the interstitial space and accumulates there. Interaction between the paramagnetic Gd³⁺ ion and adjacent water protons enhances tissue longitudinal (_T1 ) relaxation rate, generating a brighter signal in _T1 -weighted images. The magnitude and rate of signal enhancement are determined by tissue vascular fraction, perfusion, microvascular permeability, and relative volume of the extracellular extravascular space (EES). The brightness information can be converted to contrast agent concentration with the linear relationship between gadolinium concentration and _T1 relaxation rate [27]. The dynamic change of contrast agent concentration can be described by various model-free parameters or fitted to a pharmacokinetic model (Figure 1) [22-26], yielding parametric maps that give an intuitive and quantifiable illustration of tumor vascular heterogeneity. The most commonly used pharmacokinetic and model-free parameters are summarized in Tables 1 and 2, using the standard nomenclature proposed in the consensus paper by Tofts et al. [28]. Details about the use and interpretation of these parameters are beyond the scope of this paper. Interested readers are referred to the in-depth reviews written by Tofts et al. [28] and Evelhoch [29].

Table 1: Pharmacokinetic parameters often used in DCE-MRI pharmacokinetic analysis.

Parameter	Definition	Units	Alternative names	Literature
Ktrans	Volume transfer constant between EES and blood plasma	min-1	EF, KPS	[22, 23, 26]
ve	EES volume per unit tissue volume	N/A	Interstitial space	[22, 23, 26]
vp	Blood plasma volume per unit tissue volume	N/A		[23]
kep	Rate constant from EES to blood plasma	min-1	k21	[22-26]
kpe	Rate constant from blood plasma to EES	min-1	k12	[25]
kel	Elimination rate constant	min-1		[24, 25]
Amp	Amplitude of the normalized dynamic curve	N/A	A	[24, 25]

Table 2: Model-free parameters often used for DCE-MRI dynamic curve description^[dagger] .

Parameter	Definition	Units	Alternative names	Literature
Area-under-the-curve (AUC)	Area under the signal intensity or gadolinium dynamic curve	a.u·min or mmol·min/L	IAUC, AUGC, IAUGC	[29]
Relative signal intensity (RSI)	RSI=S(t)/S0	N/A		[30]
Initial slope	Maximum or average slope in the initial enhancement	a.u./min	Enhancement slope, upslope, enhancement rate	[31]
Washout slope	Maximum or average slope in the washout phase	a.u./min	Downslope, washout rate	[32]
Peak enhancement ratio (PER)	PER=(Smax -S0 )/S0	N/A	Maximum signal enhancement ratio (SERmax )	[33]
Tmax	Time from contrast agent arrive to peak enhancement	s	Time-to-peak (TTP)
Maximum intensity-time ratio (MITR)	MITR=PER/Tmax	s-1		[34]

^[dagger] Unlike the pharmacokinetic parameters, the nomenclature for model-free parameters has not been standardized.

Notation and abbreviations: S(t) : MR signal intensity at time t ; _S0 : precontrast signal intensity; _Smax : maximum signal intensity; a.u.: arbitrary unit.

In the literature, several models are available for DCE-MRI pharmacokinetic analysis. The Tofts' model ((a), also known as Kety's or Larsson's model) [22] models the gadolinium concentration dynamic change in a tissue voxel as the result of convolution between an exponential kernel and a measured or assumed arterial input function (AIF). A plasma term with volume fraction _vp is added in the extended Toft's model (b) [23] by the assumption that the influence of the intravascular compartment is nonnegligible. The Brix's model (c) [24] takes the gadolinium concentration in blood plasma as another unknown variable and solves it with the tissue dynamic curve from a set of ordinary differential equations. In the adjusted Brix's model (d) [25], an oversimplified assumption in the original Brix's model is corrected, leading to a more realistic description of the plasma signal. In the reference region model (e) [26], two dynamic curves are simultaneously measured from the target voxel and a nearby reference region with known ^Ktrans and _ve values. The need for an AIF is eliminated by jointly solving two pharmacokinetic equations for target voxel and reference region.

(a) [figure omitted; refer to PDF]

(b) [figure omitted; refer to PDF]

(c) [figure omitted; refer to PDF]

(d) [figure omitted; refer to PDF]

(e) [figure omitted; refer to PDF]

Heterogeneity in tumor vasculature is subject to change over time. During tumor growth, proliferation of tumor cells is faster than angiogenesis, leading to the formation of poorly perfused hypoxic zones near the tumor center. Rim enhancement is frequently observed in DCE-MRI images of various types of primary cancers, metastases, and xenografts. In the breast, it is a strong indicator of cancer in small lesions and malignancy in larger masses [18]. Other more complicated patterns of enhancement are also common. In general, fast progressing diseases and malignancies are associated with highly heterogeneous enhancement patterns in DCE-MRI images. This feature can be used to facilitate cancer diagnosis and staging. In human gliomas, Rose et al. found that low- and high-grade tumors are significantly different in several spatial heterogeneity statistics of DCE-MRI pharmacokinetic parameters [35, 36]. In breast imaging, different internal enhancement patterns have been identified as an important diagnostic characteristic by the ACR Breast Imaging-Reporting and Data System (BI-RADS) MRI lexicon [18].

In cancer treatment, tumor heterogeneity is usually associated with poor response, a higher chance of developing metastasis, and substantially shorter progression-free and overall survival [37, 38]. The mechanism is probably through the selection of more aggressive tumor cells that can better tolerate the hostile acidic, hypoxic environment in poorly perfused low-enhancing regions [19, 39]. Treatment-induced changes in tumor vascular heterogeneity have been reported in several preclinical and clinical settings. Checkley et al. observed that in human PC-3 prostate adenocarcinoma xenografts receiving ZD6474 treatment, the core region in the xenograft tumor revealed a larger reduction in the pharmacokinetic parameter ^Ktrans than the enhancing rims, suggesting a change in the distribution patterns of perfusion and permeability [40]. De Lussanet et al. found that primary rectal cancer patients receiving radiation therapy have lower intratumoral heterogeneity in the pharmacokinetic parameters ^Ktrans and _vp than patients without radiation therapy [41]. Several groups also explored the possibility of predicting treatment efficacy with tumor vascular heterogeneity and its early change. Johansen et al. investigated the relationship between the distributions of DCE-MRI model-free parameters and patient 5-year survival in a small cohort of 24 breast cancer patients receiving neoadjuvant chemotherapy. They found that patients surviving for more than 5 years have significantly less heterogeneous relative signal intensity (RSI) distribution than nonsurvivors in the baseline DCE-MRI study performed prior to the start of therapy [30]. In a similar study, Padhani et al. correlated early changes in the distribution of DCE-MRI pharmacokinetic parameters with treatment response assessment. Changes in vascular heterogeneity quantified by the range of ^Ktrans distribution at the end of cycle two were found to be predictive of both clinical and histopathologic responses after three to six cycles of therapy [42]. Yuh et al. analyzed serial DCE-MRI studies of 101 patients during early treatment of cervical cancer. Part of their data has low temporal resolution so a complete pharmacokinetic analysis was not possible. Tumor vascular heterogeneity was thus quantified by the lower percentiles of voxel signal intensities in the plateau phase of the dynamic curves. Low values of the 5th to 20th percentiles were found to be correlated with poor primary tumor control and short disease-free survival. The 5th percentile measured at 2-2.5 weeks of therapy provided the best prediction for 8-year primary tumor control [43].

3. Quantifying Tumor Vascular Heterogeneity

For a long time, radiologists identify intratumoral heterogeneity solely by their visual perception of the image. Even though some level of consensus can be achieved with stringent training of readers and standardization of descriptive vocabularies, this approach is still subjective. Many factors, like the imaging hardware, personal bias, or even the physical condition of the reader, can affect the readers' judgement and lead to intra- and interobserver variations in assessment [44]. This problem is now becoming even more meaningful with the development of novel quantitative imaging techniques like DCE-MRI and perfusion CT. These techniques introduce a new dimension of spatially resolved functional information into the data space, which is usually displayed as pseudocolor overlays on the conventional morphologic images. Human color perception, however, is both nonlinear to luminance and nonuniform for different colors. As demonstrated by Giesel et al., reader's perception of intratumoral homogeneity/heterogeneity can easily be exaggerated or understated by simply choosing a different display range or color scheme (Figure 2) [45, 46]. In order to tackle this problem, several alternative methods have been developed by researchers in different branches of science. All these methods are quantitative, seeking to extract a set of well-defined quantities rather than ambiguous verbal descriptions to summarize some key features of tumor vascular heterogeneity.

Visual assessment of tumor vascular heterogeneity on a pseudocolor map is inherently subjective: the same parametric map of the pharmacokinetic parameter ^Ktrans is visualized as pseudocolor overlays with different display ranges and color schemes on a morphologic image in a bladder cancer patient. Compared to the single color scheme ((a), "blackbody radiation"), the multicolor scheme "rainbow" in (b) exaggerates subtle differences in ^Ktrans near color boundaries, creating artificial structures in the pseudocolor map. If the display range is chosen too large, the parametric map is perceived as being more "homogeneous" due to underrepresentation of the spatial variation (c). A small display range, on the other hand, tends to exaggerate the size of the "hotspot", which may lead to an erroneous characterization of the disease (d).

(a) [figure omitted; refer to PDF]

(b) [figure omitted; refer to PDF]

(c) [figure omitted; refer to PDF]

(d) [figure omitted; refer to PDF]

In this section, we review these methods in separate topics. Technological issues and methodological considerations that we deem important in study design and data acquisition and analysis will be discussed in the next section.

3.1. Subregional Region-of-Interests (ROIs)

A simple, straightforward strategy for dealing with intratumoral heterogeneity is to subdivide the tumor into regions with less spatial variability. In the breast, many researchers prefer to analyze a small region with the highest enhancement (i.e., the "hot-spot") instead of the whole tumor. If the hot-spot ROI is small enough and correctly placed, the hot-spot average is equivalent to the maximum of the whole tumor histogram. The hot-spot analysis can be performed semiautomatically, thus reducing the observer-related variability in ROI placement [34, 47]. In an early study, Mussurakis et al. were unable to detect a statistically significant difference in peak enhancement ratio (PER) between 71 malignant and 37 benign lesions, and concluded that hotspot analysis may not provide any diagnostic advantage over the whole-tumor method [33]. However, their result is probably undermined by an inappropriate selection of parameter: it is now known that in DCE-MRI, the most profound difference between malignant and benign breast lesions is not the maximum signal enhancement, but the shape of the entire dynamic enhancement curve. Malignant tumors have a quick uptake of contrast agent. Their dynamic curves are characterized by a fast increase to the peak followed by a slow washout or a stable "plateau". Benign lesions usually have a persistently increasing dynamic curve in which the maximum signal intensity is achieved by the end of the observation window [48]. Liney et al. took this factor into consideration by introducing a new model-free parameter, the maximum-intensity-time-ratio (MITR), that corrects PER with the time to peak enhancement (_Tmax ). Their results from a study of similar size (81 malignant and 36 benign lesions) agreed with Mussurakis et al. conclusion about PER, but also showed that the lesion differentiation ability of the hot-spot analysis is significantly improved over the whole-tumor method when MITR is used [34]. Other parameters that have been demonstrated to be effective in hot-spot analysis include signal enhancement ratio obtained at a fixed time point 60-90 seconds postinjection [49] and the pharmacokinetic parameter ^Ktrans [50].

Inspired by the commonly seen "rim enhancement" pattern in solid tumors, some researchers also segment a tumor into a poorly enhancing "core" and a strongly enhancing "periphery" or "rim" [40, 51]. Benjaminsen et al. developed this rough segmentation methodology into a quantitative approach to describe radial heterogeneity in tumor perfusion [52]. In this method, a tumor parametric map is divided into several concentric bands with approximately the same width. Median or mean parameter values are calculated for each band and plotted against band number or radius. The Rofstad group in Norway used this method to study the radial distribution of tumor perfusion and EES fraction in a series of preclinical studies with human melanoma xenografts A-07 and R-18 between 2004 and 2008 [52-57]. At about the same time, Jia et al. applied a similar method to the model-free parameter washout slope in colorectal cancer liver metastasis, and gave this approach an intuitive name "onion-peeling" [32]. A major advantage of this method is that the radial distribution curve can be normalized and compared between parameters obtained with different assessment techniques. Therefore, it can be used as a validation approach. Graff et al. used this method to compare the radial distributions of the DCE-MRI pharmacokinetic parameter EF and tumor blood supply measured with a freely diffusible radioactive tracer ^Na99m_TcO4 . The close match of radial distribution curves partially validated the authors' hypothesis that EF measures tumor perfusion in these preclinical models [55]. Their results also showed that A-07 and R-18 xenografts are characterized by different radial distributions of EF [55] and λ (a DCE-MRI pharmacokinetic parameter proportional to _ve , [53]), suggesting a potential application in tumor characterization.

It should be noted that the success of the "onion-peeling" method in Rofstad's studies is partly due to their relatively simple preclinical model. Growth of in vivo tumors may be restricted or slowed down on certain directions by the existing anatomical structures, leading to an irregular or diffused tumor shape in which a "center" can hardly be defined. Nonradial heterogeneity patterns are also frequently seen in many human tumors (Figure 3). Therefore, this method is probably working best in small scale preclinical studies with well-controlled tumor growth conditions. In large scale clinical studies, other quantification methods that are less specific about the heterogeneity pattern, such as the texture analysis methods, might be more appropriate.

While the "onion-peeling" method has been demonstrated to be useful in animal xenografts, the specific assumption of radial heterogeneity may not apply to all in vivo human tumors. It can only be used on lesions with regular shape and a radial enhancement pattern (a). The lesion in (b) has an irregular, speculated shape that makes it difficult to define its center. Its enhancement pattern is also more stochastic, without a clear difference between the periphery and the tumor core. Applying the "onion-peeling" method to a tumor like this will not generate much useful information about the underlying tumor biology.

(a) [figure omitted; refer to PDF]

(b) [figure omitted; refer to PDF]

3.2. Histogram Analysis

Intratumoral heterogeneity is also reflected by the shapes of parameter distribution histograms. Tumors are typically characterized by a skewed distribution with a long tail towards strong enhancement or high ^Ktrans values. Multiple peaks may be observed if the tumor contains a significant fraction of poorly perfused hypoxic mass. Disease progression is usually accompanied by an expansion of the histogram to the right, with the peak broadening its width but decreasing its height. A successful treatment, on the contrary, "suppresses" the histogram leftwards, making the peak narrower and more normal [30, 42]. Functional principal component analysis also revealed that antiangiogenic treatment with ZD6474 causes the disappearance of a second peak with high ^Ktrans values in mice bearing PC-3 human prostate adenocarcinoma xenografts [58]. A number of studies have demonstrated that incorporating summary statistics describing histogram width or asymmetry leads to substantial improvement in disease diagnosis and response assessment. In primary breast tumors, both the sensitivity and specificity of malignant/benign lesion differentiation can be improved by adding the ^Ktrans standard deviation information to the conventional analysis approach using ^Ktrans mean alone [59]. In another study, median ^Ktrans was demonstrated to be less sensitive than the range of the ^Ktrans histogram in predicting treatment response in breast cancer patients receiving systemic chemotherapy [42]. Similar results were also reported in cervical cancer patients receiving radiation therapy, where Mayr et al. found that the 10th percentile of RSI is a better predictor for tumor recurrence than the mean or median values [60].

So far, there is no consensus on the optimal method for histogram analysis. Some researchers perform a bin-to-bin comparison between two normalized histograms [61]. Although the rationale for this approach is straightforward to understand, it is in general an inefficient technique: a certain number of bins are needed for the histogram to accurately reflect the shape of the underlying distribution. In clinical research, this number is typically between 10 and 30. When all these bins are simultaneously compared, the need for multicomparison correction makes the whole test very insensitive. For example, if an overall significance level of 0.05 is assigned to a comparison between two normalized histograms with 20 bins each, the individual significance level for every single bin will be only 0.05/20 = 0.0025 with the basic Bonferroni correction [62]. With such a low individual significance level, subtle changes in the histogram are very likely to be masked.

A better strategy for histogram analysis is to summarize only the most relevant information into a few descriptive quantities. Almost all summary statistics have been used in the literature to describe the distribution histogram. A partial list includes the range [42], mean, median, standard deviation [59], skewness [9], kurtosis [63], entropy [63], and various percentiles [30, 41, 43, 60, 64]. There is no decisive conclusion about which statistics is the best. However, the percentiles are probably more robust than the range and moment statistics on pharmacokinetic parametric maps. An "allowed range" is usually set for each pharmacokinetic parameter by the iterative fitting algorithm to accelerate convergence. If the true range of the actual distribution is wider than this "allowed range", the long tails outside the allowed range will be truncated and replaced by artificial "peaks" on the maximum or minimum allowed values. Unlike the range and moment statistics, the percentiles are insensitive to this artificial distortion of histogram shape because they are not affected by changes in a few extreme values.

3.3. Texture Analysis

The histogram analysis only looks at the distribution of individual voxels within a tumor ROI. The spatial correlation between voxels is ignored. Theoretically, a tumor with an enhancing rim could have an identical histogram with a tumor having many enhancing foci scattered over the entire mass. Spatial distribution information is needed to better differentiate lesions with similar histogram but distinctive enhancement patterns. A number of studies have been conducted along this direction using image texture analysis techniques [31, 65-70].

In DCE-MRI data analysis, the most frequently used texture analysis technique is the cooccurrence matrix method first described by Haralick et al. [71]. Briefly speaking, the cooccurrence matrix element _Pd,θ (i,j) measures the probability of starting from any voxel with value i in an image, moving d voxels along direction θ , and arriving at another voxel with value j . Therefore, the cooccurrence matrix is a two dimensional histogram describing a joint distribution of all possible moves with step size d and direction θ on the image. Textural features, such as contrast and correlation, can be exacted from this 2D histogram. They are then averaged over four directions (θ=0 °, 45°, 90°, and 135°) to remove angular dependency. In medical imaging studies, the step size d is usually taken as 1 voxel. The voxel values are usually discretized to reduce cooccurrence matrix dimension. Sinha et al. used this technique to obtain eight textural features that made a reasonably good discrimination (specificity 70%, sensitivity 75%) between 20 malignant and 23 benign breast lesions [65]. Chen et al. extended Haralick et al. 2D technique to 3D, and achieved a significant improvement in classification performance using features obtained from a volumetric texture analysis in a cohort of 77 malignant and 44 benign lesions [66]. Positive results were also reported with various model-free [31] and pharmacokinetic parametric maps [67] as well as on four dimensional dynamic series [68]. Nie et al. further associated the textural features (gray level entropy and sum average) with the visual descriptors in BI-RADS lexicon (enhancement intensity and internal enhancement patterns), thus giving the highly mathematical approach an intuitive interpretation [69].

Currently, the major problem associated with DCE-MRI texture analysis is the high dimensionality of the feature space. A more detailed discussion of this issue will be given in the next section. Researchers are encouraged to adopt a concise study design and dimensionality reduction techniques to improve the reliability of their conclusions.

3.4. Spatial-Parametric Hypervolume Geometry

An interesting approach of heterogeneity quantification was described in the exploratory works of Rose et al. [35, 36]. In their approach, DCE-MRI pharmacokinetic parameters were not treated in the conventional way as voxel values in a 3D imaging volume. Instead, they were interpreted as the height on a fourth dimension in an "extruded" 4D binary hypervolume. Geometric features like (hyper-)surface area, (hyper-)volume, (hyper-)surface to (hyper-)volume ratio, and box-count fractal dimension can be calculated for this spatial-parametric hypervolume, and used as quantitative descriptors of tumor vascular heterogeneity [35]. In a recent study, the Rényi family of fractal dimensions was also investigated [36]. Although the small sample sizes prevented any decisive conclusion about its value to be made at this moment, initial results in glioma grading and liver metastasis treatment response seem to be promising.

This method is conceptually interesting in that spatial and parametric information are treated equally as dimensions in the 4D hypervolume. Geometric features of the hypervolume thereby describe variations in both spatial and parametric domains by nature. BI-RADS lexicon suggests that both morphologic and heterogeneity information are critical in diagnosis of breast cancer. This is perhaps true for all solid tumors. Therefore, geometric features of the spatial-parametric hypervolume compose a pool of potential biomarkers that can simultaneously quantify tumor morphology and heterogeneity. Such a biomarker, if identified, may have important uses in tumor diagnosis and response prediction because it can improve the robustness of the classifier/predictor through reduction of input space dimensionality.

4. Technological and Methodological Considerations

4.1. Avoid Data Sparsity in Study Design

Many descriptive and textural features of imaging data can be extracted from the analysis of tumor heterogeneity. Some researchers tend to use a large number of features in their studies. In an extreme example we found more than 600 features were investigated in only 23 patients [70]. While the researchers' desire of getting the maximum yield from their data is understandable, this practice is fundamentally problematic because as the number of features increases, the problem of data sparsity arises due to Bellman's "curse of dimensionality" [72]. A sparsely-distributed sample set cannot provide adequate information to fully represent the complex behaviors in a large population. Consequently, any prediction made from such a sample set is highly unstable and thus practically useless [73]. This problem is probably responsible for our finding that some complex discriminators are claimed to have (artificially) excellent performance in a relatively small patient population but never develop into real clinical application.

There is no real remedy for the sparse data problem except for thorough consideration and careful planning in the designing stage of a study. The number of features used for classification or prediction must be strictly controlled from the very beginning. A rule of thumb is to keep the ratio between the number of patients in each class and the number of features above 10 [74], that is, for a typical study of about 100 patients divided into two groups, the number of features that can be investigated should not exceed five. This stringent restriction requires the investigators to include only the most promising features into their analysis rather than use all the parameters they can get unselectively. Of course, information about the other less promising features is also interesting and may lead to important discoveries. We recommend to report those as exploratory data. No effort should be made to get any statistical inference from these data. In this way, these data can be preserved without harming the statistical power of the whole study. They may serve as part of the raw data set for future metaanalyses which may provide statistically meaningful information that can never be derived from any single study.

Some researchers chose to classify individual voxels instead of the whole tumor or subject [67, 68, 75]. While this method does help to greatly reduce the stringent restriction on feature set size, caution is needed in the interpretation of the results. Simply stating that a voxel has a "responder-like" curve or "benign" parameters is superficial and sometimes misleading. It is more preferable to characterize voxels with imaging features that have well-recognized association with tumor biology on cellular or tissue levels. Correlation with other established biomarkers is also important when such information is available.

Dimensionality reduction techniques are also powerful tools that may help the investigators to fight the "curse of dimensionality" [74]. It should be noted that dimensionality reduction techniques are only supplements to, but not replacement for, careful study design and successful dimensionality control. They should only be applied to a small feature set carefully selected according to the rule of thumb mentioned above. Applying a dimensionality reduction technique to a poorly designed study would not add any strength to its conclusions.

4.2. Need for Standardization in DCE-MRI Data Acquisition and Analysis

Currently there is no "standard" way of performing DCE-MRI. A number of methodologies have been developed by different groups. Some of them use a _T1 -weighted fast low-angle shot (FLASH) sequence [76], while the others use saturation recovery or inversion recovery techniques [77]. The measured signal intensities may be converted to gadolinium concentration through the use of a calibration phantom [78], calculation of relative signal enhancement [79], or precontrast _T1 [80] or proton density measurements [81]. Data analysis methods are about equally variable, too. Many model-free and pharmacokinetic parameters have been described in the literature (Tables 1 and 2). Most model-free parameters do not have a clear definition that is strictly followed within the DCE-MRI community. For example, many investigators reported the model-free parameter area-under-the-curve (AUC) in their studies, but the way they measured it can be quite different: in some studies it was measured from the relative signal enhancement dynamic curve, while in the others the gadolinium concentration dynamic curve was used. The measurement time can also vary from 30 to 180 seconds postinjection. Even though pharmacokinetic parameters are generally better defined than model-free parameters, considerable variability still exists in the choice of the pharmacokinetic models [22-26] and deconvolution techniques [82, 83].

It is known that many factors in data acquisition and analysis can substantially affect the reliability of results in DCE-MRI studies. The accuracy and precision of the pharmacokinetic parameter estimates, for example, are largely determined by the signal-to-noise ratio (SNR) and temporal resolution of the MR sequence [84]. Systematic errors can be introduced from various sources, such as voluntary or involuntary patient motion, inappropriate use of postprocessing filters, inhomogeneity in _B1 excitation field and receiver coil sensitivity, breakdown of linearity in the gadolinium concentration calibration curve [85], and unequal definitions of the same pharmacokinetic parameter (such as ^Ktrans ) in different models [22, 23]. Therefore, the methodological variability described above actually contains a large number of potential error sources. Their complex yet poorly understood interaction makes it extremely difficult to compare DCE-MRI data from different sources. This problem is especially important in the study of tumor vascular heterogeneity. As we have mentioned in the discussions above, an individual study can only investigate a very small fraction of all parameters that characterize intratumoral heterogeneity. The only way to accelerate the investigation process is through meta-analysis that combines results from different studies. Obviously, this can only be achieved with high quality data acquired and analyzed following an optimized standard procedure.

A standardization effort is currently underway driven by the DCE-MRI subcommittee of the Radiological Society of North American (RSNA) Quantitative Imaging Biomarkers Alliance (QIBA) initiative [86]. The most recent progress of their collaborative effort is a comprehensive technical profile that specifies basic standards in DCE-MRI data acquisition, analysis, and quality control on 1.5 Tesla systems. It is recommended that DCE-MRI should be performed on a clinical scanner qualified with a standard phantom, using a 3D FLASH or equivalent sequence without any magnetization preparation or postprocessing filter. The image intensity should be converted to gadolinium concentration through the use of a variable flip angle precontrast _T1 map [87] after necessary corrections for patient motion and inhomogeneity in _B1 field or coil sensitivity. The pharmacokinetic parameter ^Ktrans (estimated with the standard Tofts' model) and model-free parameter IAUGC (AUC measured from the gadolinium concentration curve up to 90 seconds postinjection) are recommended as standard quantitative endpoints that should be employed in clinical trials and practice. Guidelines on the minimally acceptable and ideal ranges of various scan parameters (like temporal resolution and slice thickness) are also provided. Investigators in this area are encouraged to follow the progress of the QIBA initiative through journal publications and online updates (http://qibawiki.rsna.org/index.php?title=DCE-MRI_subctte), and adopt the most up-to-date consensus to their own studies to ensure their data have the highest quality standard and the maximum level of comparability.

An important issue that is not addressed in the present version of QIBA DCE-MRI profile is the lack of quality control in the vascular parameter fitting stage. Many researchers only report the "best fit" estimates for pharmacokinetic parameters and ignore the inherent uncertainty in those estimates. This practice leads to a substantial underestimation of the variability in pharmacokinetic parameters. It also raises a philosophical question whether it is appropriate to treat reliable and unreliable parameter estimates equally in the assessment of tumor heterogeneity. Hence, we suggest that DCE-MRI researchers should always report the asymptotic confidence intervals with the "best fit" estimates for pharmacokinetic parameters, and adopt an exclusion criterion or weighting scheme that controls the influence of unreliable estimates from noisy data. We also expect that as the awareness of this issue is raised, a consensus will be developed through the collective effort of the whole DCE-MRI community, and will be included in future standards.

4.3. Validation

It is difficult to validate DCE-MRI parameters because they measure a mixed effect of blood flow, microvascular permeability, and EES volume. A number of studies have demonstrated broad correlations between various DCE-MRI parameters and tissue perfusion [88], microvascular density (MVD) [45, 89], or angiogenic factor expression [90]. However, controversial results are also reported in the literature [91-93]. One reason for this controversy might be tumor heterogeneity at the subvoxel level. In a human cervical carcinoma xenograft study, matched DCE-MRI images and histological sections showed substantial intravoxel heterogeneity in morphology and tissue composition, even though the voxel size (0.23 × 0.47 × 2.0 mm³ ) is already ten times smaller than those used in human studies [94]. Therefore, DCE-MRI parameters may be better validated against immunohistochemical measurements averaged over several fields randomly picked from the region of interest.

Beside the ROI-averaged parameter values, the distribution of DCE-MRI parameters also needs to be validated in studies of tumor vascular heterogeneity. This task is relatively simple if the "gold standard" is another imaging modality with similar spatial resolution, for which direct comparison can be made between summary statistics [95], histogram shapes, or certain heterogeneity patterns [52, 55]. MVD values or angiogenic factor expression levels need to be compared with DCE-MRI parameters within multiple subregional ROIs or along a line profile. Special techniques, like MRI-guided biopsy [96], may be used to help reduce sampling error caused by mismatch between DCE-MRI images and histological slides.

5. Conclusion

Experimental evidence shows that vascular heterogeneity plays an important role in tumor biology. Several methods have been developed to quantify tumor vascular heterogeneity with DCE-MRI. Initial studies in patients and animal models have shown promising results in disease diagnosis, tumor grading, and response assessment. These results give us additional understanding of tumor biology and justify the necessity of continuous development along this direction. However, the quality of a DCE-MRI heterogeneity study can easily be undermined by various methodological and technological issues, such as poor study design or lack of standardization. This paper may serve as a guideline to help investigators understand and also avoid these possible pitfalls in study design, data acquisition, and analysis in future studies of tumor vascular heterogeneity.

Acknowledgments

The authors are grateful to Dr. Guang Jia for his help in creating Figure 2. This work was supported by grant ODOD AGMT TECH 03-051 and the Wright Center of Innovation in Biomedical Imaging.

References

[1] R. K. Jain, D. G. Duda, C. G. Willett, D. V. Sahani, A. X. Zhu, J. S. Loeffler, T. T. Batchelor, A. G. Sorensen, "Biomarkers of response and resistance to antiangiogenic therapy," Nature Reviews Clinical Oncology , vol. 6, no. 6, pp. 327-338, 2009.

[2] S. Rehman, G. C. Jayson, "Molecular imaging of antiangiogenic agents," Oncologist , vol. 10, no. 2, pp. 92-103, 2005.

[3] J. S. Taylor, P. S. Tofts, R. Port, J. L. Evelhoch, M. Knopp, W. E. Reddick, V. M. Runge, N. Mayr, "MR imaging of tumor microcirculation: promise for the new millenium," Journal of Magnetic Resonance Imaging , vol. 10, no. 6, pp. 903-907, 1999.

[4] A. J. de Langen, V. E. M. van den Boogaart, J. T. Marcus, M. Lubberink, "Use of _H2 ¹⁵ O -PET and DCE-MRI to measure tumor blood flow," Oncologist , vol. 13, no. 6, pp. 631-644, 2008.

[5] M. Cao, Y. Liang, C. Shen, K. D. Miller, K. M. Stantz, "Developing DCE-CT to quantify intra-tumor heterogeneity in breast tumors with differing angiogenic phenotype," IEEE Transactions on Medical Imaging , vol. 28, no. 6, pp. 861-871, 2009.

[6] J. T. Bushberg The Essential Physics of Medical Imaging , Lippincott Williams & Wilkins, Philadelphia, Pa, USA, 2002., 2nd.

[7] R. K. Jain, "Normalization of tumor vasculature: an emerging concept in antiangiogenic therapy," Science , vol. 307, no. 5706, pp. 58-62, 2005.

[8] J. P. B. O'Connor, A. Jackson, G. J. M. Parker, G. C. Jayson, "DCE-MRI biomarkers in the clinical evaluation of antiangiogenic and vascular disrupting agents," British Journal of Cancer , vol. 96, no. 2, pp. 189-195, 2007.

[9] A. Jackson, J. P. B. O'Connor, G. J. M. Parker, G. C. Jayson, "Imaging tumor vascular heterogeneity and angiogenesis using dynamic contrast-enhanced magnetic resonance imaging," Clinical Cancer Research , vol. 13, no. 12, pp. 3449-3459, 2007.

[10] B. Morgan, A. L. Thomas, J. Drevs, J. Hennig, M. Buchert, A. Jivan, M. A. Horsfield, K. Mross, H. A. Ball, L. Lee, W. Mietlowski, S. Fuxius, C. Unger, K. O'Byrne, A. Henry, G. R. Cherryman, D. Laurent, M. Dugan, D. Marmé, W. P. Steward, "Dynamic contrast-enhanced magnetic resonance imaging as a biomarker for the pharmacological response of PTK787/ZK 222584, an inhibitor of the vascular endothelial growth factor receptor tyrosine kinases, in patients with advanced colorectal cancer and liver metastases: results from two phase I studies," Journal of Clinical Oncology , vol. 21, no. 21, pp. 3955-3964, 2003., [email protected]

[11] K. Mross, J. Drevs, M. Müller, M. Medinger, D. Marmé, J. Hennig, B. Morgan, D. Lebwohl, E. Masson, YU. Y. Ho, C. Günther, D. Laurent, C. Unger, "Phase I clinical and pharmacokinetic study of PTK/ZK, a multiple VEGF receptor inhibitor, in patients with liver metastases from solid tumours," European Journal of Cancer , vol. 41, no. 9, pp. 1291-1299, 2005.

[12] N. Weidner, J. P. Semple, W. R. Welch, J. Folkman, "Tumor angiogenesis and metastasis--Correlation in invasive breast carcinoma," The New England Journal of Medicine , vol. 324, no. 1, pp. 1-8, 1991.

[13] A. M. Schor, S. Pazouki, J. Morris, R. L. Smither, L. M. Chandrachud, N. Pendleton, "Heterogeneity in microvascular density in lung tumours: comparison with normal bronchus," British Journal of Cancer , vol. 77, no. 6, pp. 946-951, 1998.

[14] M. T. Wyss, S. Hofer, M. Hefti, E. Bärtschi, C. Uhlmann, V. Treyer, U. Roelcke, "Spatial heterogeneity of low-grade gliomas at the capillary level: a PET study on tumor blood flow and amino acid uptake," Journal of Nuclear Medicine , vol. 48, no. 7, pp. 1047-1052, 2007.

[15] G. H. Heppner, "Tumor heterogeneity," Cancer Research , vol. 44, no. 6, pp. 2259-2265, 1984.

[16] M. Shipitsin, L. L. Campbell, P. Argani, S. Weremowicz, N. Bloushtain-Qimron, J. Yao, T. Nikolskaya, T. Serebryiskaya, R. Beroukhim, M. Hu, M. K. Halushka, S. Sukumar, L. M. Parker, K. S. Anderson, L. N. Harris, J. E. Garber, A. L. Richardson, S. J. Schnitt, Y. Nikolsky, R. S. Gelman, K. Polyak, "Molecular definition of breast tumor heterogeneity," Cancer Cell , vol. 11, no. 3, pp. 259-273, 2007.

[17] P. Carmeliet, R. K. Jain, "Angiogenesis in cancer and other diseases," Nature , vol. 407, no. 6801, pp. 249-257, 2000.

[18] G. Agrawal, M. Y. Su, O. Nalcioglu, S. A. Feig, J. H. Chen, "Significance of breast lesion descriptors in the ACR BI-RADS MRI lexicon," Cancer , vol. 115, no. 7, pp. 1363-1380, 2009.

[19] S. Chuthapisith, J. Eremin, M. El-Sheemey, O. Eremin, "Breast cancer chemoresistance: emerging importance of cancer stem cells," Surgical Oncology , vol. 19, no. 1, pp. 27-32, 2010.

[20] R. J. Gilbertson, J. N. Rich, "Making a tumour's bed: glioblastoma stem cells and the vascular niche," Nature Reviews Cancer , vol. 7, no. 10, pp. 733-736, 2007.

[21] J. B. Sneddon, Z. Werb, "Location, location, location: the cancer stem cell niche," Cell Stem Cell , vol. 1, no. 6, pp. 607-611, 2007.

[22] H. B. Larsson, M. Stubgaard, L. Søndergaard, O. Henriksen, "In vivo quantification of the unidirectional influx constant for Gd-DTPA diffusion across the myocardial capillaries with MR imaging," Journal of Magnetic Resonance Imaging , vol. 4, no. 3, pp. 433-440, 1994.

[23] P. S. Tofts, "Modeling tracer kinetics in dynamic Gd-DTPA MR imaging," Journal of Magnetic Resonance Imaging , vol. 7, no. 1, pp. 91-101, 1997.

[24] G. Brix, W. Semmler, R. Port, L. R. Schad, G. Layer, W. J. Lorenz, "Pharmacokinetic parameters in CNS Gd-DTPA enhanced MR imaging," Journal of Computer Assisted Tomography , vol. 15, no. 4, pp. 621-628, 1991.

[25] X. Yang, J. Liang, J. T. Heverhagen, G. Jia, P. Schmalbrock, S. Sammet, R. Koch, M. V. Knopp, "Improving the pharmacokinetic parameter measurement in dynamic contrast-enhanced MRI by use of the arterial input function: theory and clinical application," Magnetic Resonance in Medicine , vol. 59, no. 6, pp. 1448-1456, 2008.

[26] T. E. Yankeelov, J. J. Luci, M. Lepage, R. Li, L. Debusk, P. C. Lin, R. R. Price, J. C. Gore, "Quantitative pharmacokinetic analysis of DCE-MRI data without an arterial input function: a reference region model," Magnetic Resonance Imaging , vol. 23, no. 4, pp. 519-529, 2005.

[27] E. J. Goldstein, K. R. Burnett, J. R. Hansell, J. Casaia, J. Dizon, B. Farrar, D. Gelblum, G. L. Wolf, "Gadolinium DTPA (an NMR proton imaging contrast agent): chemical structure, paramagnetic properties and pharmacokinetics," Physiological Chemistry and Physics and Medical NMR , vol. 16, no. 2, pp. 97-104, 1984.

[28] P. S. Tofts, G. Brix, D. L. Buckley, J. L. Evelhoch, E. Henderson, M. V. Knopp, H. B. W. Larsson, T. Y. Lee, N. A. Mayr, G. J. M. Parker, R. E. Port, J. Taylor, R. M. Weisskoff, "Estimating kinetic parameters from dynamic contrast-enhanced T₁ -weighted MRI of a diffusable tracer: standardized quantities and symbols," Journal of Magnetic Resonance Imaging , vol. 10, no. 3, pp. 223-232, 1999.

[29] J. L. Evelhoch, "Key factors in the acquisition of contrast kinetic data for oncology," Journal of Magnetic Resonance Imaging , vol. 10, no. 3, pp. 254-259, 1999.

[30] R. Johansen, L. R. Jensen, J. Rydland, P. E. Goa, K. A. Kvistad, T. F. Bathen, D. E. Axelson, S. Lundgren, I. S. Gribbestad, "Predicting survival and early clinical response to primary chemotherapy for patients with locally advanced breast cancer using DCE-MRI," Journal of Magnetic Resonance Imaging , vol. 29, no. 6, pp. 1300-1307, 2009.

[31] A. Karahaliou, K. Vassiou, N. S. Arikidis, S. Skiadopoulos, T. Kanavou, L. Costaridou, "Assessing heterogeneity of lesion enhancement kinetics in dynamic contrast-enhanced MRI for breast cancer diagnosis," British Journal of Radiology , vol. 83, no. 988, pp. 296-306, 2010.

[32] G. Jia, C. O'Dell, J. T. Heverhagen, X. Yang, J. Liang, R. V. Jacko, S. Sammet, T. Pellas, P. Cole, M. V. Knopp, "Colorectal liver metastases: contrast agent diffusion coefficient for quantification of contrast enhancement heterogeneity at MR imaging," Radiology , vol. 248, no. 3, pp. 901-909, 2008.

[33] S. Mussurakis, P. Gibbs, A. Horsman, "Primary breast abnormalities: selective pixel sampling on dynamic gadolinium enhanced MR images," Radiology , vol. 206, no. 2, pp. 465-473, 1998.

[34] G. P. Liney, P. Gibbs, C. Hayes, M. O. Leach, L. W. Turnbull, "Dynamic contrast-enhanced MRI in the differentiation of breast tumors: user-defined versus semi-automated region-of-interest analysis," Journal of Magnetic Resonance Imaging , vol. 10, no. 6, pp. 945-949, 1999.

[35] C. J. Rose, S. Mills, J. P. O'Connor, G. A. Buonaccorsi, C. Roberts, Y. Watson, B. Whitcher, G. Jayson, A. Jackson, G. J. Parker, "Quantifying heterogeneity in dynamic contrast-enhanced MRI parameter maps," Medical Image Computing and Computer-Assisted Intervention , vol. 10, no. 2, pp. 376-384, 2007.

[36] C. J. Rose, S. J. Mills, J. P. B. O'Connor, G. A. Buonaccorsi, C. Roberts, Y. Watson, S. Cheung, S. Zhao, B. Whitcher, A. Jackson, G. J. M. Parker, "Quantifying spatial heterogeneity in dynamic contrast-enhanced MRI parameter maps," Magnetic Resonance in Medicine , vol. 62, no. 2, pp. 488-499, 2009.

[37] L. Simpson-Herren, P. E. Noker, S. D. Wagoner, "Variability of tumor response to chemotherapy II. Contribution of tumor heterogeneity," Cancer Chemotherapy and Pharmacology , vol. 22, no. 2, pp. 131-136, 1988.

[38] F. L. Giesel, H. Bischoff, H. von Tengg-Kobligk, M. A. Weber, C. M. Zechmann, H. U. Kauczor, M. V. Knopp, "Dynamic contrast-enhanced mri of malignant pleural mesothelioma: a feasibility study of noninvasive assessment, therapeutic follow-up, and possible predictor of improved outcome," Chest , vol. 129, no. 6, pp. 1570-1576, 2006.

[39] L. L. Campbell, K. Polyak, "Breast tumor heterogeneity: cancer stem cells or clonal evolution?," Cell Cycle , vol. 6, no. 19, pp. 2332-2338, 2007.

[40] D. Checkley, J. J. Tessier, J. Kendrew, J. C. Waterton, S. R. Wedge, "Use of dynamic contrast-enhanced MRI to evaluate acute treatment with ZD6474, a VEGF signalling inhibitor, in PC-3 prostate tumours," British Journal of Cancer , vol. 89, no. 10, pp. 1889-1895, 2003.

[41] Q. G. de Lussanet, W. H. Backes, A. W. Griffioen, A. R. Padhani, C. I. Baeten, A. van Baardwijk, P. Lambin, G. L. Beets, J. M. A. van Engelshoven, R. G. H. Beets-Tan, "Dynamic contrast-enhanced magnetic resonance imaging of radiation therapy-induced microcirculation changes in rectal cancer," International Journal of Radiation Oncology Biology Physics , vol. 63, no. 5, pp. 1309-1315, 2005.

[42] A. R. Padhani, C. Hayes, L. Assersohn, T. Powles, A. Makris, J. Suckling, M. O. Leach, J. E. Husband, "Prediction of clinicopathologic response of breast cancer to primary chemotherapy at contrast-enhanced mr imaging: initial clinical results," Radiology , vol. 239, no. 2, pp. 361-374, 2006.

[43] W. T. C. Yuh, N. A. Mayr, D. Jarjoura, D. Wu, J. C. Grecula, S. S. Lo, S. M. Edwards, V. A. Magnotta, S. Sammet, H. Zhang, J. F. Montebello, J. Fowler, M. V. Knopp, J. Z. Wang, "Predicting control of primary tumor and survival by DCE MRI during early therapy in cervical cancer," Investigative Radiology , vol. 44, no. 6, pp. 343-350, 2009.

[44] S. Mussurakis, D. L. Buckley, A. M. Coady, L. W. Turnbull, A. Horsman, "Observer variability in the interpretation of contrast enhanced MRI of the breast," British Journal of Radiology , vol. 69, no. 827, pp. 1009-1016, 1996.

[45] F. L. Giesel, P. L. Choyke, A. Mehndiratta, C. M. Zechmann, H. von Tengg-Kobligk, K. Kayser, H. Bischoff, C. Hintze, S. Delorme, M. A. Weber, M. Essig, H. U. Kauczor, M. V. Knopp, "Pharmacokinetic analysis of malignant pleural mesothelioma-initial results of tumor microcirculation and its correlation to microvessel density (CD-34)," Academic Radiology , vol. 15, no. 5, pp. 563-570, 2008.

[46] D. Borland, R. M. Taylor, "Rainbow color map (still) considered harmful," IEEE Computer Graphics and Applications , vol. 27, no. 2, pp. 14-17, 2007.

[47] S. Mussurakis, D. L. Buckley, A. Horsman, "Dynamic MRI of invasive breast cancer: assessment of three region-of-interest analysis methods," Journal of Computer Assisted Tomography , vol. 21, no. 3, pp. 431-438, 1997.

[48] L. W. Turnbull, "Dynamic contrast-enhanced MRI in the diagnosis and management of breast cancer," NMR in Biomedicine , vol. 22, no. 1, pp. 28-39, 2009.

[49] I. S. Gribbestad, G. Nilsen, H. E. Fjøsne, S. Kvinnsland, O. A. Haugen, P. A. Rinck, "Comparative signal intensity measurements in dynamic gadolinium-enhanced MR mammography," Journal of Magnetic Resonance Imaging , vol. 4, no. 3, pp. 477-480, 1994.

[50] C. Hayes, A. R. Padhani, M. O. Leach, "Assessing changes in tumour vascular function using dynamic contrast-enhanced magnetic resonance imaging," NMR in Biomedicine , vol. 15, no. 2, pp. 154-163, 2002.

[51] L. Jiang, D. Zhao, A. Constantinescu, R. P. Mason, "Comparison of BOLD contrast and Gd-DTPA dynamic contrast-enhanced imaging in rat prostate tumor," Magnetic Resonance in Medicine , vol. 51, no. 5, pp. 953-960, 2004.

[52] I. C. Benjaminsen, B. A. Graff, K. G. Brurberg, E. K. Rofstad, "Assessment of tumor blood perfusion by high-resolution dynamic contrast-enhance MRI: a preclinical study of human melanoma xenografts," Magnetic Resonance in Medicine , vol. 52, no. 2, pp. 269-276, 2004.

[53] I. C. Benjaminsen, K. G. Brurberg, E. B. M. Ruud, E. K. Rofstad, "Assessment of extravascular extracellular space fraction in human melanoma xenografts by DCE-MRI and kinetic modeling," Magnetic Resonance Imaging , vol. 26, no. 2, pp. 160-170, 2008.

[54] B. A. Graff, I. C. Benjaminsen, E. A. Melås, K. G. Brurberg, E. K. Rofstad, "Changes in intratumor heterogeneity in blood perfusion in intradermal human melanoma xenografts during tumor growth assessed by DCE-MRI," Magnetic Resonance Imaging , vol. 23, no. 9, pp. 961-966, 2005.

[55] B. A. Graff, I. C. Benjaminsen, K. G. Brurberg, E. B. M. Ruud, E. K. Rofstad, "Comparison of tumor blood by perfusion assessed by dynamic contrast-enhanced MRI with tumor blood supply assessed by invasive imaging," Journal of Magnetic Resonance Imaging , vol. 21, no. 3, pp. 272-281, 2005.

[56] J. V. Gaustad, I. C. Benjaminsen, B. A. Graff, K. G. Brurberg, E. B. M. Ruud, E. K. Rofstad, "Intratumor heterogeneity in blood perfusion in orthotopic human melanoma xenografts assessed by dynamic contrast-enhanced magnetic resonance imaging," Journal of Magnetic Resonance Imaging , vol. 21, no. 6, pp. 792-800, 2005.

[57] K. G. Brurberg, I. C. Benjaminsen, L. M. R. Dørum, E. K. Rofstad, "Fluctuations in tumor blood perfusion assessed by dynamic contrast-enhanced MRI," Magnetic Resonance in Medicine , vol. 58, no. 3, pp. 473-481, 2007.

[58] E. O'Connor, N. Fieller, A. Holmes, J. Waterton, "How to analyse dynamic MRI in oncology: advanced histogram analysis gives better statistical power & insight than simple averaging," in Proceedings of the 13th Scientific Meeting & Exhibition (ISMRM '05), Miami, Fla, USA, May 2005.

[59] B. Issa, D. L. Buckley, L. W. Turnbull, "Heterogeneity analysis of Gd-DTPA uptake: improvement in breast lesion differentiation," Journal of Computer Assisted Tomography , vol. 23, no. 4, pp. 615-621, 1999.

[60] N. A. Mayr, W. T. C. Yuh, J. C. Arnholt, J. C. Ehrhardt, J. I. Sorosky, V. A. Magnotta, K. S. Berbaum, W. Zhen, A. C. Paulino, L. W. Oberley, A. K. Sood, J. M. Buatti, "Pixel analysis of MR perfusion imaging in predicting radiation therapy outcome in cervical cancer," Journal of Magnetic Resonance Imaging , vol. 12, no. 6, pp. 1027-1033, 2000.

[61] Y. Watson, S. Cheung, C. Roberts, "Prognostic power of DCE-MRI heterogeneity analysis in patient with advanced solid tumours," in Proceedings of the 14th Scientific Meeting & Exhibition (ISMRM '06), Seattle, Wash, USA, August 2006.

[62] P. Armitage, G. Berry Statistical Methods in Medical Research , Blackwell Science, Cambridge, UK, 1994., 3rd.

[63] G. Ertas, H. Ö. Gülçür, M. Tunaci, "Improved lesion detection in MR mammography: three-dimensional segmentation, moving voxel sampling, and normalized maximum intensity-time ratio entropy," Academic Radiology , vol. 14, no. 2, pp. 151-161, 2007.

[64] N. A. Mayr, J. Z. Wang, D. Zhang, J. F. Montebello, J. C. Grecula, S. S. Lo, J. M. Fowler, W. T. C. Yuh, "Synergistic effects of hemoglobin and tumor perfusion on tumor control and survival in cervical cancer," International Journal of Radiation Oncology Biology Physics , vol. 74, no. 5, pp. 1513-1521, 2009.

[65] S. Sinha, F. A. Lucas-Quesada, N. D. DeBruhl, J. Sayre, D. Farria, D. P. Gorczyca, L. W. Bassett, "Multifeature analysis of Gd-enhanced MR images of breast lesions," Journal of Magnetic Resonance Imaging , vol. 7, no. 6, pp. 1016-1026, 1997.

[66] W. Chen, M. L. Giger, H. Li, U. Bick, G. M. Newstead, "Volumetric texture analysis of breast lesions on contrast-enhanced magnetic resonance images," Magnetic Resonance in Medicine , vol. 58, no. 3, pp. 562-571, 2007.

[67] M. C. Kale, B. D. Clymer, R. M. Koch, J. T. Heverhagen, S. Sammet, R. Stevens, M. V. Knopp, "Multispectral co-occurrence with three random variables in dynamic contrast enhanced magnetic resonance imaging of breast cancer," IEEE Transactions on Medical Imaging , vol. 27, no. 10, pp. 1425-1431, 2008.

[68] B. J. Woods, B. D. Clymer, T. Kurc, J. T. Heverhagen, R. Stevens, A. Orsdemir, O. Bulan, M. V. Knopp, "Malignant-lesion segmentation using 4D co-occurrence texture analysis applied to dynamic contrast-enhanced magnetic resonance breast image data," Journal of Magnetic Resonance Imaging , vol. 25, no. 3, pp. 495-501, 2007.

[69] K. Nie, J. H. Chen, H. J. Yu, Y. Chu, O. Nalcioglu, M. Y. Su, "Quantitative analysis of lesion morphology and texture features for diagnostic prediction in breast MRI," Academic Radiology , vol. 15, no. 12, pp. 1513-1525, 2008.

[70] J. W. Prescott, D. Zhang, J. Z. Wang, N. A. Mayr, W. T. C. Yuh, J. Saltz, M. Gurcan, "Temporal analysis of tumor heterogeneity and volume for cervical cancer treatment outcome prediction: preliminary evaluation," Journal of Digital Imaging , vol. 23, no. 3, pp. 342-357, 2010.

[71] R. M. Haralick, K. Shanmugam, I. Dinstein, "Textural features for image classification," IEEE Transactions on Systems, Man and Cybernetics , vol. 3, no. 6, pp. 610-621, 1973.

[72] R. Bellman Dynamic Programming , Princeton University Press, Princeton, NJ, USA, 1957.

[73] T. Hastie, R. Tibshirani, J. H. Friedman The Elements of Statistical Learning Data Mining, Inference, and Prediction , Springer, New York, NY, USA, 2001.

[74] A. K. Jain, R. P. W. Duin, J. Mao, "Statistical pattern recognition: a review," IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 22, no. 1, pp. 4-37, 2000.

[75] S. H. Lee, J. H. Kim, K. G. Kim, J. S. Park, S. J. Park, W. K. Moon, "Optimal clustering of kinetic patterns on malignant breast lesions: comparison between K-means clustering and three-time-points method in dynamic contrast-enhanced MRI," in Proceedings of the 29th Annual International Conference of IEEE Engineering in Medicine and Biology Society (EMBS '07), pp. 2089-2093, Lyon, France, August 2007.

[76] S. M. Galbraith, M. A. Lodge, N. J. Taylor, G. J. S. Rustin, S. Bentzen, J. J. Stirling, A. R. Padhani, "Reproducibility of dynamic contrast-enhanced MRI in human muscle and tumours: comparison of quantitative and semi-quantitative analysis," NMR in Biomedicine , vol. 15, no. 2, pp. 132-142, 2002.

[77] S. Makkat, R. Luypaert, S. Sourbron, T. Stadnik, J. De Mey, "Quantification of perfusion and permeability in breast tumors with a deconvolution-based analysis of second-bolus T1-DCE data," Journal of Magnetic Resonance Imaging , vol. 25, no. 6, pp. 1159-1167, 2007.

[78] D. M. McGrath, D. P. Bradley, J. L. Tessier, T. Lacey, C. J. Taylor, G. J. M. Parker, "Comparison of model-based arterial input functions for dynamic contrast-enhanced MRI in tumor bearing rats," Magnetic Resonance in Medicine , vol. 61, no. 5, pp. 1173-1184, 2009.

[79] C. Yang, W. M. Stadler, G. S. Karczmar, M. Milosevic, I. Yeung, M. A. Haider, "Comparison of quantitative parameters in cervix cancer measured by dynamic contrast-enhanced MRI and CT," Magnetic Resonance in Medicine , vol. 63, no. 6, pp. 1601-1609, 2010.

[80] A. N. Priest, A. B. Gill, M. Kataoka, M. A. McLean, I. Joubert, M. J. Graves, J. R. Griffiths, R. A. F. Crawford, H. Earl, J. D. Brenton, D. J. Lomas, E. Sala, "Dynamic contrast-enhanced MRI in ovarian cancer: initial experience at 3 tesla in primary and metastatic disease," Magnetic Resonance in Medicine , vol. 63, no. 4, pp. 1044-1049, 2010.

[81] R. Meng, S. D. Chang, E. C. Jones, S. L. Goldenberg, P. Kozlowski, "Comparison between population average and experimentally measured arterial input function in predicting biopsy results in prostate cancer," Academic Radiology , vol. 17, no. 4, pp. 520-525, 2010.

[82] S. Makkat, R. Luypaert, S. Sourbron, T. Stadnik, J. De Mey, "Assessment of tumor blood flow in breast tumors with T1-dynamic contrast-enhanced MR imaging: impact of dose reduction and the use of a prebolus technique on diagnostic efficacy," Journal of Magnetic Resonance Imaging , vol. 31, no. 3, pp. 556-561, 2010.

[83] J. U. Fluckiger, M. C. Schabel, E. V. R. DiBella, "Model-based blind estimation of kinetic parameters in dynamic contrast enhanced (DCE)-MRI," Magnetic Resonance in Medicine , vol. 62, no. 6, pp. 1477-1486, 2009.

[84] E. Henderson, B. K. Rutt, T. Y. Lee, "Temporal sampling requirements for the tracer kinetics modeling of breast disease," Magnetic Resonance Imaging , vol. 16, no. 9, pp. 1057-1073, 1998.

[85] E. Jackson, S. Gupta, M. Rosen, "QIBA DCE-MRI technical committee update: phantom studies and first DCE-MRI profile," in Proceedings of the 96th Scientific Assembly and Annual Meeting of the Radiological Society of North America (RSNA '10), Chicago, Ill, USA, December 2010.

[86] E. Jackson, E. Ashton, J. L. Evelhoch, "Multivendor, multisite DCE-MRI phantom validation study," in Proceedings of the 95th Scientific Assembly and Annual Meeting of the Radiological Society of North America (RSNA '10), Chicago, Ill, USA, December 2009.

[87] R. K. Gupta, "A new look at the method of variable nutation angle for the measurement of spin-lattice relaxation times using fourier transform NMR," Journal of Magnetic Resonance , vol. 25, no. 1, pp. 231-235, 1977.

[88] J. P. Pärkkä, P. Niemi, A. Saraste, J. W. Koskenvuo, M. Komu, V. Oikonen, J. O. Toikka, T. O. Kiviniemi, J. Knuuti, H. Sakuma, J. J. Hartiala, "Comparison of MRI and positron emission tomography for measuring myocardial perfusion reserve in healthy humans," Magnetic Resonance in Medicine , vol. 55, no. 4, pp. 772-779, 2006.

[89] D. L. Buckley, P. J. Drew, S. Mussurakis, J. R. T. Monson, A. Horsman, "Microvessel density in invasive breast cancer assessed by dynamic Gd-DTPA enhanced MRI," Journal of Magnetic Resonance Imaging , vol. 7, no. 3, pp. 461-464, 1997.

[90] M. V. Knopp, E. Weiss, H. P. Sinn, J. Mattern, H. Junkermann, J. Radeleff, A. Magener, G. Brix, S. Delorme, I. Zuna, G. van Kaick, "Pathophysiologic basis of contrast enhancement in breast tumors," Journal of Magnetic Resonance Imaging , vol. 10, no. 3, pp. 260-266, 1999.

[91] C. A. Hulka, W. B. Edmister, B. L. Smith, L. Tan, D. C. Sgroi, T. Campbell, D. B. Kopans, R. M. Weisskoff, "Dynamic echo-planar imaging of the breast: experience in diagnosing breast carcinoma and correlation with tumor angiogenesis," Radiology , vol. 205, no. 3, pp. 837-842, 1997.

[92] M. Y. Su, Y. C. Cheung, J. P. Fruehauf, H. Yu, O. Nalcioglu, E. Mechetner, A. Kyshtoobayeva, S. C. Chen, S. Hsueh, C. E. McLaren, Y. L. Wan, "Correlation of dynamic contrast enhancement MRI parameters with microvessel density and VEGF for assessment of angiogenesis in breast cancer," Journal of Magnetic Resonance Imaging , vol. 18, no. 4, pp. 467-477, 2003.

[93] N. G. Costouros, D. Lorang, Y. Zhang, M. S. Miller, F. E. Diehn, S. M. Hewitt, M. V. Knopp, K. C. P. Li, P. L. Choyke, H. R. Alexander, S. K. Libutti, "Microarray gene expression analysis of murine tumor heterogeneity defined by dynamic contrast-enhanced MRI," Molecular Imaging , vol. 1, no. 3, pp. 301-308, 2002.

[94] C. Ellingsen, T. A. M. Egeland, K. Galappathi, E. K. Rofstad, "Dynamic contrast-enhanced magnetic resonance imaging of human cervical carcinoma xenografts: pharmacokinetic analysis and correlation to tumor histomorphology," Radiotherapy and Oncology , vol. 97, no. 2, pp. 217-224, 2010., [email protected]

[95] R. G. J. Kierkels, W. H. Backes, M. H. M. Janssen, J. Buijsen, R. G. H. Beets-Tan, P. Lambin, G. Lammering, M. C. Oellers, H. J. W. L. Aerts, "Comparison between perfusion computed tomography and dynamic contrast-enhanced magnetic resonance imaging in rectal cancer," International Journal of Radiation Oncology Biology Physics , vol. 77, no. 2, pp. 400-408, 2010.

[96] C. Ménard, R. C. Susil, P. Choyke, J. Coleman, R. Grubb, A. Gharib, A. Krieger, P. Guion, D. Thomasson, K. Ullman, S. Gupta, V. Espina, L. Liotta, E. Petricoin, G. Fitchtinger, L. L. Whitcomb, E. Atalar, C. Norman Coleman, K. Camphausen, "An interventional magnetic resonance imaging technique for the molecular characterization of intraprostatic dynamic contrast enhancement," Molecular Imaging , vol. 4, no. 1, pp. 63-66, 2005.