Are quantitative features of lung nodules reproducible at different CT acquisition and reconstruction parameters?

Abstract

Consistency and duplicability in Computed Tomography (CT) output is essential to quantitative imaging for lung cancer detection and monitoring. This study of CT-detected lung nodules investigated the reproducibility of volume-, density-, and texture-based features (outcome variables) over routine ranges of radiation dose, reconstruction kernel, and slice thickness. CT raw data of 23 nodules were reconstructed using 320 acquisition/reconstruction conditions (combinations of 4 doses, 10 kernels, and 8 thicknesses). Scans at 12.5%, 25%, and 50% of protocol dose were simulated; reduced-dose and full-dose data were reconstructed using conventional filtered back-projection and iterative-reconstruction kernels at a range of thicknesses (0.6–5.0 mm). Full-dose/B50f kernel reconstructions underwent expert segmentation for reference Region-Of-Interest (ROI) and nodule volume per thickness; each ROI was applied to 40 corresponding images (combinations of 4 doses and 10 kernels). Typical texture analysis metrics (including 5 histogram features, 13 Gray Level Co-occurrence Matrix, 5 Run Length Matrix, 2 Neighboring Gray-Level Dependence Matrix, and 3 Neighborhood Gray-Tone Difference Matrix) were computed per ROI. Reconstruction conditions resulting in no significant change in volume, density, or texture metrics were identified as “compatible pairs” for a given outcome variable. Our results indicate that as thickness increases, volumetric reproducibility decreases, while reproducibility of histogram- and texture-based features across different acquisition and reconstruction parameters improves. To achieve concomitant reproducibility of volumetric and radiomic results across studies, balanced standardization of the imaging acquisition parameters is required.

About the Authors:

Barbaros S. Erdal

Roles Conceptualization, Data curation, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Visualization, Writing – original draft, Writing – review & editing

Current address: Department of Radiology, Mayo Clinic, Jacksonville, Florida, United States of America

Affiliation: Department of Radiology, The Ohio State University College of Medicine, Columbus, Ohio, United States of America

ORCID logo https://orcid.org/0000-0003-3637-0102

Mutlu Demirer

Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing – original draft, Writing – review & editing

Current address: Department of Radiology, Mayo Clinic, Jacksonville, Florida, United States of America

Affiliation: Department of Radiology, The Ohio State University College of Medicine, Columbus, Ohio, United States of America

ORCID logo https://orcid.org/0000-0002-6842-282X

Kevin J. Little

Roles Conceptualization, Data curation, Writing – review & editing

* E-mail: [email protected]

Affiliation: Department of Radiology, The Ohio State University College of Medicine, Columbus, Ohio, United States of America

ORCID logo https://orcid.org/0000-0002-7241-0012

Chiemezie C. Amadi

Roles Data curation, Resources, Writing – review & editing

Affiliation: Department of Radiology, The Ohio State University College of Medicine, Columbus, Ohio, United States of America

Gehan F. M. Ibrahim

Roles Data curation, Resources, Writing – review & editing

Affiliation: Department of Radiology, The Ohio State University College of Medicine, Columbus, Ohio, United States of America

Thomas P. O’Donnell

Roles Project administration, Writing – review & editing

Affiliation: Siemens Healthineers, Malvern, Pennsylvania, United States of America and Erlangen, Germany

Rainer Grimmer

Roles Project administration, Writing – review & editing

Affiliation: Siemens Healthineers, Malvern, Pennsylvania, United States of America and Erlangen, Germany

Vikash Gupta

Roles Methodology, Writing – review & editing

Current address: Department of Radiology, Mayo Clinic, Jacksonville, Florida, United States of America

Affiliation: Department of Radiology, The Ohio State University College of Medicine, Columbus, Ohio, United States of America

Luciano M. Prevedello

Roles Conceptualization, Methodology, Writing – review & editing

Affiliation: Department of Radiology, The Ohio State University College of Medicine, Columbus, Ohio, United States of America

Richard D. White

Roles Conceptualization, Funding acquisition, Methodology, Supervision, Writing – review & editing

Current address: Department of Radiology, Mayo Clinic, Jacksonville, Florida, United States of America

Affiliation: Department of Radiology, The Ohio State University College of Medicine, Columbus, Ohio, United States of America

Introduction

Lung nodules have traditionally been evaluated with two-dimensional (2-D) linear measurements on chest Computed Tomography (CT) (e.g., Response Evaluation Criteria in Solid Tumors (RECIST) [1]). However, three-dimensional (3-D) volumetric assessments of lung nodules are gaining importance due to: 1. improved representation of disease extent and therapeutic-response; and 2. less user-dependency and higher reproducibility of the results [2]. Concurrently, new diagnostic and treatment paradigms increasingly emphasize the value of quantitative radiomic features of lung nodules and surrounding lung tissue as indicators of tumor type, aggressiveness, and/or responsiveness to treatment [3, 4]. However, the associated quantitative metrics assume adequacy and uniformity in CT data acquisition and reconstruction despite well-recognized wide inter-scan variability in: 1. examination protocoling for image acquisition; 2. image reconstructions and displays; and 3. CT scanning capabilities and performances. Although the susceptibility of several lung nodule measurements (volumetric or radiomic) to variations in individual CT acquisition parameters has already been recognized based on preliminary studies of clinical or phantom data [5–8], the concomitant effects that ranges of radiation-dose, reconstruction kernel, and slice thickness have on nodule volume and texture features have not yet been fully investigated.

The purpose of this research was to: 1. determine the impact of a variety of imaging acquisition and reconstruction parameters on lung nodule volumes and radiomic features; 2. identify potential imaging acquisition parameters that allow consistency and reproducibility of volumetric and/or radiomic features of lung nodules.

Materials and methods

This work was approved by The Ohio State University Institutional Review Board (2015H0185, 2017H0100). This retrospective study used de-identified images, and consent was not obtained.

Acquisition and reconstruction parameter variations

Raw Digital Imaging and Communications in Medicine (DICOM) data from 23 non-contrast-enhanced chest CT examinations demonstrating a lung nodule were acquired from either single-source (Definition AS, AS Plus, Edge) or dual-source (Definition Flash) multi-detector CT systems [Somatom series: Siemens Healthineers, Forchheim, Germany (https://www.healthcare.siemens.com/computed-tomography)] with ranges of settings (Tube voltage = 100–140 kV, Q.ref.mAs = 50–100 mAs, Eff.mAs = 41–154 mAs, and pitch = 0.6–1.0). Based on this data, chest CT data sets underwent the following:

1. Simulation at different dose levels (simulated 12.5%, 25%, 50%) using a previously validated simulation tool [9–11], as well as reconstruction at 100% of total dose of clinical protocols [ReconCT: Siemens Healthineers, Forchheim, Germany].

2. Reproduction with 10 different kernels based on either Filtered Back Projection (FBP) or Iterative Reconstruction (IR) (FBP: B31f, B40f, B50f, B60f, B70f; SAFIRE (strength 3): I26f, I31f, I40f, I50f, I70f) [ReconCT: Siemens Healthineers, Forchheim, Germany].

3. Recreation at 8 different slice thicknesses (0.6, 0.75, 1.0, 1.5, 2.0, 3.0, 4.0, and 5.0 mm) using an offline reconstruction system [ReconCT: Siemens Healthineers, Forchheim, Germany].

Based on combinations of the aforementioned variations in acquisition or reconstruction parameters, 320 versions of each examination (i.e., 4 doses x 10 kernels x 8 thicknesses) were created.

Segmentation and volume measurement

In order to display the nodules to dedicated thoracic radiologists (CCA and GFMI each with 6–8 years of post-fellowship experience), a custom Graphical User Interface (GUI) [12] was built [MeVisLab version 2.8, Windows 64 bit, VS2013: Bremen, Germany (https://www.mevislab.de/)] (Fig 1). This GUI integrated the functionalities of a commercial nodule segmentation algorithm [syngo.via MM Oncology: Siemens Healthineers, Forchheim, Germany (https://www.siemens-healthineers.com/medical-imaging-it/syngoviaspecialtopics/syngo-via-for-oncology)] as they would appear within an established commercial post-processing platform [syngo.via: Siemens Healthineers, Forchheim, Germany (https://www.healthcare.siemens.com/medical-imaging-it/advanced-visualization-solutions/syngovia)].

[Figure omitted. See PDF.]

Fig 1. A custom GUI allowed thoracic radiologists to evaluate nodules.

For the purposes of this study, only the solid components were considered. Semi-automated segmentations made by radiologists for a given nodule are shown. Top left in Yellow: Radiologist 1; top right in Red: Radiologist 2; bottom left: non-segmented original; bottom right in Orange, both radiologist’s segmentations combined.

https://doi.org/10.1371/journal.pone.0240184.g001

For each chest CT examination, sets of 100%-dose/FBP B50f-kernel images were reconstructed at the aforementioned 8 different slice thicknesses, with nodule extent and shape previously independently confirmed by consensus between the two dedicated thoracic radiologists. This resulted in 8 reference Region-Of-Interest (ROI) stacks, each corresponding to a specific slice thickness, which were then applied to the 40 different dose-kernel combinations (i.e., 4 doses x 10 kernels) at constant slice thickness (Fig 2).

[Figure omitted. See PDF.]

Fig 2. Segmentation procedure for lung nodules.

Each nodule was segmented from 100%-dose/FBP B50f-kernel images reconstructed at 8 different slice thicknesses. The resulting 8 thickness-specific ROI stacks were then applied to the corresponding images reconstructed at 4 different dose levels and 10 different kernels at stable slice thickness.

https://doi.org/10.1371/journal.pone.0240184.g002

Analysis of reproducibility of volumetric measurements

Using automatic 3-D segmentation, 8 volumetric measurements were obtained for each nodule (Fig 3A) corresponding to each slice thickness (Fig 3B) and normalized by their averages using (1) as follows (Vi: nodule volume for i-th thickness):(1)

[Figure omitted. See PDF.]

Fig 3. Methodologies used during volumetric assessment of lung nodules.

The following represent expert segmentation of lung nodules: (a) Reference radiologist-segmented ROI stacks obtained using 100%-dose level, a FBP B50f kernel, and 1-mm slice thickness. Nodules are sorted by their average volumes and numbered 1 to 23. (b) Calculated volumes for each nodule using 8 reference ROIs depending on slice thicknesses (colors indicate the corresponding nodules shown on the left). (c) Normalized volumes of nodules obtained using their average volumes.

https://doi.org/10.1371/journal.pone.0240184.g003

Using the distributions of normalized volumes for each slice thickness (Fig 3C), t statistics were calculated using 2-tailed t-test (2) to evaluate the compatibilities of slice thicknesses based on volumetric measurements. If t<1.96 (P<0.05), slice thicknesses are accepted as compatible with 95% confidence interval; m1, m2: means; s1, s2: standard deviations; n1, n2: number of samples (nodules):(2)

Image-feature extraction

A range of extracted radiomic image features was assessed (Table 1). They included 28 image-texture features (available in MeVisLab) as follows: 1. 5 Histogram-based features; 2. 13 Gray Level Co-occurrence Matrix (GLCM)-based features; 3. 5 Run Length Matrix (RLM)-based features; 4. 2 Neighboring Gray-Level Dependence Matrix (NGLDM)-based features, and 5. 3 Neighborhood Gray-Tone Difference Matrix (NGTDM) based features which were computed for all 320 segmented image volumes of each nodule.

[Figure omitted. See PDF.]

Table 1. Extracted features.

https://doi.org/10.1371/journal.pone.0240184.t001

Compatibility of image reconstruction conditions based on radiomic features

Means and standard deviations of the aforementioned 28 image features were computed for each of the 320 segmented images of each nodule. Statistically significant changes in image features were evaluated using 2-tailed t-test. Reconstruction condition pairs that satisfy t values of lower than <1.96 (P<0.05) were accepted as compatible with 95% confidence interval. Compatibility ratios were calculated using Eq 3; : compatibility ratio of reconstruction condition pair (Ri,Rj); Ri: i-th reconstruction condition (i = 1,2,…,320); : compatibility of (Ri,Rj) pair for feature f (1,…,28) and patient p (1,…,23) calculated using t-test (Eq 2):(3)

We produced a compatibility map of reconstruction conditions (Fig 4) to highlight effects of changes in slice thickness (T), kernel sharpness (K), and dose (D). Reconstruction condition parameters are sorted based on total number of compatibilities. Thickness order: 5mm, 4mm, 3mm, 2mm, 1.5mm, 1mm, 0.75mm, 0.6mm. Kernel order: I26f, I31f, B31f, I40f, B40f, I50f, B50f, I70f, B60f, B70f. Dose level order: 100%, 50%, 25%, 12.5%.

[Figure omitted. See PDF.]

Fig 4. Reconstruction-condition compatibility map based on extracted features and patients.

Intersections of conditions are highlighted (Red: incompatible, Green: compatible) based on their compatibility ratios calculated using t-test. Diagonal shows 100% compatibility which satisfies all comparisons (28x23). Changing reconstruction parameters (thickness/dose/kernel-sharpness) decreases the compatibility. In order to obtain higher compatibility, changes to the reconstruction parameters should be applied carefully. For example if thickness needs to be switched from 2 mm to 0.75 mm, softer kernels and/or higher dose levels are needed as seen from the intersection of two thicknesses. Data are available in S1 Data.

https://doi.org/10.1371/journal.pone.0240184.g004

Results

Reproducibility of volumetric measurements

Volume comparisons of the segmented nodules were performed on 8 slice thicknesses on all 23 cases (Fig 5). The slice thickness resulting in the least amount of volumetric variability was 2mm, with +0.39%±1.59 (mean±SD) variation from average volume. On the other hand, 1.5-mm thickness gave the highest volumes with +2.16%±2.07, and 5-mm the lowest volumes with -4.22%±4.65. While standard deviations were relatively stable for slice thicknesses below 2mm, increasing thickness beyond 2mm was associated with rapidly increasing standard deviation. The assessment of volumetric reproducibility shows that increasing slice thickness decreases compatibility (Fig 6).

[Figure omitted. See PDF.]

Fig 5. Normalized volumetric measurements and trend lines based on slice thickness.

https://doi.org/10.1371/journal.pone.0240184.g005

[Figure omitted. See PDF.]

Fig 6. P values for compatibility analysis of slice thicknesses based on volumetric measurements.

Higher P value indicates higher compatibility. Intersection of compatible thicknesses with P values higher than 0.05 are highlighted with green; P values lower than 0.05 (highlighted with orange to red) indicate incompatible slice thicknesses.

https://doi.org/10.1371/journal.pone.0240184.g006

Reproducibility of radiomic features

320 reconstruction conditions were compared to each other on 28 texture features and 23 cases, totaling 65,945,600 (320x320x28x23) comparisons. As shown in Fig 4, highest average compatibility of 24.47% was achieved using the combination of highest slice thickness, smoothest kernel and highest dose level (5mm/I26f/100%). Compatibility decreases while decreasing slice thickness and/or kernel smoothness and/or dose level. The lowest average compatibility of 2.65% was at the combination of lowest slice thickness, sharpest kernel and lowest dose level (0.6mm/B70f/12.5%). Fig 7 shows percentage compatibilities of radiomic features based on dose, kernel, and thickness changes. The most robust feature was the density against dose changes (87.45% compatibility); and skewness was most robust for kernel and slice-thickness changes (53.73%-82.51% compatibility). Deviation was the weakest feature for all cases. On average, the GLCM based feature group was the most vulnerable feature group (19.41% average compatibility). In addition, results showing percentage compatibilities of conditions based on kernel sharpness, slice thickness, and dose levels are presented in Figs 8, 9 and 10 respectively.

[Figure omitted. See PDF.]

Fig 7. Percentage compatibilities of features based on dose, kernel, and thickness changes.

https://doi.org/10.1371/journal.pone.0240184.g007

[Figure omitted. See PDF.]

Fig 8. Percentage of compatible texture features in different kernel pairs while keeping slice thickness and dose levels fixed for all 23 patients and 28 features.

For example, by changing the kernel from I31f to I26f, on average, 54.7% of the 28 texture features will be statistically the same (compatible) in our patient population under the same slice thickness and dose level configurations (32 different conditions).

https://doi.org/10.1371/journal.pone.0240184.g008

[Figure omitted. See PDF.]

Fig 9. Percentage of compatible texture features in different slice thickness pairs while keeping kernel and dose levels fixed for all 23 patients.

https://doi.org/10.1371/journal.pone.0240184.g009

[Figure omitted. See PDF.]

Fig 10. Percentage of compatible texture features in different dose level pairs while keeping kernel and slice thickness fixed for all 23 patients.

https://doi.org/10.1371/journal.pone.0240184.g010

As demonstrated in the figures, while stable volumetric measurements can be obtained (e.g. 2mm slice thickness) and volumetric measurement errors can be predicted when slice thicknesses are changed (Fig 5), that is not the case for radiomic features (Figs 6 and 7). Texture measurements can be very unstable when conditions are altered.

Discussion

Robust image features are vital for designing and standardizing anatomic and radiomic-based diagnostic and prognostic decision-making [13–17]. In this study, we investigated the effects of image acquisition and reconstruction conditions on volumetric and radiomic features of lung nodules derived from chest CT scans. These conditions contained an extensive list of combinations (320 versions of each examination: 4 doses x 10 kernels x 8 thicknesses).

Lung nodule detection is enhanced when thinner CT slices are produced [18–20]; screening CT scans are preferentially performed with 1 to 2.5-mm-thick slices [18, 21], as was done in the National Lung Screening Trial [22]. Solid nodules ≥ 4-mm in diameter are currently considered important [18] and were the focus of this research.

Accurate and reliable measurement of lung nodule size from CT scans is a key biomarker in the diagnosis of lung cancer. The estimation of nodule growth rates serves as a predictor of malignancy, and size change reflects efficacy of a treatment [23, 24]. A related challenge is the consistency in establishing lung nodule size [18]. Reliable sizing of nodules has traditionally been limited by subjectivity in selection of the desired dimensions to measure and by non-uniformity of manual manipulation of digital calipers; this represents a source of disagreement between interpreters and reference standards [2, 18]. Manually measuring nodule size involves laboriously inspecting all images that include the lesion [23]. To provide a standard method for nodule-size measurement, the RECIST working group proposed the common use of the 1-D maximal diameter as an efficient standard estimator of lesion volume [1, 23, 25]. While such basic mass size measurements are typically used in clinical practice, 3-D volume measurements are growing in importance due to evidence that 3-D volumetry is more robust for quantifying tumor size [24, 26]. Poor agreement between 3-D and simpler methods is commonly seen when the nodule does not conform to spherical or ellipsoidal assumptions that underlie 1-D and 2-D measurements, respectively; in the context of spatial extent for a 3-D object without restrictions on shape, size is best expressed by the volume occupied by the object [23].

Consequently, there is increasing interest in computer-assisted methods aiding the radiologist in measuring the size of lung nodules using volumetric methods [27–30], despite the fact that their calibration and validation becomes a new challenge [23]. Although lung-nodule 3D volumetry has the potential to improve patient management, there is considerable and largely unpredictable variability in its execution related to produced slice thickness, reconstruction algorithm, and scan dose [24, 26, 31, 32]. Our research addressed this issue and demonstrated that while increasing thickness decreases volumetric reproducibility, it improves the reproducibility of histogram- and texture-based features across different acquisition and reconstruction parameters.

Texture analysis is promising for phenotyping and segmenting cancerous tissues [6, 7]. However, Buch et al. [6] highlighted the major looming problem pertaining to radiomics and big data, that despite collecting increasing numbers of radiological images at an exponentially growing rate, the medical field is far from a completely data-driven artificially intelligent diagnosis. The main reason is a lack of data harmony across multi-site studies, which keeps the training data substantially low for a truly large-scale study. Based on the data-mining approach we performed on our results from various parameters, our recommendations are as follows: 1) During volumetric measurements, changes in slice thickness may produce relatively low errors, however the effects on texture are most dramatic. Hence, if possible, slice thickness should not be altered between studies if serial radiomic features are being compared. 2) If scanning/reconstruction changes are inevitable, they should be limited to a single parameter. For example, only dose level or only sharpness should be changed (Figs 8–10), and those changes should be kept to a minimum. Multiple parameter changes, in general, produce greater measurement errors (Fig 4).

We performed a large-scale data mining approach for finding a “compatible” set of parameters, however, as it can be seen from the results, compatibilities are very limited. In an earlier study, Young et al. [24] raised another concern pertaining to radiation dosage and kernel usage in CT lung-nodule volumetry; they found that the volume of lung nodules was extremely robust to the dose and reconstruction kernels. On the other hand, Chen et al. [8, 33–35] showed that the choice of reconstruction algorithm slightly affects the measurement of lesion volume. Regardless, reduced dosage comes at the cost of decreased image quality, which in turn make the results less reproducible for textures. Lo et al. [7] discussed these specific issues pertaining to lung CT and lung nodule quantification. Many authors recognize that it will be desirable to reduce the dose levels among patients, and many vendors are already working towards quantitative imaging based on reduced dose levels [36–39]. In addition, other reports indicated that iterative reconstruction algorithms offer an opportunity to substantially reduce radiation dose in CT scans while maintaining good image resolution for visualization and nodule detections [8, 40–43]. However, the quantitative measurements from iterative algorithms can be significantly different from the traditional FBP algorithms due to varying noise and resolution properties [44]. Novel artificial intelligence-based reconstruction methods may also offer comparable imaging at reduced radiation doses, and these methods should similarly receive careful consideration for quantitative measurements.

Based on earlier studies, high-resolution texture characterization requires image reconstruction using thin slices. However, thin slices also increase image noise; increasing slice thickness will decrease the noise while inducing blur. Based on our results, increasing slice thickness decreases the reproducibility of volumetric measurements but increases the reproducibility of histogram- and texture- based features. Our view based on these results is that there might not be a best universal set of parameters that simultaneously covers both volumetric measures and radiomic features. Increasing slice thickness causes information loss due to smoothing effects [45]. This may be the reason for increased compatibility for texture features.

Our methodology with full control of the reconstruction parameters had advantages and disadvantages. While limiting the number of scanners gave us the full advantage of image reconstruction algorithm compatibility and standardization, our results at this point are limited to few scanner types (e.g. Siemens Definition Flash, Definition AS, AS Plus, Edge) within a single healthcare system. Our data therefore lacked the heterogeneity that exists across CT imaging vendors and providers, and we hope to collaborate with additional institutions in future work. However, the current approach helped us to point out some potential issues in terms of CT image reconstruction.

Another limitation was our relatively small sample size (n = 23). This was due to fact that raw images can only be stored in our scanners for only 2 to 4 weeks (depending on the scanner and due to space restrictions). The study was initially designed as a retrospective analysis on nodule measurements, instead of classification of nodules. Under our circumstances, this would not be possible with retrospective analysis, and we would have needed to conduct a prospective study, which could have taken much longer to complete. Our sample included variations in acquisition parameters, such as tube voltage and tube current, that were not considered independently due to the small sample size. However, these differences were indicative of the heterogeneity in the acquisition parameters of clinical studies at our institution at the time of data collection due to factors such as automatic exposure control and protocol variations between scanners.

Due to the small sample size, we only looked at nodules under 2~2.5 cm for their solid components. If any nodule had surrounding ground glass tissue, these components were ignored (two of our cases had minor ground glass features surrounding them), and only the solid components were included in the measurements and comparisons.

While some texture features are not stable with changing imaging parameters, this does not necessarily mean that a particular feature does not have predictive value within a given algorithm. For a given feature, any recognized instability should be accounted for in algorithm design to limit outcome variance. Future work from our group will consider how feature variations due to imaging parameters affect the outcome of algorithms.

In conclusion, we found that slice thickness is the main factor impacting the reproducibility of the image features we investigated. It is difficult to maintain both volumetric and radiomic measurement reproducibility and reliability simultaneously. However, our findings indicate that at a thickness of approximately 2mm volumetric measurement reproducibility can be maintained. However, especially for reproducibility in radiomic features, image scanning and reconstruction protocols need to remain stable. Standardization of the imaging acquisition parameters would become even more important in larger scale studies, where images are collected from multiple institutions. As we have shown here, even with scanners with compatible image reconstruction parameters in a highly controlled environment, it is hard to maintain measurement reproducibility when parameters are changed. With images coming from multiple sites and multiple vendors, if studies are not designed and scanning protocols are not aligned properly; it can be very hard to produce reliable results that can be utilized within clinical studies.

Supporting information

[Figure omitted. See PDF.]

S1 Data. Collection of numerical data.

https://doi.org/10.1371/journal.pone.0240184.s001

(XLSX)

Acknowledgments

We thank Andreas Wimmer (Siemens Healthineers) for his assistance with post-processing packages.

Citation: Erdal BS, Demirer M, Little KJ, Amadi CC, Ibrahim GFM, O’Donnell TP, et al. (2020) Are quantitative features of lung nodules reproducible at different CT acquisition and reconstruction parameters? PLoS ONE 15(10): e0240184. https://doi.org/10.1371/journal.pone.0240184

References

1. Bankier AA, MacMahon H, Goo JM, Rubin GD, Schaefer-Prokop CM, Naidich DP. Recommendations for measuring pulmonary nodules at CT: a statement from the Fleischner Society. Radiology. 2017;285(2):584–600. pmid:28650738

2. Heuvelmans MA, Walter JE, Vliegenthart R, van Ooijen PMA, De Bock GH, De Koning HJ, et al. Disagreement of diameter and volume measurements for pulmonary nodule size estimation in CT lung cancer screening. Thorax. 2018;73(8):779–81. pmid:29056601

3. Aerts HJWL, Velazquez ER, Leijenaar RTH, Parmar C, Grossmann P, Carvalho S, et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nature communications. 2014;5:4006. pmid:24892406

4. Gillies RJ, Kinahan PE, Hricak H. Radiomics: images are more than pictures, they are data. Radiology. 2015;278(2):563–77. pmid:26579733

5. Zhao B, Tan Y, Tsai W-Y, Qi J, Xie C, Lu L, et al. Reproducibility of radiomics for deciphering tumor phenotype with imaging. Scientific reports. 2016;6:23428. pmid:27009765

6. Buch K, Li B, Qureshi MM, Kuno H, Anderson SW, Sakai O. Quantitative assessment of variation in CT parameters on texture features: pilot study using a nonanatomic phantom. American Journal of Neuroradiology. 2017;38(5):981–5. pmid:28341714

7. Lo P, Young S, Kim HJ, Brown MS, McNitt-Gray MF. Variability in CT lung-nodule quantification: Effects of dose reduction and reconstruction methods on density and texture based features. Medical physics. 2016;43(8Part1):4854–65.

8. Chen B, Ramirez Giraldo JC, Solomon J, Samei E. Evaluating iterative reconstruction performance in computed tomography. Medical Physics. 2014;41(12):121913. pmid:25471973

9. Yu L, Shiung M, Jondal D, McCollough CH. Development and Validation of a Practical Lower-Dose-Simulation Tool for Optimizing Computed Tomography Scan Protocols: Journal of Computer Assisted Tomography. 2012;36(4):477–87. pmid:22805680

10. Kramer M, Ellmann S, Allmendinger T, Eller A, Kammerer F, May MS, et al. Computed Tomography Angiography of Carotid Arteries and Vertebrobasilar System: A Simulation Study for Radiation Dose Reduction. Medicine. 2015 Jul;94(26):e1058. pmid:26131822

11. Ellmann S, Kammerer F, Brand M, Allmendinger T, May MS, Uder M, et al. A Novel Pairwise Comparison-Based Method to Determine Radiation Dose Reduction Potentials of Iterative Reconstruction Algorithms, Exemplified Through Circle of Willis Computed Tomography Angiography: Investigative Radiology. 2016 May;51(5):331–9.

12. Demirer M, Candemir S, Bigelow MT, Yu SM, Gupta V, Prevedello LM, et al. A User Interface for Optimizing Radiologist Engagement in Image-Data Curation for Artificial Intelligence. Radiology: Artificial Intelligence, Paper RYAI-18-0095.

13. Buch K, Fujita A, Li B, Kawashima Y, Qureshi MM, Sakai O. Using texture analysis to determine human papillomavirus status of oropharyngeal squamous cell carcinomas on CT. American Journal of Neuroradiology. 2015;36(7):1343–8. pmid:25836725

14. Fave X, Cook M, Frederick A, Zhang L, Yang J, Fried D, et al. Preliminary investigation into sources of uncertainty in quantitative imaging features. Computerized Medical Imaging and Graphics. 2015;44:54–61. pmid:26004695

15. Fried D V, Tucker SL, Zhou S, Liao Z, Mawlawi O, Ibbott G, et al. Prognostic value and reproducibility of pretreatment CT texture features in stage III non-small cell lung cancer. International Journal of Radiation Oncology* Biology* Physics. 2014;90(4):834–42.

16. Ganeshan B, Miles KA. Quantifying tumour heterogeneity with CT. Cancer Imaging. 2013;13(1):140–9.

17. Miles KA, Ganeshan B, Griffiths MR, Young RCD, Chatwin CR. Colorectal Cancer: Texture Analysis of Portal Phase Hepatic CT Images as a Potential Marker of Survival. RADIOLOGY. 2009;250(2):444–52. pmid:19164695

18. Rubin GD. Lung Nodule and Cancer Detection in Computed Tomography Screening. Journal of Thoracic Imaging. 2015;30(2):130–8. pmid:25658477

19. Fischbach F, Knollmann F, Griesshaber V, Freund T, Akkol E, Felix R. Detection of pulmonary nodules by multislice computed tomography: improved detection rate with reduced slice thickness. European radiology. 2003;13(10):2378–83. pmid:12743736

20. Sinsuat M, Saita S, Kawata Y, Niki N, Ohmatsu H, Tsuchida T, et al. Influence of Slice Thickness on Diagnoses of Pulmonary Nodules Using Low-dose CT: Potential Dependence of Detection and Diagnostic Agreement on Features and Location of Nodule. Academic Radiology. 2011;18(5):594–604. pmid:21377905

21. Kazerooni EA, Austin JH, Black WC, Dyer DS, Hazelton TR, Leung AN, et al. ACR-STR practice parameter for the performance and reporting of lung cancer screening thoracic computed tomography (CT): 2014 (Resolution 4). Journal of thoracic imaging. 2014;29(5):310–6. pmid:24992501

22. Team NLSTR. The national lung screening trial: overview and study design. Radiology. 2011;258(1):243–53. pmid:21045183

23. Reeves AP, Biancardi AM, Apanasovich T V, Meyer CR, MacMahon H, van Beek EJR, et al. The Lung Image Database Consortium (LIDC): a comparison of different size metrics for pulmonary nodule measurements. Academic radiology. 2007;14(12):1475–85. pmid:18035277

24. Young S, Kim HJG, Ko MM, Ko WW, Flores C, McNitt-Gray MF. Variability in CT lung-nodule volumetry: Effects of dose reduction and reconstruction methods. Medical physics. 2015;42(5):2679–89. pmid:25979066

25. Therasse P, Arbuck SG, Eisenhauer EA, Wanders J, Kaplan RS, Rubinstein L, et al. New guidelines to evaluate the response to treatment in solid tumors. Journal of the National Cancer Institute. 2000;92(3):205–16. pmid:10655437

26. Petrick N, Kim HJG, Clunie D, Borradaile K, Ford R, Zeng R, et al. Comparison of 1D, 2D, and 3D nodule sizing methods by radiologists for spherical and complex nodules on thoracic CT phantom images. Academic radiology. 2014;21(1):30–40. pmid:24331262

27. Kuhnigk J-M, Dicken V, Bornemann L, Wormanns D, Krass S, Peitgen H-O. Fast automated segmentation and reproducible volumetry of pulmonary metastases in CT-scans for therapy monitoring. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. 2004. p. 933–41.

28. Goodman LR, Gulsun M, Washington L, Nagy PG, Piacsek KL. Inherent variability of CT lung nodule measurements in vivo using semiautomated volumetric measurements. American journal of roentgenology. 2006;186(4):989–94. pmid:16554568

29. Revel M-P, Merlin A, Peyrard S, Triki R, Couchon S, Chatellier G, et al. Software volumetric evaluation of doubling times for differentiating benign versus malignant pulmonary nodules. American Journal of Roentgenology. 2006;187(1):135–42. pmid:16794167

30. Reeves AP, Chan AB, Yankelevitz DF, Henschke CI, Kressler B, Kostis WJ. On measuring the change in size of pulmonary nodules. IEEE transactions on medical imaging. 2006;25(4):435–50. pmid:16608059

31. Zhao B, Tan Y, Bell DJ, Marley SE, Guo P, Mann H, et al. Exploring intra-and inter-reader variability in uni-dimensional, bi-dimensional, and volumetric measurements of solid tumors on CT scans reconstructed at different slice intervals. European journal of radiology. 2013;82(6):959–68. pmid:23489982

32. Gavrielides MA, Kinnard LM, Myers KJ, Petrick N. Noncalcified lung nodules: volumetric assessment with thoracic CT. Radiology. 2009;251(1):26–37. pmid:19332844

33. Chen B, Marin D, Richard S, Husarik D, Nelson R, Samei E. Precision of iodine quantification in hepatic CT: effects of iterative reconstruction with various imaging parameters. American Journal of Roentgenology. 2013;200(5):W475—W482. pmid:23617515

34. Chen B, Christianson O, Wilson JM, Samei E. Assessment of volumetric noise and resolution performance for linear and nonlinear CT reconstruction methods. Medical Physics. 2014;41(7):71909.

35. Chen B, Barnhart H, Richard S, Robins M, Colsher J, Samei E. Volumetric quantification of lung nodules in CT with iterative reconstruction (ASiR and MBIR). Medical Physics. 2013;40(11):111902. pmid:24320435

36. Hunter LA, Krafft S, Stingo F, Choi H, Martel MK, Kry SF, et al. High quality machine-robust image features: identification in nonsmall cell lung cancer computed tomography images. Medical physics. 2013;40(12):121916. pmid:24320527

37. Mackin D, Fave X, Zhang L, Fried D, Yang J, Taylor B, et al. Measuring Computed Tomography Scanner Variability of Radiomics Features. Investigative Radiology. 2015;50(11):757–65. pmid:26115366

38. Fave X, Mackin D, Yang J, Zhang J, Fried D, Balter P, et al. Can radiomics features be reproducibly measured from CBCT images for patients with non-small cell lung cancer? Medical physics. 2015;42(12):6784–97. pmid:26632036

39. Nyflot MJ, Yang F, Byrd D, Bowen SR, Sandison GA, Kinahan PE. Quantitative radiomics: impact of stochastic effects on textural feature analysis implies the need for standards. Journal of Medical Imaging. 2015;2(4):41002.

40. Beister M, Kolditz D, Kalender WA. Iterative reconstruction methods in X-ray CT. Physica medica. 2012;28(2):94–108. pmid:22316498

41. Thibault J-B, Sauer KD, Bouman CA, Hsieh J. A three-dimensional statistical approach to improved image quality for multislice helical CT. Medical physics. 2007;34(11):4526–44. pmid:18072519

42. Solomon J, Mileto A, Ramirez-Giraldo JC, Samei E. Diagnostic performance of an advanced modeled iterative reconstruction algorithm for low-contrast detectability with a third-generation dual-source multidetector CT scanner: potential for radiation dose reduction in a multireader study. Radiology. 2015;275(3):735–45. pmid:25751228

43. Pickhardt PJ, Lubner MG, Kim DH, Tang J, Ruma JA, del Rio AM, et al. Abdominal CT with model-based iterative reconstruction (MBIR): initial results of a prospective trial comparing ultralow-dose with standard-dose imaging. American journal of roentgenology. 2012;199(6):1266–74. pmid:23169718

44. Schulz B, Beeres M, Bodelle B, Bauer R, Al-Butmeh F, Thalhammer A, et al. Performance of iterative image reconstruction in CT of the paranasal sinuses: a phantom study. American Journal of Neuroradiology. 2013;34(5):1072–6. pmid:23221946

45. Zhao B, Tan Y, Tsai WY, Schwartz LH, Lu L. Exploring variability in CT characterization of tumors: a preliminary phantom study. Translational oncology. 2014;7(1):88–93. pmid:24772211