Artificial intelligence to predict the BRAFV600E mutation in patients with thyroid cancer

Abstract

Purpose

To investigate whether a computer-aided diagnosis (CAD) program developed using the deep learning convolutional neural network (CNN) on neck US images can predict the BRAFV600E mutation in thyroid cancer.

Methods

469 thyroid cancers in 469 patients were included in this retrospective study. A CAD program recently developed using the deep CNN provided risks of malignancy (0–100%) as well as binary results (cancer or not). Using the CAD program, we calculated the risk of malignancy based on a US image of each thyroid nodule (CAD value). Univariate and multivariate logistic regression analyses were performed including patient demographics, the American College of Radiology (ACR) Thyroid Imaging, Reporting and Data System (TIRADS) categories and risks of malignancy calculated through CAD to identify independent predictive factors for the BRAFV600E mutation in thyroid cancer. The predictive power of the CAD value and final multivariable model for the BRAFV600E mutation in thyroid cancer were measured using the area under the receiver operating characteristic (ROC) curves.

Results

In this study, 380 (81%) patients were positive and 89 (19%) patients were negative for the BRAFV600E mutation. On multivariate analysis, older age (OR = 1.025, p = 0.018), smaller size (OR = 0.963, p = 0.006), and higher CAD value (OR = 1.016, p = 0.004) were significantly associated with the BRAFV600E mutation. The CAD value yielded an AUC of 0.646 (95% CI: 0.576, 0.716) for predicting the BRAFV600E mutation, while the multivariable model yielded an AUC of 0.706 (95% CI: 0.576, 0.716). The multivariable model showed significantly better performance than the CAD value alone (p = 0.004).

Conclusion

Deep learning-based CAD for thyroid US can help us predict the BRAFV600E mutation in thyroid cancer. More multi-center studies with more cases are needed to further validate our study results.

About the Authors:

Jiyoung Yoon

Roles Data curation, Formal analysis, Investigation, Methodology, Validation, Visualization, Writing – original draft

Affiliation: Department of Radiology, Severance Hospital, Research Institute of Radiological Science, Yonsei University, College of Medicine, Seoul, South Korea

Eunjung Lee

Roles Data curation, Formal analysis, Methodology, Software, Validation, Writing – review & editing

Affiliation: Department of Computational Science and Engineering, Yonsei University, Seoul, South Korea

Ja Seung Koo

Roles Data curation, Formal analysis, Supervision, Writing – review & editing

Affiliation: Department of Pathology, Severance Hospital, Yonsei University, College of Medicine, Seoul, South Korea

Jung Hyun Yoon

Roles Data curation, Investigation, Validation, Writing – review & editing

Affiliation: Department of Radiology, Severance Hospital, Research Institute of Radiological Science, Yonsei University, College of Medicine, Seoul, South Korea

Kee-Hyun Nam

Roles Data curation, Formal analysis, Investigation, Validation, Visualization, Writing – review & editing

Affiliation: Department of Surgery, Severance Hospital, Yonsei University, College of Medicine, Seoul, South Korea

ORCID logo https://orcid.org/0000-0002-6852-1190

Jandee Lee

Roles Data curation, Formal analysis, Methodology, Validation, Writing – review & editing

Affiliation: Department of Surgery, Severance Hospital, Yonsei University, College of Medicine, Seoul, South Korea

Young Suk Jo

Roles Data curation, Formal analysis, Investigation, Methodology, Resources, Validation, Writing – review & editing

Affiliation: Department of Internal Medicine, Severance Hospital, Yonsei University, College of Medicine, Seoul, South Korea

Hee Jung Moon

Roles Data curation, Investigation, Methodology, Validation, Writing – review & editing

Affiliation: Department of Radiology, Severance Hospital, Research Institute of Radiological Science, Yonsei University, College of Medicine, Seoul, South Korea

Vivian Youngjean Park

Roles Data curation, Formal analysis, Investigation, Methodology, Validation, Writing – review & editing

Affiliation: Department of Radiology, Severance Hospital, Research Institute of Radiological Science, Yonsei University, College of Medicine, Seoul, South Korea

Jin Young Kwak

Roles Conceptualization, Funding acquisition, Supervision, Writing – review & editing

* E-mail: [email protected]

Affiliation: Department of Radiology, Severance Hospital, Research Institute of Radiological Science, Yonsei University, College of Medicine, Seoul, South Korea

ORCID logo https://orcid.org/0000-0002-6212-1495

Introduction

The BRAFV600E mutation is the most commonly detected oncogene in thyroid cancer and is highly specific for papillary thyroid cancer (PTC) [1, 2]. Because of this high specificity, the mutation has been used in diagnostic methods adjunctive to fine needle aspiration (FNA) for thyroid nodules, especially those with indeterminate cytology results [1–5]. Also, the BRAFV600E mutation is a known predictor of aggressive PTCs as it has been associated with higher cancer stage and a higher rate of extrathyroidal extension and lymph node metastases [6–8]. Considering that ultrasonography (US) features have also been associated with the BRAFV600E mutation [3–5, 9], we can assume that the mutation test is a cost-effective tool for thyroid nodules with suspicious US features. However, US itself is inherently limited by its subjectivity, leading to the low reproducibility of its results [10, 11].

Artificial intelligence (AI) is being widely studied in the medical field with various applications being thought feasible in actual healthcare systems [12]. When diagnosing thyroid nodules on US, AI has shown comparable diagnostic accuracies to radiologists, with publications reporting the accuracies to be from 83% to 98% [13–15]. Compared to how diagnostic performances can vary according to the operator’s level of experience, AI can provide more objective results for US.

Several previous studies have focused on US features to predict BRAF status, and these studies have found that US characteristics associated with malignancy (marked hypoechogenicity, taller-than-wide shape, etc.) are also associated with BRAF positivity [9, 16, 17]. However, to the best of our knowledge, no studies have associated the BRAFV600E mutation with US features using a computer-aided diagnosis (CAD) program. Therefore, we investigated whether a CAD program that was recently developed using the deep convolutional neural network (CNN) to diagnose thyroid cancer from US images of thyroid nodules could also help predict the BRAFV600E mutation in thyroid nodules.

Materials & methods

This retrospective study was approved by the Severance Hospital Institutional Review Board, with a waiver for patient consent (Approval number: 4-2019-1223). Signed informed consent was obtained from all patients prior to biopsy or surgical procedures.

Patients

We collected patient data from Feburary 2019 to September 2019, during which 1817 patients were referred to our institution for thyroid surgery after undergoing biopsies at outside clinics and preoperative staging US at our institution, a tertiary referral center. Among them, 1348 patients were excluded because they were younger than 19 years old (n = 8), had nodules smaller than 10mm on US (n = 993), did not proceed with surgery (n = 230), had a final pathology of benignity (n = 87), and did not have the BRAF test performed on a pathologic specimen (n = 30). Finally, 469 thyroid cancers in 469 patients were included. Mean age of the patients was 42.3 years ± 12.8 (range, 19–86 years). Mean size of the tumors was 16.9 mm ± 8.3 (range, 10–62 mm). There were 353 (75.3%) women and 116 (24.7%) men.

US examinations

Preoperative staging US was performed with a 5–12 MHz linear array transducer (iU22; Philips Medical Systems or EPIQ 7; Philips Medical Systems, Bothell, WA, USA) by one of nine radiologists specializing in thyroid imaging. All physicians were informed of previous cytology results and the location of the index lesion that underwent FNA or biopsy at outside clinics. Since June 2012, the physicians in our institution who perform staging US have filled out a report format for index nodules that includes nodule size and US features such as composition (solid, predominantly cyst, predominantly solid, spongiform, cyst), echogenicity (hyper-, iso-, hypo- and marked hypoechoic), margin (well, microlobulated, irregular), shape (parallel, not parallel) and calcification (eggshell, micro-, macro- and mixed calcification) [18]. One radiologist (K.J.Y.) who had 19 years of experience in thyroid imaging re-assigned the nodules according to the American College of Radiology (ACR) Thyroid Imaging, Reporting and Data System (TIRADS) using the prospectively recorded US features. Suspicious lymph nodes on US were also evaluated at the time of staging US. Finally, clinical T and N stages were reported according to the 8th American Joint Committee on Cancer (AJCC) cancer staging system [19].

Image acquisition and CNN evaluation

One radiologist (K.J.Y) who had 19 years of experience in thyroid imaging reviewed the preoperative US images, selected a representative image for each thyroid tumor, saved it as a JPEG file in the picture archiving and communication system, and drew square region-of-interests (ROIs) to cover the whole nodule using the Microsoft Paint program (version 6.1; Microsoft Corporation, Redmond, WA, USA).

Recently, we developed a CAD program to diagnose thyroid cancer on US from 13,560 US images of thyroid nodules using the deep CNN (see S1 File for details on the CAD program). The CAD program provides risks of malignancy (0–100%) as well as binary results (cancer or not). Using the CAD program, we calculated the risks of malignancy from US images of the thyroid nodules (CAD value) (Figs 1 and 2).

[Figure omitted. See PDF.]

Fig 1. A 33-year-old woman with a thyroid nodule confirmed as suspicious for malignancy on US guided-fine needle aspiration (US-FNA).

https://doi.org/10.1371/journal.pone.0242806.g001

[Figure omitted. See PDF.]

Fig 2. A 40-year-old woman with a thyroid nodule confirmed as suspicious for malignancy on US-FNA.

https://doi.org/10.1371/journal.pone.0242806.g002

US showed a 13 mm-sized solid, hypoechoic, and taller-than-wide shaped nodule with lobulated margin and punctate echogenic foci, which was categorized as “highly suspicious” on the American College of Radiology Thyroid Imaging Reporting and Data System (ACR TIRADS) (a). The computer-aided diagnosis (CAD) program calculated the risk of malignancy as 98.86% (CAD value). (b). The patient underwent total thyroidectomy and final pathology was conventional papillary thyroid carcinoma. Pyrosequencing was positive for the BRAFV600E mutation.

US shows a 34 mm-sized predominantly solid, hypoechoic, and wider-than-tall shaped nodule with smooth margin and without echogenic foci, which was categorized as “mildly suspicious” on the American College of Radiology Thyroid Imaging Reporting and Data System (ACR TIRADS). (a). The computer-aided diagnosis (CAD) program calculated the risk of malignancy as 3.71% (CAD value) (b). The patient underwent left lobectomy and the final pathology was the follicular variant of papillary thyroid carcinoma. Pyrosequencing was negative for the BRAFV600E mutation.

BRAFV600E mutation analysis

BRAF V600E mutants were evaluated using pyrosequencing per the manufacturer’s instructions. PCR amplification was performed with a forward primer (5′- biotin- TTCTTCATGAAGACCTCACAGTAA-3′) and a reverse primer (5′- CCAGACAACTGTTCAAACTGATG-3′) on a C1000 thermal cycler (BIO-RAD, California, USA). The pyrosequencing reaction was performed with a sequencing primer (5′- GGACCCACTCCCATCGAGATTT-3′) on a Pyromark Q24 instrument (Qiagen). The produced pyrogram was analyzed with the PyroMark Q24 software (Qiagen) to distinguish mutant versus wild-type alleles by relative peak height.

Statistical analysis

We compared the demographics, ACR TIRADS categories and risks of malignancy from CAD between patients with and without the BRAFV600E mutation using the independent two-sample t-test for continuous data and the Chi-square test or Fisher’s exact test for categorical data. Univariate and multivariate logistic regression analyses were performed to identify independent factors for predicting the BRAFV600E mutation in thyroid cancer. As the ACR TIRADS and CAD values were assessed from the US images of thyroid nodules which could be correlated with each other, multicollinearity between the ACR TIRADS and CAD values was assessed with variance inflation factors (VIF). Multicollinearity was considered high when the VIF was > 10.

The multivariable model was only constructed with factors independently associated with the BRAFV600E mutation. The predictive power of the CAD value and the multivariable model for the BRAFV600E mutation in thyroid cancer were measured and compared with the area under the receiver operating characteristic (ROC) curves. The optimal cutoff for the CAD value to predict the BRAFV600E mutation was identified using the Youden index. The performances of the CAD value and the multivariable model were compared using Delong’s method. All statistical analyses in this study were performed with SPSS statistical software (SPSS for Windows, version 25.0; IBM Corporation, Armonk, NY) and R (version 4.0.2.; R Foundation for Statistical Computing, Vienna, Austria). P-values of less than 0.05 were considered to indicate statistical significance.

Results

In this study, 380 (81%) patients were positive and 89 (19%) patients were negative for the BRAFV600E mutation. There were 422 (90%) conventional papillary thyroid carcinomas (PTCs), 26 (5.6%) follicular variant PTC, 9 (1.9%) diffuse sclerosing variant PTCs, 4 (0.9%) solid variant PTCs, 2 (0.4%) tall cell variant PTCs, 2 (0.4%) oncocytic variant PTCs, 1 (0.2%) Hobnail variant PTC, 1 (0.2%) Warthin-like variant PTC, 1 (0.2%) minimally invasive follicular carcinoma, and 1 (0.2%) medullary carcinoma.

Table 1 summarizes patient demographics and clinical characteristics according to the BRAFV600E mutation. Patients with the BRAFV600E mutation were older and had tumors of smaller size, higher ACR TIRADS scores, and higher CAD values than ones without the mutation.

[Figure omitted. See PDF.]

Table 1. Comparison of patient demographics, ACR TIRADS categories and CAD values according to the BRAFV600E mutation status.

https://doi.org/10.1371/journal.pone.0242806.t001

To determine potential predictors directly related to the BRAFV600E mutation, univariate and multivariate logistic regression were performed. On univariate analysis, the BRAFV600E mutation was associated with older age, smaller size, higher ACR TIRADS scores, and higher CAD values (Table 2). Multivariate logistic regression analysis was performed to assess the independent associations of the BRAFV600E mutation with clinical factors. As the ACR TIRADS and CAD values did not show multicollinearity in the model (VIF was 1.366), we used both parameters in the regression model. On multivariate analysis, older age, smaller size, and higher CAD values were significantly associated with the BRAFV600E mutation (Table 2).

[Figure omitted. See PDF.]

Table 2. Univariate and multivariate logistic regression analysis for clinical factors based on the presence or absence of the BRAFV600E mutation in thyroid carcinoma patients.

https://doi.org/10.1371/journal.pone.0242806.t002

The multivariable logistic model was constructed using age, size, and the CAD value to predict the BRAFV600E mutation. The ROC curves of the CAD value and the multivariable model were plotted to show the performances of the CAD value and the multivariable model for predicting the BRAFV600E mutation in thyroid cancer patients. The CAD value yielded an AUC of 0.646 (95% CI: 0.576, 0.716) for predicting the BRAFV600E mutation. The cutoff for the CAD value to obtain maximum accuracy for the diagnosis of the BRAFV600E mutation was 57.7. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the CAD value for predicting the BRAFV600E mutation was 85.3%, 41.6%, 59.4%, and 73.9%, respectively. The multivariable model yielded an AUC of 0.706 (95% CI: 0.644, 0.767) for predicting the BRAFV600E mutation. The multivariable model showed significantly better performance than the CAD value only (p = 0.004) (Fig 3).

[Figure omitted. See PDF.]

Fig 3. Receiver operating characteristic curves of the CAD value and the multivariable logistic model for predicting the BRAFV600E mutation.

https://doi.org/10.1371/journal.pone.0242806.g003

The CAD value yielded an AUC of 0.646 (95% CI: 0.576, 0.716) for predicting the BRAFV600E mutation, while the multivariable model yielded an AUC of 0.706 (95% CI: 0.576, 0.716). The multivariable model showed significantly better performance than the CAD value only (p = 0.004).

Discussion

The prevalence of the BRAF mutation in PTC ranges from 29% to 83% and almost all of the mutations are the BRAFV600E form [20]. As the BRAFV600E mutation is highly specific for PTC [21], a combination of cytology and the BRAFV600E mutation has been considered a good diagnostic approach for thyroid nodules with indeterminate results on cytology [1]. Furthermore, the BRAFV600E mutation is a known predictor of aggressive PTC; previous studies have reported that thyroid malignancies with the BRAFV600E mutation present with higher cancer stage and higher rate of extrathyroidal extension and lymph node metastases than those without the mutation [6–8]. As the BRAFV600E mutation can be used in both diagnosing thyroid cancer and predicting its aggressiveness, preoperative detection of the mutation may also help optimize the initial operative approach in patients with thyroid nodules [22–24]. However, the mutation analysis requires invasive procedures such as biopsy or surgical resection to retrieve specimens.

Several previous studies have tried to identify US features that can predict BRAF status [9, 16, 17]. A recent review article concluded that US features of PTCs correlate with their particular molecular mutations, and BRAFV600E mutation is known to be associated with suspicious US findings, such as hypoechogenicity, non-parallel orientation, taller-than-wide shape, spiculated/microlobulated margins, and the presence of microcalcifications [25]. However, interpretation of US image is operator-dependent and inter-observer variability is moderate to substantial [10, 11, 26]. Radiomics is one of the emerging methods for predicting molecular characteristics of tumors, using quantitative imaging features extracted using data-characterization algorithms [27, 28]. A recent study evaluated the value of US-based radiomics for predicting BRAFV600E mutation in pathologically proven PTCs, but radiomics features extracted from US had limited value [29].

In our study, the ACR TIRADS category assessed by radiologists was associated with the BRAFV600E mutation in univariate analysis, but the correlation disappeared in multivariable analysis. In contrast, the CAD value, which was calculated from a program that we developed using deep CNN, was associated with the BRAFV600E mutation in multivariable analysis. Our findings with the AUCs of the ROC curve indicate that the CAD value can help predict the BRAFV600E mutation in thyroid cancer. Furthermore, the multivariable model which was composed with patient age, tumor size, and the CAD value showed significantly increased predictability. Considering that a deep learning algorithm allows consistent prediction, we can avoid or even overcome problems arising from inter-observer variability in US evaluations with the CAD value [30]. Therefore, the CAD program used in this study is expected to be a non-invasive and objective biomarker for prediction of the BRAFV600E mutation in PTCs.

A number of CAD algorithms have been developed and implemented for the accurate diagnosis of disease, including binary logistic regression, support vector machines, and artificial neural networks [31]. Among them, deep learning with CNN has recently gained attention for its high performance in image recognition, and other researchers have developed deep learning CNN models by using radiological images for lesion detection, lesion evaluation, estimation of patient survival, etc. [32]. Many CNN models have already benefited clinical practice and this is especially so for thyroid malignancies, as deep learning CNN models developed for the detection, segmentation, and classification of thyroid nodules using neck US images have shown good performances [13, 14, 33–36]. A recent study reported that a deep learning CNN model developed to discriminate the BRAFV600E and RAS mutations of PTCs from histological images showed 95% accurate predictions [37]. However, ours is the first study using deep learning-based CAD to predict the BRAFV600E mutation in patients diagnosed with thyroid cancer. Previous studies reported associations between suspicious US features (such as marked hypoechogenicity and taller-than-wide shape) and the BRAF mutation, but most of these features showed weak correlation and the study results were inconsistent, probably due to intra- and inter-observer variability [9, 10, 16, 17]. As the CAD program provides objective probabilities of malignancy, it can potentially help diagnose BRAFV600E mutations with more accuracy.

There are several limitations to this study. First, our study was of retrospective design and patient data were collected from a single tertiary referral center, which means a selection bias was inevitable. Second, an experienced radiologist retrospectively re-assigned categories to thyroid nodules according to ACR TIRADS using US images prospectively recorded by 9 radiologists. The re-assigned categories might not have fully reflected the prospectively recorded US features. Third, the correlation between the CAD value and cancer prognosis could not be confirmed due to the lack of a follow-up period. Further studies are needed to confirm whether the CAD value can be a predictor of cancer prognosis. Last, the prevalence of BRAFV600E mutations among PTCs differs from country to country. The Korean population is known for its high mutation prevalence and 81% of the PTCs in our study showed the BRAFV600E mutation [38]. Results might differ when the same study is conducted on populations from other countries.

In conclusion, the CAD program developed with deep learning can help predict the BRAFV600E mutation in thyroid cancer. More multi-center studies with more cases are needed to further validate our study results.

Supporting information

[Figure omitted. See PDF.]

S1 File. Computer-aided diagnosis (CAD) program using the convolutional neural network (CNN).

https://doi.org/10.1371/journal.pone.0242806.s001

(DOCX)

Citation: Yoon J, Lee E, Koo JS, Yoon JH, Nam K-H, Lee J, et al. (2020) Artificial intelligence to predict the BRAFV600E mutation in patients with thyroid cancer. PLoS ONE 15(11): e0242806. https://doi.org/10.1371/journal.pone.0242806

References

1. Seo JY, Kim EK, Baek JH, Shin JH, Han KH, Kwak JY. Can ultrasound be as a surrogate marker for diagnosing a papillary thyroid cancer? Comparison with BRAF mutation analysis. Yonsei medical journal. 2014;55(4):871–8.

2. Moon HJ, Kwak JY, Kim EK, Choi JR, Hong SW, Kim MJ, et al. The role of BRAFV600E mutation and ultrasonography for the surgical management of a thyroid nodule suspicious for papillary thyroid carcinoma on cytology. Ann Surg Oncol. 2009;16(11):3125–31. pmid:19644722

3. Rho M, Kim EK, Moon HJ, Yoon JH, Park VY, Han K, et al. Clinical Parameter for Deciding the BRAFV600E Mutation Test in Atypia of Undetermined Significance/Follicular Lesion of Undetermined Significance Thyroid Nodules: US Features According to TIRADS. Ultrasound Q. 2017;33(4):284–8. pmid:28877096

4. Moon HJ, Kim EK, Chung WY, Choi JR, Yoon JH, Kwak JY. Diagnostic value of BRAF(V600E) mutation analysis of thyroid nodules according to ultrasonographic features and the time of aspiration. Ann Surg Oncol. 2011;18(3):792–9. pmid:20945104

5. Lee EJ, Song KH, Kim DL, Jang YM, Hwang TS, Kim SK. The BRAF(V600E) mutation is associated with malignant ultrasonographic features in thyroid nodules. Clin Endocrinol (Oxf). 2011;75(6):844–50. pmid:21707687

6. Xing M. BRAF mutation in papillary thyroid cancer: pathogenic role, molecular bases, and clinical implications. Endocrine reviews. 2007;28(7):742–62. pmid:17940185

7. Park JY, Yi JW, Park CH, Lim Y, Lee KH, Lee KE, et al. Role of BRAF and RAS Mutations in Extrathyroidal Extension in Papillary Thyroid Cancer. Cancer genomics & proteomics. 2016;13(2):171–81. pmid:26912807

8. Lee KC, Li C, Schneider EB, Wang Y, Somervell H, Krafft M, et al. Is BRAF mutation associated with lymph node metastasis in patients with papillary thyroid cancer? Surgery. 2012;152(6):977–83. pmid:23062653

9. Kabaker AS, Tublin ME, Nikiforov YE, Armstrong MJ, Hodak SP, Stang MT, et al. Suspicious ultrasound characteristics predict BRAF V600E-positive papillary thyroid carcinoma. Thyroid. 2012;22(6):585–9. pmid:22524468

10. Choi SH, Kim EK, Kwak JY, Kim MJ, Son EJ. Interobserver and intraobserver variations in ultrasound assessment of thyroid nodules. Thyroid. 2010;20(2):167–72. pmid:19725777

11. Kim HG, Kwak JY, Kim EK, Choi SH, Moon HJ. Man to man training: can it help improve the diagnostic performances and interobserver variabilities of thyroid ultrasonography in residents? European journal of radiology. 2012;81(3):e352–6. pmid:22137098

12. Kelly CJ, Karthikesalingam A, Suleyman M, Corrado G, King D. Key challenges for delivering clinical impact with artificial intelligence. BMC medicine. 2019;17(1):195. pmid:31665002

13. Ma J, Wu F, Zhu J, Xu D, Kong D. A pre-trained convolutional neural network based method for thyroid nodule diagnosis. Ultrasonics. 2017;73:221–30. pmid:27668999

14. Ko SY, Lee JH, Yoon JH, Na H, Hong E, Han K, et al. Deep convolutional neural network for the diagnosis of thyroid nodules on ultrasound. Head Neck. 2019;41(4):885–91. pmid:30715773

15. Li X, Zhang S, Zhang Q, Wei X, Pan Y, Zhao J, et al. Diagnosis of thyroid cancer using deep convolutional neural network models applied to sonographic images: a retrospective, multicohort, diagnostic study. The Lancet Oncology. 2019;20(2):193–201. pmid:30583848

16. Kwak JY, Kim EK, Chung WY, Moon HJ, Kim MJ, Choi JR. Association of BRAFV600E mutation with poor clinical prognostic factors and US features in Korean patients with papillary thyroid microcarcinoma. Radiology. 2009;253(3):854–60. pmid:19710001

17. Hwang J, Shin JH, Han BK, Ko EY, Kang SS, Kim JW, et al. Papillary thyroid carcinoma with BRAFV600E mutation: sonographic prediction. AJR American journal of roentgenology. 2010;194(5):W425–30. pmid:20410389

18. Yoon JH, Lee HS, Kim EK, Moon HJ, Kwak JY. Malignancy Risk Stratification of Thyroid Nodules: Comparison between the Thyroid Imaging Reporting and Data System and the 2014 American Thyroid Association Management Guidelines. Radiology. 2016;278(3):917–24. pmid:26348102

19. Amin MB, Greene FL, Edge SB, Compton CC, Gershenwald JE, Brookland RK, et al. The Eighth Edition AJCC Cancer Staging Manual: Continuing to build a bridge from a population-based to a more "personalized" approach to cancer staging. CA: a cancer journal for clinicians. 2017;67(2):93–9. pmid:28094848

20. Trovisco V, Soares P, Sobrinho-Simoes M. B-RAF mutations in the etiopathogenesis, diagnosis, and prognosis of thyroid carcinomas. Human pathology. 2006;37(7):781–6. pmid:16784975

21. Cohen Y, Xing M, Mambo E, Guo Z, Wu G, Trink B, et al. BRAF mutation in papillary thyroid carcinoma. Journal of the National Cancer Institute. 2003;95(8):625–7. pmid:12697856

22. Melck AL, Yip L, Carty SE. The utility of BRAF testing in the management of papillary thyroid cancer. The oncologist. 2010;15(12):1285–93. pmid:21147872

23. O’Neill CJ, Bullock M, Chou A, Sidhu SB, Delbridge LW, Robinson BG, et al. BRAF(V600E) mutation is associated with an increased risk of nodal recurrence requiring reoperative surgery in patients with papillary thyroid cancer. Surgery. 2010;148(6):1139–45; discussion 45–6. pmid:21134544

24. Yip L, Nikiforova MN, Carty SE, Yim JH, Stang MT, Tublin MJ, et al. Optimizing surgical treatment of papillary thyroid carcinoma associated with BRAF mutation. Surgery. 2009;146(6):1215–23. pmid:19958951

25. Lewinski A, Adamczewski Z, Zygmunt A, Markuszewski L, Karbownik-Lewinska M, Stasiak M. Correlations between Molecular Landscape and Sonographic Image of Different Variants of Papillary Thyroid Carcinoma. Journal of clinical medicine. 2019;8(11). pmid:31717363

26. Park CS, Kim SH, Jung SL, Kang BJ, Kim JY, Choi JJ, et al. Observer variability in the sonographic evaluation of thyroid nodules. J Clin Ultrasound. 2010;38(6):287–93. pmid:20544863

27. Gevaert O, Echegaray S, Khuong A, Hoang CD, Shrager JB, Jensen KC, et al. Predictive radiogenomics modeling of EGFR mutation status in lung cancer. Sci Rep. 2017;7:41674. pmid:28139704

28. Braman NM, Etesami M, Prasanna P, Dubchuk C, Gilmore H, Tiwari P, et al. Intratumoral and peritumoral radiomics for the pretreatment prediction of pathological complete response to neoadjuvant chemotherapy based on breast DCE-MRI. Breast cancer research: BCR. 2017;19(1):57. pmid:28521821

29. Yoon JH, Han K, Lee E, Lee J, Kim EK, Moon HJ, et al. Radiomics in predicting mutation status for thyroid cancer: A preliminary study using radiomics features for predicting BRAFV600E mutations in papillary thyroid carcinoma. PloS one. 2020;15(2):e0228968. pmid:32053670

30. Buda M, Wildman-Tobriner B, Hoang JK, Thayer D, Tessler FN, Middleton WD, et al. Management of Thyroid Nodules Seen on US Images: Deep Learning May Match Performance of Radiologists. Radiology. 2019;292(3):695–701. pmid:31287391

31. Lim KJ, Choi CS, Yoon DY, Chang SK, Kim KK, Han H, et al. Computer-aided diagnosis for the differentiation of malignant from benign thyroid nodules on ultrasonography. Academic radiology. 2008;15(7):853–8. pmid:18572120

32. Yasaka K, Akai H, Kunimatsu A, Kiryu S, Abe O. Deep learning with convolutional neural network in radiology. Japanese journal of radiology. 2018;36(4):257–72. pmid:29498017

33. Song W, Li S, Liu J, Qin H, Zhang B, Zhang S, et al. Multitask Cascade Convolution Neural Networks for Automatic Thyroid Nodule Detection and Recognition. IEEE journal of biomedical and health informatics. 2019;23(3):1215–24. pmid:29994412

34. Ma J, Wu F, Jiang T, Zhao Q, Kong D. Ultrasound image-based thyroid nodule automatic segmentation using convolutional neural networks. International journal of computer assisted radiology and surgery. 2017;12(11):1895–910. pmid:28762196

35. Li H, Weng J, Shi Y, Gu W, Mao Y, Wang Y, et al. An improved deep learning approach for detection of thyroid papillary cancer in ultrasound images. Sci Rep. 2018;8(1):6600. pmid:29700427

36. Ma J, Wu F, Jiang T, Zhu J, Kong D. Cascade convolutional neural networks for automatic detection of thyroid nodules in ultrasound images. Medical physics. 2017;44(5):1678–91. pmid:28186630

37. Tsou P, Wu CJ. Mapping Driver Mutations to Histopathological Subtypes in Papillary Thyroid Carcinoma: Applying a Deep Convolutional Neural Network. Journal of clinical medicine. 2019;8(10). pmid:31614962

38. Kim KH, Kang DW, Kim SH, Seong IO, Kang DY. Mutations of the BRAF gene in papillary thyroid carcinoma in a Korean population. Yonsei medical journal. 2004;45(5):818–21. pmid:15515191