Intraoperative Prediction Of Non-Sentinel Lymph Node Metastasis Based On The Molecular Assay In Breast Cancer Patients

Abstract

Purpose: The aim of the study is to construct an intraoperative nomogram for the prediction of non-sentinel lymph node (NSLN) metastasis based on the one-step nucleic acid amplification assay in breast cancer patients.

Methods: A total of 552 patients were enrolled in the training study and 1090 patients were enrolled in the validation study. The nomogram was constructed based on the molecular assay with logistic multivariate regression analysis in the training study and was validated in the validation study.

Results: A novel nomogram model was constructed with the total tumor load, the clinical primary tumor size, the number of positive and negative sentinel lymph nodes. The area under the receiver operating characteristic curve (AUC) of the model was 0.842. The AUC of the model which was sensitive to discern the patients with the stage of pN1 and ≥pN2 was 0.861.

Conclusion: The nomogram model will help to guide the axillary management intraoperatively and precisely confirm the target region of radiotherapy postoperatively.

Introduction

Sentinel lymph node biopsy (SLNB) has become a universal procedure in node-negative breast cancer patients. It provides accurate axillary stage and reduces the morbidity associated with the unnecessary axillary lymph node dissection (ALND).

One-step nucleic acid amplification (OSNA) assay (Sysmex, Kobe, Japan) is an objective molecular technique combining node tissue homogenization and subsequent reverse-transcription loop-mediated isothermal amplification of CK-19 mRNA in a single reaction. The assay can accurately detect metastases of >0.2 mm in sentinel lymph nodes (SLNs) intraoperatively. Two consequent studies from China have validated the intraoperative OSNA assay. One trial (hereinafter referred to as the training study) consisting of five centers (n=552) have confirmed the good performances of the assay in 2010.¹ Another trial (or the validation study) containing three centers (n=1090) has been conducted to accelerate the approval of the assay in China (ClinicalTrial No. CBCSG-001c).

It is reported that most patients with positive SLNs does not have non-sentinel lymph node (NSLN, the other axillary lymph node except SLNs) metastasis,² and these patients cannot benefit from ALND. Therefore, it is of very remarkable clinical significance to construct a predictive model for SLN-positive patients to avoid ALND. In the training study, a novel predictive nomogram that can predict NSLN metastasis intraoperatively is constructed based on the molecular assay. Then, the nomogram is verified in the validation study.

Materials And Methods Patients

From February to December 2010, 552 consecutive breast cancer patients scheduled for SLNB were enrolled in the training study from five centers. From June 2015 to May 2017, 1090 consecutive breast cancer patients were enrolled in the validation study from three centers. The study was approved by the ethics committee of each center (Shandong Cancer Hospital Affiliated to Shandong University; Cancer Hospital, Tianjin Medical University; Sun Yat-sen University Cancer Center; Cancer Hospital, Beijing University; Cancer Hospital, Fudan University; Guangdong General Hospital; Cancer Hospital, Tianjin Medical University). Each patient signed the written informed consent. Patients who had undergone previous ipsilateral axillary surgery were excluded from this study.

Sampling Method

SLNs were defatted after SLNB. If the length was less than 4 mm, the node was sliced into two blocks along the long axis (a, b). Intraoperatively, the block a was prepared for the OSNA assay. Postoperatively, the block b was assessed by the histology evaluation. If the length was more than 4 mm, the node was sliced into four blocks (a, b, c, d). Intraoperatively, blocks a and c were prepared for the OSNA assay. Postoperatively, blocks b and d were subjected to the histology evaluation.

Intraoperative OSNA Assay

The OSNA assay was carried out according to the manufacturer's instructions (Sysmex). Three different calibrators with defined CK-19 mRNA copy concentrations were used to construct a standard curve on the amplification instrument (Sysmex). Node tissues were then homogenized in 4-mL homogenizing buffer LYNORHAG (Sysmex). The homogenate was briefly centrifuged and directly used as a template for reverse-transcription loop-mediated isothermal amplification. Amplification of CK-19 mRNA was automatically performed in the amplification instrument with a ready-to-use reagent LYNOAMP (Sysmex) kit which consisted of a primer-nucleotide-mix, enzymes, and CK-19 mRNA calibrators as well as positive and negative controls. All the results were presented in semi-quantitative categories [++, +, -] and further specified by CK-19 mRNA copy number/μL: ~250 copies [-], 250–5000 copies [+], and 5000– [++]. The result [+] was comparable to the presence of a micro-metastasis and [++] to a macro-metastasis.

Postoperative Histological Evaluation

All node blocks used for histological evaluation were fixed in 10% buffered formalin and paraffin embedded. Four 4- to 6-μm-thick slides, 200 μm apart, were taken from each block. Metastases larger than 0.2 mm were considered positive in this study. Metastases were classified according to the 7th criterion of the American Joint Cancer Committee. Macro-metastases (≥2 mm) and micro-metastases (0.2`2 mm, pT1mic) were considered node positive. Isolated tumor cells [≤0.2 mm, ITCs, pT0(i+)] were considered node negative.

Statistical Methods

Variables selected from the existing predictive models such as MSKCC, MD Anderson (MDA), Mayo, Tenon, Cambridge, Stanford, Helsinki,^3–9 and the total tumor load (TTL, the sum of the CK-19 mRNA copy number/μL of all positive lymph nodes) were adopted as the clinicopathological indicators to construct the model. The data of TTL were analyzed by the Log form to avoid the influence of nonlinear data in the statistical process. Univariate correlation analysis was performed on the data to determine which indicators were predictors of NSLN metastasis. Differences in clinicopathologic indicators between the groups were assessed using the t-tests for continuous variables and the χ²-test for categorical variables. Indicators that were statistically significant at p<0.05 were fit into Logistic multivariate regression analysis. A nomogram to predict the likelihood of NSLN metastasis was developed on the basis of the results. The sensitivity and specificity of the model were measured using the area under the receiver operating characteristic (ROC) curve (AUC). R software (version 3.4.0) was used for statistical analyses and p<0.05 was considered to be statistically significant.

Results Performance Of The OSNA Assay

The Training Study The sensitivity, specificity, and accuracy of the OSNA assay compared with histological evaluation based on patient were 87.7% (121/138), 89.6% (371/414), and 89.1% (492/552), respectively.

A total of 1188 SLNs were removed (1–3, mean 2.6) from 552 patients. The sensitivity, specificity, and accuracy of the OSNA assay compared with histological evaluation based on nodes were 83.7% (159/190), 92.9% (927/998), and 91.4% (1086/1188), respectively.

The mean turnaround time of the assay (the time from receiving the nodes to acquiring the results) was 37.3 mins. There was no significant difference in the turnaround times at each center (p=0.074).

The Validation Study The sensitivity, specificity, and accuracy of the OSNA assay compared with histological evaluation based on patient were 88.7% (173/195), 91.3% (817/895), and 90.8% (990/1090), respectively.

A total of 2108 SLNs were removed (1–3, mean 1.94) from 1090 patients. The sensitivity, specificity, and accuracy of the OSNA assay compared with histological evaluation based on nodes were 88.0% (228/259), 94.7% (1750/1849), and 93.8% (1978/2108), respectively.

Construction Of The Nomogram Model In The Training Study In the training study, data of 103 patients receiving ALND as a result of positive SLNs were collected to construct the nomogram and the variables selected were analyzed by univariate correlation analysis. The clinicopathologic characteristics of 103 patients are shown in Table 1. The clinical primary tumor size (diagnosed by the ultrasound examination preoperatively, p=0.001), the vascular invasion (p=0.007), the TTL (p<0.001), the largest size of SLN metastases (p<0.001), the number of positive SLNs (p<0.001), the number of negative SLNs (p<0.001), the number of positive SLNs/total number of SLNs (p<0.001) were all associated with the presence of NSLN metastasis, while the age (p=0.940), the histological classification (p=0.993), the tumor grade (p=0.300), the status of ER (p=0.074), PR (p=0.073), and HER-2 (p=0.695) were not associated with NSLN metastasis. Logistic multivariate regression analysis was performed on the statistically significant variables, and the TTL (p=0.002), the clinical primary tumor size (p=0.013), the number of positive SLNs (p<0.001), and the number of negative SLNs (p<0.001) were found to be independent predictors. The regression coefficients were 0.48, 0.567, 1.081, and 0.661, respectively. The Logistic regression model was

Table 1 Clinicopathologic Characteristics Of Enrolled Patients In The Validation And Training Study

where P was the probability of NSLN metastasis; X1 was the Log (TTL); X2 was the clinical primary tumor size; X3 and X4 were the number of positive and negative SLNs, respectively.

According to the construction process of the MSKCC nomogram, a novel nomogram model for predicting NSLN metastasis was established by using the TTL, the clinical primary tumor size, and the number of positive and negative SLNs (Figure 1). The AUC of the model in trial 1 was 0.814 (Figure 2). As seen in the nomogram, the first line was the point assignment for each variable. Lines 2 to 5 contained the variables included in the model. For an individual patient, made the vertical line from the variable to the first line score respectively and determined the corresponding score. Added all the scores to be the total score. Finally, made the vertical line between the total score and the metastasis risk line to determine the corresponding metastasis risk value.

Figure 1 The nomogram for prediction of NSLN metastasis.Abbreviations: tumorsize, the clinical primary tumor size; NoSLNP, the number of positive SLNs; NoSLNN, the number of negative SLNs.

Figure 2 The AUCs of the nomogram for prediction of NSLN metastasis in the training study and the validation study.

Validation Of The Nomogram Model In The Validation Study In the validation study, data of 159 patients undergoing ALND as a result of positive SLNs were utilized to validate the nomogram. The clinicopathologic characteristics of 159 patients are also shown in Table 1. The AUC of the model in the validation study was 0.842 (Figure 2). The sensitivity, specificity, positive predictive value, and negative predictive value were 81.5%, 79.4%, 75.9%, and 84.4%, respectively.

A compared analysis of the model with the existing predictive model (represented by the MDA and Tenon models) was conducted on the data of the training study. The AUCs of the MDA and Tenon models were 0.745 and 0.623, respectively. The AUC of the model was statistically superior to the MDA and Tenon ones (p=0.045; p=0.016) (Table 2).

Table 2 Comparison Of The Model With The Existing Predictive Model

In addition, to explore the predictive power of the nomogram in patients with 1–3 metastases and ≥4 metastases in axillary lymph nodes, a combination data of 262 patients from the validation study and the training study encompassing 193 patients with pN1 and 69 with ≥pN2 were involved. The clinicopathologic characteristics of 262 patients are shown in Table 3. The cut-off value of the model which was sensitive to discern the patients with pN1 and ≥pN2 was 45.4% and the AUC was 0.861 (p<0.001).

Table 3 Clinicopathologic Characteristics Of Patients With Different pN Stages

Discussion

The procedure of SLNB has become the standard treatment for patients with clinical node-negative invasive carcinoma except for those patients with T4d stage disease. Results from two large randomized trials have facilitated the clinical practice of breast cancer patients with low burden SLN metastases. ACOSOG Z0011 trial has demonstrated that patients with T1–2 tumors who received breast-conserving treatment can be exempted from ALND and axillary radiotherapy for the number of metastatic SLN is merely 1–2.^2,10 EORTC 10981–22023 AMAROS trial suggests that if axillary treatment is necessary in SLN-positive patients, axillary radiotherapy will be preferable to supplant ALND for axillary radiotherapy provides similar local control with less morbidity.¹¹ Thus, for SLN-positive patients who failed to meet the criteria of the Z0011 and AMAROS trial, ALND is the standard treatment. In addition, for SLN-positive patients who undergo mastectomy and meet the criteria of the AMAROS trial, ALND is one of the alternatives. However, it has also been reported that 20% to 60% of the patients with positive SLNs have not developed to NSLN metastasis and ALND seems to be an overtreatment for these patients.^2,7,12 Therefore, the predictive nomogram model available to distinguish SLN-positive patients who have no need of ALND is of very important clinical significance. Surgeons have called for a great demand for the predictive model regarding as an indispensable model for guiding subsequent surgical treatment.

Meanwhile, radiotherapists pay more attention to the value of the predictive model as the results of MA.20 and EORTC 22922/10925 trials publish. MA.20 trial finds that additional regional nodal radiotherapy (including the internal mammary) to whole breast irradiation reduces the rate of breast cancer recurrence in node-positive and high-risk node-negative breast cancer patients.¹³ EORTC 22922/10925 study, similarly evaluating the value of additional regional nodal radiotherapy (including the internal mammary) to chest wall or whole breast irradiation, shows that regional nodal radiotherapy significantly improves the disease-free survival, the distant metastasis-free survival, and reduced the breast cancer mortality.¹⁴ As a result, the 2016 breast cancer NCCN guidelines have recommended irradiation of the internal mammary for patients with more than 4 positive axillary lymph nodes (category 1) and will strongly consider internal mammary radiation for patients with 1–3 positive axillary lymph nodes (category 2A). Because the local regional treatment of patients with axillary lymph node stage of pN1 and ≥pN2 is distinct, the predictive model may guide radiotherapists delineate the radiotherapy target more accurately. In this sense, predicting the risk of NSLN metastasis is also important for the subsequent radiotherapy.

Researchers around the world endeavor to develop models for predicting the latent risks of NSLN metastasis. The existing models contain MSKCC, Tenon, MDA, Mayo, Cambridge, Stanford, Helsinki, etc., some of which have already been applied in clinical practice.¹⁵ The characteristics of different models are shown in Table 4 as we summarize. A meta-analysis shows that the predictive accuracy of Stanford and Tenon model is influenced by the tumor burden in SLNs, while the MSKCC and Cambridge models have better predictive accuracies.¹⁶

Table 4 Characteristics Of Different Models

However, the existing predictive models are not perfect. In general, firstly, histological evaluations of SLNs are prevalent in all models and no unified standard has been confirmed. Although step section or serial section HE slides are recommended as the golden standard of SLNs by many guidelines, absolute consensus regarding the optimal interval has not yet been reached till now. In Europe, the European Working Group for Breast Screening Pathology recommends serial sectioning at an interval of 200 μm.¹⁷ In America, the American Society of Clinical Oncology guideline advises top level plus one or two step sections cut at 200–500μm intervals of each 2-mm-thick blocks.¹⁸ The American Pathological Association advises that three levels step sections of each 2-mm-thick SLN blocks.¹⁹ Secondly, it is difficult to precisely measure the maximum size of a complex three-dimensional metastasis in SLNs accurately by the conventional histological evaluation of two-dimensional sections. As a contrast, the OSNA assay was a reverse-transcription loop-mediated isothermal amplification-based assay that detected the presence of metastasis in breast lymph nodes by measuring the expression of CK-19. It can significantly minimize the sampling limitation compared to histological evaluations. Based on the patients and nodes, the good performance of the assay in this study is similar to other published researches and meta-analysis.^20–23 Thirdly, the construction of existing models was based on postoperative histological information, including tumor grade, multi-focus and vascular invasion, etc., which means that it is impossible to adopt the existing model in the operation to guide the subsequent axillary operation. In other words, the status of axillary lymph nodes can only be evaluated according to the postoperative information.

Compared with the existing ones, our model established on the basis of the intraoperative molecular diagnostics. Information of the clinical primary clinical tumor size, the TTL, and the number of positive and negative SLNs can be obtained preoperatively and intraoperatively. The mean turnaround time of the assay is less than 40 mins. As the assay results can inform surgeons just after the operation on the breast is finished, the process of the prediction process seems fitted together and compact with little if any downtime. Our model has made rapid intraoperative prediction to guide subsequent axillary treatment timely. At the same time, the model could distinguish patients with axillary lymph node stage of pN1 and ≥pN2, which was helpful to define the target area of radiotherapy postoperatively in clinical practice.

Several studies have shown that ALND can be avoided when the predictive value of MSKCC nomogram is 10%,^24,25 though the clinical utility of this standard is limited. In this study, all patients with predictive values of less than 10% do not develop to NSLN metastasis. We consider that a risk of NSLN involvement of 10% may be an acceptable cut-off value to select patients who may avoid ALND with our new nomogram. However, another prospective validation study is still needed with larger number of patients with positive SLNs who undergo additional ALND before the cut-off value will be applied to clinical practice.

Conclusion

The nomogram for predicting NSLN metastasis based on the TTL (intraoperative molecular assay), the clinical primary clinical tumor size, and the number of positive and negative SLNs will help to guide the axillary management intraoperatively and precisely confirm the target region of radiotherapy postoperatively. The nomogram model has a good prospect of application in clinical practice.

Disclosure

The authors report no conflicts of interest in this work.

References

1. Wang YS, Ou-yang T, Wu J, et al. Comparative study of one-step nucleic acid amplification assay, frozen section, and touch imprint cytology for intraoperative assessment of breast sentinel lymph node in Chinese patients. Cancer Sci. 2012;103:1989–1993. doi:10.1111/cas.12001 2. Giuliano AE, Hunt KK, Ballman KV, Beitsch PD, Whitworth PW, Blumencranz PW. Axillary dissection vs no axillary dissection in women with invasive breast cancer and sentinel node metastasis: a randomized clinical trial. JAMA. 2011;305:569–575. doi:10.1001/jama.2011.90 3. Van Zee KJ, Manasseh DM, Bevilacqua JL, et al. A nomogram for predicting the likelihood of additional nodal metastasis in breast cancer patients with a positive sentinel node biopsy. Ann Surg Oncol. 2003;10:1140–1151. doi:10.1245/ASO.2003.03.015 4. Hwang RF, Krishnamurthy S, Hunt KK, et al. Clinicopathologic factors predicting involvement of non-sentinel axillary nodes in women with breast cancer. Ann Surg Oncol. 2003;10:248–254. doi:10.1245/ASO.2003.05.020 5. Degnim AC, Reynolds C, Pantvaidya G, et al. Non-sentinel node metastasis in breast cancer patients: assessment of an existing and a new predictive nomogram. Am J Surg. 2005;190:543–550. doi:10.1016/j.amjsurg.2005.06.008 6. Barranger E, Coutant C, Flahault A, Delpech Y, Darai E, Uzan S. An axilla scoring system to predict non-sentinel lymph node status in breast cancer patients with sentinel lymph node involvement. Breast Cancer Res Treat. 2005;91:113–119. doi:10.1007/s10549-004-5781-z 7. Pal A, Provenzano E, Duffy SW, Pinder SE, Purushotham AD. A model for predicting non-sentinel lymph node metastatic disease when the sentinel lymph node is positive. Br J Surg. 2008;95:302–309. doi:10.1002/(ISSN)1365-2168 8. Kohrt HE, Olshen RA, Bermas HR, et al. New models and online calculator for predicting non-sentinel lymph node status in sentinel lymph node positive breast cancer patients. BMC Cancer. 2008;8:66. doi:10.1186/1471-2407-8-66 9. Meretoja TJ, Leidenius MH, Heikkilä PS, et al. International multicenter tool to predict the risk of non-sentinel node metastases in breast cancer. J Natl Cancer Inst. 2012;104:1888–1896. doi:10.1093/jnci/djs455 10. Giuliano AE, Ballman KV, McCall L, et al. Effect of axillary dissection vs no axillary dissection on 10-year overall survival among women with invasive breast cancer and sentinel node metastasis: the ACOSOG Z0011 (Alliance) randomized clinical trial. JAMA. 2017;318:918–926. doi:10.1001/jama.2017.11470 11. Donker M, van Tienhoven G, Straver ME, et al. Radiotherapy or surgery of the axilla after a positive sentinel node in breast cancer (EORTC 10981-22023 AMAROS): a randomised, multicentre, open-label, phase 3 non-inferiority trial. Lancet Oncol. 2014;15:1303–1310. doi:10.1016/S1470-2045(14)70460-7 12. Chen K, Zhu L, Jia W, et al. Validation and comparison of models to predict non-sentinel lymph node metastasis in breast cancer patients. Cancer Sci. 2012;103:274–281. doi:10.1111/j.1349-7006.2011.02148.x 13. Whelan TJ, Olivotto IA, Parulekar WR, et al. MA.20 Study Investigators. Regional nodal irradiation in early-stage breast cancer. N Engl J Med. 2015;373:307–316. doi:10.1056/NEJMoa1415340 14. Poortmans PM, Collette S, Kirkove C, et al. Internal mammary and medial supraclavicular irradiation in breast cancer. N Engl J Med. 2015;373:317–327. doi:10.1056/NEJMoa1415369 15. Hessman CJ, Naik AM, Kearney NM, et al. Comparative validation of online nomograms for predicting nonsentinel lymph node status in sentinel lymph node-positive breast cancer. Arch Surg. 2011;146:35–40. doi:10.1001/archsurg.2011.201 16. Zhu L, Jin L, Li S, et al. Which nomogram is best for predicting non-sentinel lymph node metastasis in breast cancer patients? A meta-analysis. Breast Cancer Res Treat. 2013;137:783–795. doi:10.1007/s10549-012-2360-6 17. Cserni G, Amendoeira I, Apostolikas N, et al. Pathological work-up of sentinel lymph nodes in breast cancer. Review of current data to be considered for the formulation of guidelines. Eur J Cancer. 2003;39:1654–67. 18. Lyman GH, Somerfield MR, Bosserman LD, Perkins CL, Weaver DL, Giuliano AE. Sentinel lymph node biopsy for patients with early-stage breast cancer: american Society of clinical oncology clinical practice guideline update. J Clin Oncol. 2017;35:561–564. doi:10.1200/JCO.2016.71.0947 19. Schwartz GF, Giuliano AE, Veronesi U. Consensus Conference Committee. Proceedings of the consensus conference on the role of sentinel lymph node biopsy in carcinoma of the breast, April 19–22, 2001, Philadelphia, Pennsylvania. Cancer 2002; 94: 2542–2551. doi:10.1002/cncr.10539 20. Visser M, Jiwa M, Horstman A, et al. Intraoperative rapid diagnostic method based on CK19 mRNA expression for the detection of lymph node metastases in breast cancer. Int J Cancer. 2008;122:2562–2567. doi:10.1002/ijc.23451 21. Schem C, Maass N, Bauerschlag DO, et al. One-step nucleic acid amplification-a molecular method for the detection of lymph node metastases in breast cancer patients; results of the German study group. Virchows Arch. 2009;454:203–210. doi:10.1007/s00428-008-0703-9 22. Tamaki Y, Akiyama F, Iwase T, et al. Molecular detection of lymph node metastases in breast cancer patients: results of a multicenter trial using the one-step nucleic acid amplification assay. Clin Cancer Res. 2009;15:2879–2884. doi:10.1158/1078-0432.CCR-08-1881 23. Shi F, Zhang Q, Liang Z, Zhang M, Liu X. One-step nucleic acid amplification assay is an accurate technique for sentinel lymph node biopsy of breast cancer patients: a meta-analysis. Br J Cancer. 2017;117:1185–1191. doi:10.1038/bjc.2017.262 24. Coutant C, Olivier C, Lambaudie E, et al. Comparison of models to predict nonsentinel lymph node status in breast cancer patients with metastatic sentinel lymph nodes: a prospective multicenter study. J Clin Oncol. 2009;27:2800–2808. doi:10.1200/JCO.2008.19.7418 25. Lambert LA, Ayers GD, Hwang RF, et al. Validation of a breast cancer nomogram for predicting nonsentinel lymph node metastasis after a positive sentinel node biopsy. Ann Surg Oncol. 2006;13:310–320. doi:10.1245/ASO.2006.03.078

Introduction Sentinel lymph node biopsy (SLNB) has become a universal procedure in node-negative breast cancer patients. It provides accurate axillary stage and reduces the morbidity associated with the unnecessary axillary lymph node dissection (ALND). One-step nucleic acid amplification (OSNA) assay (Sysmex, Kobe, Japan) is an objective molecular technique combining node tissue homogenization and subsequent reverse-transcription loop-mediated isothermal amplification of CK-19 mRNA in a single reaction. The assay can accurately detect metastases of >0.2 mm in sentinel lymph nodes (SLNs) intraoperatively. Two consequent studies from China have validated the intraoperative OSNA assay. One trial (hereinafter referred to as the training study) consisting of five centers (n=552) have confirmed the good performances of the assay in 2010.¹ Another trial (or the validation study) containing three centers (n=1090) has been conducted to accelerate the approval of the assay in China (ClinicalTrial No. CBCSG-001c). It is reported that most patients with positive SLNs does not have non-sentinel lymph node (NSLN, the other axillary lymph node except SLNs) metastasis,² and these patients cannot benefit from ALND. Therefore, it is of very remarkable clinical significance to construct a predictive model for SLN-positive patients to avoid ALND. In the training study, a novel predictive nomogram that can predict NSLN metastasis intraoperatively is constructed based on the molecular assay. Then, the nomogram is verified in the validation study. Materials And Methods Patients From February to December 2010, 552 consecutive breast cancer patients scheduled for SLNB were enrolled in the training study from five centers. From June 2015 to May 2017, 1090 consecutive breast cancer patients were enrolled in the validation study from three centers. The study was approved by the ethics committee of each center (Shandong Cancer Hospital Affiliated to Shandong University; Cancer Hospital, Tianjin Medical University; Sun Yat-sen University Cancer Center; Cancer Hospital, Beijing University; Cancer Hospital, Fudan University; Guangdong General Hospital; Cancer Hospital, Tianjin Medical University). Each patient signed the written informed consent. Patients who had undergone previous ipsilateral axillary surgery were excluded from this study. Sampling Method SLNs were defatted after SLNB. If the length was less than 4 mm, the node was sliced into two blocks along the long axis (a, b). Intraoperatively, the block a was prepared for the OSNA assay. Postoperatively, the block b was assessed by the histology evaluation. If the length was more than 4 mm, the node was sliced into four blocks (a, b, c, d). Intraoperatively, blocks a and c were prepared for the OSNA assay. Postoperatively, blocks b and d were subjected to the histology evaluation. Intraoperative OSNA Assay The OSNA assay was carried out according to the manufacturer’s instructions (Sysmex). Three different calibrators with defined CK-19 mRNA copy concentrations were used to construct a standard curve on the amplification instrument (Sysmex). Node tissues were then homogenized in 4-mL homogenizing buffer LYNORHAG (Sysmex). The homogenate was briefly centrifuged and directly used as a template for reverse-transcription loop-mediated isothermal amplification. Amplification of CK-19 mRNA was automatically performed in the amplification instrument with a ready-to-use reagent LYNOAMP (Sysmex) kit which consisted of a primer-nucleotide-mix, enzymes, and CK-19 mRNA calibrators as well as positive and negative controls. All the results were presented in semi-quantitative categories [++, +, -] and further specified by CK-19 mRNA copy number/μL: ~250 copies [-], 250–5000 copies [+], and 5000– [++]. The result [+] was comparable to the presence of a micro-metastasis and [++] to a macro-metastasis. Postoperative Histological Evaluation All node blocks used for histological evaluation were fixed in 10% buffered formalin and paraffin embedded. Four 4- to 6-μm-thick slides, 200 μm apart, were taken from each block. Metastases larger than 0.2 mm were considered positive in this study. Metastases were classified according to the 7th criterion of the American Joint Cancer Committee. Macro-metastases (≥2 mm) and micro-metastases (0.2`2 mm, pT1mic) were considered node positive. Isolated tumor cells [≤0.2 mm, ITCs, pT0(i+)] were considered node negative. Statistical Methods Variables selected from the existing predictive models such as MSKCC, MD Anderson (MDA), Mayo, Tenon, Cambridge, Stanford, Helsinki,^3–9 and the total tumor load (TTL, the sum of the CK-19 mRNA copy number/μL of all positive lymph nodes) were adopted as the clinicopathological indicators to construct the model. The data of TTL were analyzed by the Log form to avoid the influence of nonlinear data in the statistical process. Univariate correlation analysis was performed on the data to determine which indicators were predictors of NSLN metastasis. Differences in clinicopathologic indicators between the groups were assessed using the t-tests for continuous variables and the χ²-test for categorical variables. Indicators that were statistically significant at p Results Performance Of The OSNA Assay The Training Study The sensitivity, specificity, and accuracy of the OSNA assay compared with histological evaluation based on patient were 87.7% (121/138), 89.6% (371/414), and 89.1% (492/552), respectively. A total of 1188 SLNs were removed (1–3, mean 2.6) from 552 patients. The sensitivity, specificity, and accuracy of the OSNA assay compared with histological evaluation based on nodes were 83.7% (159/190), 92.9% (927/998), and 91.4% (1086/1188), respectively. The mean turnaround time of the assay (the time from receiving the nodes to acquiring the results) was 37.3 mins. There was no significant difference in the turnaround times at each center (p=0.074). The Validation Study The sensitivity, specificity, and accuracy of the OSNA assay compared with histological evaluation based on patient were 88.7% (173/195), 91.3% (817/895), and 90.8% (990/1090), respectively. A total of 2108 SLNs were removed (1–3, mean 1.94) from 1090 patients. The sensitivity, specificity, and accuracy of the OSNA assay compared with histological evaluation based on nodes were 88.0% (228/259), 94.7% (1750/1849), and 93.8% (1978/2108), respectively. Construction Of The Nomogram Model In The Training Study In the training study, data of 103 patients receiving ALND as a result of positive SLNs were collected to construct the nomogram and the variables selected were analyzed by univariate correlation analysis. The clinicopathologic characteristics of 103 patients are shown in Table 1. The clinical primary tumor size (diagnosed by the ultrasound examination preoperatively, p=0.001), the vascular invasion (p=0.007), the TTL (p Table 1 Clinicopathologic Characteristics Of Enrolled Patients In The Validation And Training Study [/RAW_REF_TEXT] where P was the probability of NSLN metastasis; X1 was the Log (TTL); X2 was the clinical primary tumor size; X3 and X4 were the number of positive and negative SLNs, respectively. According to the construction process of the MSKCC nomogram, a novel nomogram model for predicting NSLN metastasis was established by using the TTL, the clinical primary tumor size, and the number of positive and negative SLNs (Figure 1). The AUC of the model in trial 1 was 0.814 (Figure 2). As seen in the nomogram, the first line was the point assignment for each variable. Lines 2 to 5 contained the variables included in the model. For an individual patient, made the vertical line from the variable to the first line score respectively and determined the corresponding score. Added all the scores to be the total score. Finally, made the vertical line between the total score and the metastasis risk line to determine the corresponding metastasis risk value. Figure 1 The nomogram for prediction of NSLN metastasis.Abbreviations: tumorsize, the clinical primary tumor size; NoSLNP, the number of positive SLNs; NoSLNN, the number of negative SLNs. [/RAW_REF_TEXT] Figure 2 The AUCs of the nomogram for prediction of NSLN metastasis in the training study and the validation study. [/RAW_REF_TEXT] Validation Of The Nomogram Model In The Validation Study In the validation study, data of 159 patients undergoing ALND as a result of positive SLNs were utilized to validate the nomogram. The clinicopathologic characteristics of 159 patients are also shown in Table 1. The AUC of the model in the validation study was 0.842 (Figure 2). The sensitivity, specificity, positive predictive value, and negative predictive value were 81.5%, 79.4%, 75.9%, and 84.4%, respectively. A compared analysis of the model with the existing predictive model (represented by the MDA and Tenon models) was conducted on the data of the training study. The AUCs of the MDA and Tenon models were 0.745 and 0.623, respectively. The AUC of the model was statistically superior to the MDA and Tenon ones (p=0.045; p=0.016) (Table 2). Table 2 Comparison Of The Model With The Existing Predictive Model [/RAW_REF_TEXT] In addition, to explore the predictive power of the nomogram in patients with 1–3 metastases and ≥4 metastases in axillary lymph nodes, a combination data of 262 patients from the validation study and the training study encompassing 193 patients with pN1 and 69 with ≥pN2 were involved. The clinicopathologic characteristics of 262 patients are shown in Table 3. The cut-off value of the model which was sensitive to discern the patients with pN1 and ≥pN2 was 45.4% and the AUC was 0.861 (p Table 3 Clinicopathologic Characteristics Of Patients With Different pN Stages [/RAW_REF_TEXT] Discussion The procedure of SLNB has become the standard treatment for patients with clinical node-negative invasive carcinoma except for those patients with T4d stage disease. Results from two large randomized trials have facilitated the clinical practice of breast cancer patients with low burden SLN metastases. ACOSOG Z0011 trial has demonstrated that patients with T1–2 tumors who received breast-conserving treatment can be exempted from ALND and axillary radiotherapy for the number of metastatic SLN is merely 1–2.^2,10 EORTC 10981–22023 AMAROS trial suggests that if axillary treatment is necessary in SLN-positive patients, axillary radiotherapy will be preferable to supplant ALND for axillary radiotherapy provides similar local control with less morbidity.¹¹ Thus, for SLN-positive patients who failed to meet the criteria of the Z0011 and AMAROS trial, ALND is the standard treatment. In addition, for SLN-positive patients who undergo mastectomy and meet the criteria of the AMAROS trial, ALND is one of the alternatives. However, it has also been reported that 20% to 60% of the patients with positive SLNs have not developed to NSLN metastasis and ALND seems to be an overtreatment for these patients.^2,7,12 Therefore, the predictive nomogram model available to distinguish SLN-positive patients who have no need of ALND is of very important clinical significance. Surgeons have called for a great demand for the predictive model regarding as an indispensable model for guiding subsequent surgical treatment. Meanwhile, radiotherapists pay more attention to the value of the predictive model as the results of MA.20 and EORTC 22922/10925 trials publish. MA.20 trial finds that additional regional nodal radiotherapy (including the internal mammary) to whole breast irradiation reduces the rate of breast cancer recurrence in node-positive and high-risk node-negative breast cancer patients.¹³ EORTC 22922/10925 study, similarly evaluating the value of additional regional nodal radiotherapy (including the internal mammary) to chest wall or whole breast irradiation, shows that regional nodal radiotherapy significantly improves the disease-free survival, the distant metastasis-free survival, and reduced the breast cancer mortality.¹⁴ As a result, the 2016 breast cancer NCCN guidelines have recommended irradiation of the internal mammary for patients with more than 4 positive axillary lymph nodes (category 1) and will strongly consider internal mammary radiation for patients with 1–3 positive axillary lymph nodes (category 2A). Because the local regional treatment of patients with axillary lymph node stage of pN1 and ≥pN2 is distinct, the predictive model may guide radiotherapists delineate the radiotherapy target more accurately. In this sense, predicting the risk of NSLN metastasis is also important for the subsequent radiotherapy. Researchers around the world endeavor to develop models for predicting the latent risks of NSLN metastasis. The existing models contain MSKCC, Tenon, MDA, Mayo, Cambridge, Stanford, Helsinki, etc., some of which have already been applied in clinical practice.¹⁵ The characteristics of different models are shown in Table 4 as we summarize. A meta-analysis shows that the predictive accuracy of Stanford and Tenon model is influenced by the tumor burden in SLNs, while the MSKCC and Cambridge models have better predictive accuracies.¹⁶ Table 4 Characteristics Of Different Models [/RAW_REF_TEXT] However, the existing predictive models are not perfect. In general, firstly, histological evaluations of SLNs are prevalent in all models and no unified standard has been confirmed. Although step section or serial section HE slides are recommended as the golden standard of SLNs by many guidelines, absolute consensus regarding the optimal interval has not yet been reached till now. In Europe, the European Working Group for Breast Screening Pathology recommends serial sectioning at an interval of 200 μm.¹⁷ In America, the American Society of Clinical Oncology guideline advises top level plus one or two step sections cut at 200–500μm intervals of each 2-mm-thick blocks.¹⁸ The American Pathological Association advises that three levels step sections of each 2-mm-thick SLN blocks.¹⁹ Secondly, it is difficult to precisely measure the maximum size of a complex three-dimensional metastasis in SLNs accurately by the conventional histological evaluation of two-dimensional sections. As a contrast, the OSNA assay was a reverse-transcription loop-mediated isothermal amplification-based assay that detected the presence of metastasis in breast lymph nodes by measuring the expression of CK-19. It can significantly minimize the sampling limitation compared to histological evaluations. Based on the patients and nodes, the good performance of the assay in this study is similar to other published researches and meta-analysis.^20–23 Thirdly, the construction of existing models was based on postoperative histological information, including tumor grade, multi-focus and vascular invasion, etc., which means that it is impossible to adopt the existing model in the operation to guide the subsequent axillary operation. In other words, the status of axillary lymph nodes can only be evaluated according to the postoperative information. Compared with the existing ones, our model established on the basis of the intraoperative molecular diagnostics. Information of the clinical primary clinical tumor size, the TTL, and the number of positive and negative SLNs can be obtained preoperatively and intraoperatively. The mean turnaround time of the assay is less than 40 mins. As the assay results can inform surgeons just after the operation on the breast is finished, the process of the prediction process seems fitted together and compact with little if any downtime. Our model has made rapid intraoperative prediction to guide subsequent axillary treatment timely. At the same time, the model could distinguish patients with axillary lymph node stage of pN1 and ≥pN2, which was helpful to define the target area of radiotherapy postoperatively in clinical practice. Several studies have shown that ALND can be avoided when the predictive value of MSKCC nomogram is 10%,^24,25 though the clinical utility of this standard is limited. In this study, all patients with predictive values of less than 10% do not develop to NSLN metastasis. We consider that a risk of NSLN involvement of 10% may be an acceptable cut-off value to select patients who may avoid ALND with our new nomogram. However, another prospective validation study is still needed with larger number of patients with positive SLNs who undergo additional ALND before the cut-off value will be applied to clinical practice. Conclusion The nomogram for predicting NSLN metastasis based on the TTL (intraoperative molecular assay), the clinical primary clinical tumor size, and the number of positive and negative SLNs will help to guide the axillary management intraoperatively and precisely confirm the target region of radiotherapy postoperatively. The nomogram model has a good prospect of application in clinical practice. Disclosure The authors report no conflicts of interest in this work. References [RAW_REF_TEXT] 1. Wang YS, Ou-yang T, Wu J, et al. Comparative study of one-step nucleic acid amplification assay, frozen section, and touch imprint cytology for intraoperative assessment of breast sentinel lymph node in Chinese patients. Cancer Sci. 2012;103:1989–1993. doi:10.1111/cas.12001 [/RAW_REF_TEXT] [RAW_REF_TEXT] 2. Giuliano AE, Hunt KK, Ballman KV, Beitsch PD, Whitworth PW, Blumencranz PW. Axillary dissection vs no axillary dissection in women with invasive breast cancer and sentinel node metastasis: a randomized clinical trial. JAMA. 2011;305:569–575. doi:10.1001/jama.2011.90 [/RAW_REF_TEXT] [RAW_REF_TEXT] 3. Van Zee KJ, Manasseh DM, Bevilacqua JL, et al. A nomogram for predicting the likelihood of additional nodal metastasis in breast cancer patients with a positive sentinel node biopsy. Ann Surg Oncol. 2003;10:1140–1151. doi:10.1245/ASO.2003.03.015 [/RAW_REF_TEXT] [RAW_REF_TEXT] 4. Hwang RF, Krishnamurthy S, Hunt KK, et al. Clinicopathologic factors predicting involvement of non-sentinel axillary nodes in women with breast cancer. Ann Surg Oncol. 2003;10:248–254. doi:10.1245/ASO.2003.05.020 [/RAW_REF_TEXT] [RAW_REF_TEXT] 5. Degnim AC, Reynolds C, Pantvaidya G, et al. Non-sentinel node metastasis in breast cancer patients: assessment of an existing and a new predictive nomogram. Am J Surg. 2005;190:543–550. doi:10.1016/j.amjsurg.2005.06.008 [/RAW_REF_TEXT] [RAW_REF_TEXT] 6. Barranger E, Coutant C, Flahault A, Delpech Y, Darai E, Uzan S. An axilla scoring system to predict non-sentinel lymph node status in breast cancer patients with sentinel lymph node involvement. Breast Cancer Res Treat. 2005;91:113–119. doi:10.1007/s10549-004-5781-z [/RAW_REF_TEXT] [RAW_REF_TEXT] 7. Pal A, Provenzano E, Duffy SW, Pinder SE, Purushotham AD. A model for predicting non-sentinel lymph node metastatic disease when the sentinel lymph node is positive. Br J Surg. 2008;95:302–309. doi:10.1002/(ISSN)1365-2168 [/RAW_REF_TEXT] [RAW_REF_TEXT] 8. Kohrt HE, Olshen RA, Bermas HR, et al. New models and online calculator for predicting non-sentinel lymph node status in sentinel lymph node positive breast cancer patients. BMC Cancer. 2008;8:66. doi:10.1186/1471-2407-8-66 [/RAW_REF_TEXT] [RAW_REF_TEXT] 9. Meretoja TJ, Leidenius MH, Heikkilä PS, et al. International multicenter tool to predict the risk of non-sentinel node metastases in breast cancer. J Natl Cancer Inst. 2012;104:1888–1896. doi:10.1093/jnci/djs455 [/RAW_REF_TEXT] [RAW_REF_TEXT] 10. Giuliano AE, Ballman KV, McCall L, et al. Effect of axillary dissection vs no axillary dissection on 10-year overall survival among women with invasive breast cancer and sentinel node metastasis: the ACOSOG Z0011 (Alliance) randomized clinical trial. JAMA. 2017;318:918–926. doi:10.1001/jama.2017.11470 [/RAW_REF_TEXT] [RAW_REF_TEXT] 11. Donker M, van Tienhoven G, Straver ME, et al. Radiotherapy or surgery of the axilla after a positive sentinel node in breast cancer (EORTC 10981-22023 AMAROS): a randomised, multicentre, open-label, phase 3 non-inferiority trial. Lancet Oncol. 2014;15:1303–1310. doi:10.1016/S1470-2045(14)70460-7 [/RAW_REF_TEXT] [RAW_REF_TEXT] 12. Chen K, Zhu L, Jia W, et al. Validation and comparison of models to predict non-sentinel lymph node metastasis in breast cancer patients. Cancer Sci. 2012;103:274–281. doi:10.1111/j.1349-7006.2011.02148.x [/RAW_REF_TEXT] [RAW_REF_TEXT] 13. Whelan TJ, Olivotto IA, Parulekar WR, et al. MA.20 Study Investigators. Regional nodal irradiation in early-stage breast cancer. N Engl J Med. 2015;373:307–316. doi:10.1056/NEJMoa1415340 [/RAW_REF_TEXT] [RAW_REF_TEXT] 14. Poortmans PM, Collette S, Kirkove C, et al. Internal mammary and medial supraclavicular irradiation in breast cancer. N Engl J Med. 2015;373:317–327. doi:10.1056/NEJMoa1415369 [/RAW_REF_TEXT] [RAW_REF_TEXT] 15. Hessman CJ, Naik AM, Kearney NM, et al. Comparative validation of online nomograms for predicting nonsentinel lymph node status in sentinel lymph node-positive breast cancer. Arch Surg. 2011;146:35–40. doi:10.1001/archsurg.2011.201 [/RAW_REF_TEXT] [RAW_REF_TEXT] 16. Zhu L, Jin L, Li S, et al. Which nomogram is best for predicting non-sentinel lymph node metastasis in breast cancer patients? A meta-analysis. Breast Cancer Res Treat. 2013;137:783–795. doi:10.1007/s10549-012-2360-6 [/RAW_REF_TEXT] [RAW_REF_TEXT] 17. Cserni G, Amendoeira I, Apostolikas N, et al. Pathological work-up of sentinel lymph nodes in breast cancer. Review of current data to be considered for the formulation of guidelines. Eur J Cancer. 2003;39:1654–67. [/RAW_REF_TEXT] [RAW_REF_TEXT] 18. Lyman GH, Somerfield MR, Bosserman LD, Perkins CL, Weaver DL, Giuliano AE. Sentinel lymph node biopsy for patients with early-stage breast cancer: american Society of clinical oncology clinical practice guideline update. J Clin Oncol. 2017;35:561–564. doi:10.1200/JCO.2016.71.0947 [/RAW_REF_TEXT] [RAW_REF_TEXT] 19. Schwartz GF, Giuliano AE, Veronesi U. Consensus Conference Committee. Proceedings of the consensus conference on the role of sentinel lymph node biopsy in carcinoma of the breast, April 19–22, 2001, Philadelphia, Pennsylvania. Cancer 2002; 94: 2542–2551. doi:10.1002/cncr.10539 [/RAW_REF_TEXT] [RAW_REF_TEXT] 20. Visser M, Jiwa M, Horstman A, et al. Intraoperative rapid diagnostic method based on CK19 mRNA expression for the detection of lymph node metastases in breast cancer. Int J Cancer. 2008;122:2562–2567. doi:10.1002/ijc.23451 [/RAW_REF_TEXT] [RAW_REF_TEXT] 21. Schem C, Maass N, Bauerschlag DO, et al. One-step nucleic acid amplification-a molecular method for the detection of lymph node metastases in breast cancer patients; results of the German study group. Virchows Arch. 2009;454:203–210. doi:10.1007/s00428-008-0703-9 [/RAW_REF_TEXT] [RAW_REF_TEXT] 22. Tamaki Y, Akiyama F, Iwase T, et al. Molecular detection of lymph node metastases in breast cancer patients: results of a multicenter trial using the one-step nucleic acid amplification assay. Clin Cancer Res. 2009;15:2879–2884. doi:10.1158/1078-0432.CCR-08-1881 [/RAW_REF_TEXT] [RAW_REF_TEXT] 23. Shi F, Zhang Q, Liang Z, Zhang M, Liu X. One-step nucleic acid amplification assay is an accurate technique for sentinel lymph node biopsy of breast cancer patients: a meta-analysis. Br J Cancer. 2017;117:1185–1191. doi:10.1038/bjc.2017.262 [/RAW_REF_TEXT] [RAW_REF_TEXT] 24. Coutant C, Olivier C, Lambaudie E, et al. Comparison of models to predict nonsentinel lymph node status in breast cancer patients with metastatic sentinel lymph nodes: a prospective multicenter study. J Clin Oncol. 2009;27:2800–2808. doi:10.1200/JCO.2008.19.7418 [/RAW_REF_TEXT] [RAW_REF_TEXT] 25. Lambert LA, Ayers GD, Hwang RF, et al. Validation of a breast cancer nomogram for predicting nonsentinel lymph node metastasis after a positive sentinel node biopsy. Ann Surg Oncol. 2006;13:310–320. doi:10.1245/ASO.2006.03.078 [/RAW_REF_TEXT]

References

1. Wang YS, Ou-yang T, Wu J, et al. Comparative study of one-step nucleic acid amplification assay, frozen section, and touch imprint cytology for intraoperative assessment of breast sentinel lymph node in Chinese patients. Cancer Sci. 2012;103:1989–1993. doi:10.1111/cas.12001

2. Giuliano AE, Hunt KK, Ballman KV, Beitsch PD, Whitworth PW, Blumencranz PW. Axillary dissection vs no axillary dissection in women with invasive breast cancer and sentinel node metastasis: a randomized clinical trial. JAMA. 2011;305:569–575. doi:10.1001/jama.2011.90

3. Van Zee KJ, Manasseh DM, Bevilacqua JL, et al. A nomogram for predicting the likelihood of additional nodal metastasis in breast cancer patients with a positive sentinel node biopsy. Ann Surg Oncol. 2003;10:1140–1151. doi:10.1245/ASO.2003.03.015

4. Hwang RF, Krishnamurthy S, Hunt KK, et al. Clinicopathologic factors predicting involvement of non-sentinel axillary nodes in women with breast cancer. Ann Surg Oncol. 2003;10:248–254. doi:10.1245/ASO.2003.05.020

5. Degnim AC, Reynolds C, Pantvaidya G, et al. Non-sentinel node metastasis in breast cancer patients: assessment of an existing and a new predictive nomogram. Am J Surg. 2005;190:543–550. doi:10.1016/j.amjsurg.2005.06.008

6. Barranger E, Coutant C, Flahault A, Delpech Y, Darai E, Uzan S. An axilla scoring system to predict non-sentinel lymph node status in breast cancer patients with sentinel lymph node involvement. Breast Cancer Res Treat. 2005;91:113–119. doi:10.1007/s10549-004-5781-z

7. Pal A, Provenzano E, Duffy SW, Pinder SE, Purushotham AD. A model for predicting non-sentinel lymph node metastatic disease when the sentinel lymph node is positive. Br J Surg. 2008;95:302–309. doi:10.1002/(ISSN)1365-2168

8. Kohrt HE, Olshen RA, Bermas HR, et al. New models and online calculator for predicting non-sentinel lymph node status in sentinel lymph node positive breast cancer patients. BMC Cancer. 2008;8:66. doi:10.1186/1471-2407-8-66

9. Meretoja TJ, Leidenius MH, Heikkilä PS, et al. International multicenter tool to predict the risk of non-sentinel node metastases in breast cancer. J Natl Cancer Inst. 2012;104:1888–1896. doi:10.1093/jnci/djs455

10. Giuliano AE, Ballman KV, McCall L, et al. Effect of axillary dissection vs no axillary dissection on 10-year overall survival among women with invasive breast cancer and sentinel node metastasis: the ACOSOG Z0011 (Alliance) randomized clinical trial. JAMA. 2017;318:918–926. doi:10.1001/jama.2017.11470

11. Donker M, van Tienhoven G, Straver ME, et al. Radiotherapy or surgery of the axilla after a positive sentinel node in breast cancer (EORTC 10981-22023 AMAROS): a randomised, multicentre, open-label, phase 3 non-inferiority trial. Lancet Oncol. 2014;15:1303–1310. doi:10.1016/S1470-2045(14)70460-7

12. Chen K, Zhu L, Jia W, et al. Validation and comparison of models to predict non-sentinel lymph node metastasis in breast cancer patients. Cancer Sci. 2012;103:274–281. doi:10.1111/j.1349-7006.2011.02148.x

13. Whelan TJ, Olivotto IA, Parulekar WR, et al. MA.20 Study Investigators. Regional nodal irradiation in early-stage breast cancer. N Engl J Med. 2015;373:307–316. doi:10.1056/NEJMoa1415340

14. Poortmans PM, Collette S, Kirkove C, et al. Internal mammary and medial supraclavicular irradiation in breast cancer. N Engl J Med. 2015;373:317–327. doi:10.1056/NEJMoa1415369

15. Hessman CJ, Naik AM, Kearney NM, et al. Comparative validation of online nomograms for predicting nonsentinel lymph node status in sentinel lymph node-positive breast cancer. Arch Surg. 2011;146:35–40. doi:10.1001/archsurg.2011.201

16. Zhu L, Jin L, Li S, et al. Which nomogram is best for predicting non-sentinel lymph node metastasis in breast cancer patients? A meta-analysis. Breast Cancer Res Treat. 2013;137:783–795. doi:10.1007/s10549-012-2360-6

17. Cserni G, Amendoeira I, Apostolikas N, et al. Pathological work-up of sentinel lymph nodes in breast cancer. Review of current data to be considered for the formulation of guidelines. Eur J Cancer. 2003;39:1654–67.

18. Lyman GH, Somerfield MR, Bosserman LD, Perkins CL, Weaver DL, Giuliano AE. Sentinel lymph node biopsy for patients with early-stage breast cancer: american Society of clinical oncology clinical practice guideline update. J Clin Oncol. 2017;35:561–564. doi:10.1200/JCO.2016.71.0947

19. Schwartz GF, Giuliano AE, Veronesi U. Consensus Conference Committee. Proceedings of the consensus conference on the role of sentinel lymph node biopsy in carcinoma of the breast, April 19–22, 2001, Philadelphia, Pennsylvania. Cancer 2002; 94: 2542–2551. doi:10.1002/cncr.10539

20. Visser M, Jiwa M, Horstman A, et al. Intraoperative rapid diagnostic method based on CK19 mRNA expression for the detection of lymph node metastases in breast cancer. Int J Cancer. 2008;122:2562–2567. doi:10.1002/ijc.23451

21. Schem C, Maass N, Bauerschlag DO, et al. One-step nucleic acid amplification-a molecular method for the detection of lymph node metastases in breast cancer patients; results of the German study group. Virchows Arch. 2009;454:203–210. doi:10.1007/s00428-008-0703-9

22. Tamaki Y, Akiyama F, Iwase T, et al. Molecular detection of lymph node metastases in breast cancer patients: results of a multicenter trial using the one-step nucleic acid amplification assay. Clin Cancer Res. 2009;15:2879–2884. doi:10.1158/1078-0432.CCR-08-1881

23. Shi F, Zhang Q, Liang Z, Zhang M, Liu X. One-step nucleic acid amplification assay is an accurate technique for sentinel lymph node biopsy of breast cancer patients: a meta-analysis. Br J Cancer. 2017;117:1185–1191. doi:10.1038/bjc.2017.262

24. Coutant C, Olivier C, Lambaudie E, et al. Comparison of models to predict nonsentinel lymph node status in breast cancer patients with metastatic sentinel lymph nodes: a prospective multicenter study. J Clin Oncol. 2009;27:2800–2808. doi:10.1200/JCO.2008.19.7418

25. Lambert LA, Ayers GD, Hwang RF, et al. Validation of a breast cancer nomogram for predicting nonsentinel lymph node metastasis after a positive sentinel node biopsy. Ann Surg Oncol. 2006;13:310–320. doi:10.1245/ASO.2006.03.078