Metformin and Gastrointestinal Cancer Development in Newly Diagnosed Type 2 Diabetes: A Population-Based Study in Korea

Abstract

INTRODUCTION:

Clinical studies have produced conflicting results on the effects of metformin on gastrointestinal cancer development. We aimed to investigate the association between metformin use and stomach, colon, liver, and pancreatic cancer development among patients with newly diagnosed, drug-naïve type 2 diabetes.

METHODS:

This retrospective study evaluated propensity score-matched patients with newly diagnosed type 2 diabetes from the Korean National Health Insurance Service database. Metformin users were categorized into tertiles according to the cumulative dose or duration of metformin treatment, and the risks of gastrointestinal cancers were compared.

RESULTS:

Metformin users had reduced risks of developing stomach cancer (hazard ratio [HR]: 0.841, 95% confidence interval [CI]: 0.797–0.887), colon cancer (HR: 0.865, 95% CI: 0.822–0.91), and liver cancer (HR: 0.709, 95% CI: 0.675–0.746; P < 0.001). However, metformin users did not have a reduced overall risk of pancreatic cancer (HR: 1.335, 95% CI: 1.209–1.475; P < 0.001). The risks tended to decrease at higher cumulative doses and durations of metformin use, with significantly reduced risks of all 4 cancers at the highest cumulative dose (≥1,200,000 mg) and the longest duration (≥2,000 days) of metformin use.

DISCUSSION:

This population-based data suggest that metformin could be associated with reductions in the risks of stomach, colon, and liver cancers, as well a reduced risk of pancreatic cancer in some subgroups. Metformin has benefit as a first-line treatment for type 2 diabetes mellitus. A further role in cancer risk reduction could be studied in controlled trials.

INTRODUCTION

Diabetes mellitus (DM) is a common chronic endocrine disease that is a known risk factor for many types of cancer. The association between DM and cancer development is associated with insulin resistance, hyperinsulinemia, and the insulin-like growth factor (IGF) system (1). DM is known to be associated with gastrointestinal (GI) cancer, which has high global rates of incidence and mortality (2). Moreover, the consensus report by the American Diabetes Association and the American Cancer Society indicated that DM is associated with an increased risk of developing cancers involving the liver, pancreas, colon, and rectum (3).

Metformin is the first-line medication for type 2 DM, and preclinical studies have indicated that it also has anticancer effects (4). This drug mainly increases the insulin sensitivity of tissues and thereby reduces insulin levels (5), although metformin also activates AMP-activated protein kinase (AMPK). This kinase negatively regulates the mammalian target of rapamycin (mTOR) pathway, which is known to inhibit protein synthesis and cell proliferation (6). Previous studies have indicated that metformin inhibits GI cancer development in vivo and in vitro via various signal pathways (7–9). These pathways may explain how metformin influences the process of cancer development.

Despite these preclinical findings, clinical and epidemiological studies have reported conflicting results on the association between metformin use and GI cancer development. Several meta-analyses have indicated that metformin use reduced the risk of developing GI cancers, including stomach, colon, liver, and pancreatic cancers (10–13). By contrast, other studies that were not included in these meta-analyses reported conflicting results. de Jong et al. (14) reported that metformin use was not associated with a decreased risk of any type of GI cancer, whereas Bodmer et al. (15) reported that metformin use was not associated with a decreased risk of colorectal cancer.

There are limited data on the association between metformin use and GI cancer in Asian populations because previous studies have generally focused on Western populations. However, Asian populations have a rapidly increasing prevalence of type 2 DM (16), and the burden of GI cancer is also increasing in Asian countries (17), possibly because of changes in lifestyle and other socioenvironmental factors. Therefore, it is important to examine the association between metformin use and GI cancer development among Asian individuals.

This study aimed to investigate the association between metformin use and GI cancer development among patients with newly diagnosed, drug-naïve type 2 DM using claims data from the Korean National Health Insurance Service (KNHIS) database. We also evaluated the effects of metformin use on the development of GI cancer according to age, sex, cumulative metformin dose, and duration of metformin use.

METHODS

Data source

This retrospective study evaluated claims from the KNHIS database that were made between January 1, 2005, and December 31, 2014. The KNHIS was launched by the Korean government in February 1999 and is the only public medical insurance system in Korea. This system is a compulsory social insurance system that covers the entire Korean population. All clinics and hospitals submit patient data, including information on diagnoses and medical costs, to the KNHIS database. The KNHIS then provides deidentified claims data to researchers. This study used cohort data (NHIS-2020-1-389) provided by the KNHIS. The KNHIS database has been used extensively for various epidemiologic research, and its background and data configuration have been described elsewhere in detail (18,19). The study protocol was approved by the institutional review board of the National Health Insurance Service Ilsan Hospital (2016-03-004). Written informed consent from individual subjects was waived.

Study population

This study evaluated subjects with newly diagnosed type 2 diabetes, which was determined using the International Classification of Diseases, 10th Revision, Clinical Modification codes E11–E14 (16). These subjects had been prescribed one or more antidiabetic drugs at least twice between January 2005 and December 2009. We selected subjects with newly diagnosed diabetes by excluding subjects with previous claims for any antidiabetic drugs or any diabetes-related claims codes during a 3-year washout period before January 2005. Based on these conditions, we identified 1,322,981 eligible subjects. We further refined the cohort to exclude subjects with any cancer diagnosis before their DM diagnosis, subjects who were not health insurance subscribers, subjects who were followed for <180 days after their DM diagnosis, and subjects who were exposed to metformin more than once in the nonuser group. We also excluded subjects who had received >3 insulin prescriptions from an outpatient clinic during the study period because those subjects were presumed to have uncontrolled type 2 diabetes at the time of diagnosis.

Definition of metformin exposure

Metformin users were defined as subjects who had been prescribed metformin for ≥180 days in a 365-day period. Metformin nonusers were defined as subjects who had never been prescribed metformin or who had received metformin for <180 days during the study period. Based on these definitions, we identified 578,207 metformin users and 149,605 metformin nonusers. We then performed 1:1 propensity score matching according to age, sex, region of residence, income level, and use of nonmetformin antidiabetic drugs. The final analysis included a matched sample of 131,877 metformin users and 131,877 metformin nonusers (Figure 1). To assess the effect of cumulative metformin exposure on GI cancer development, we evaluated the cumulative prescribed dose (mg/d × days of use) and the cumulative treatment duration (days), with subjects divided into tertiles for each parameter.

Figure 1.

Study flowchart.

[Figure omitted. See PDF]

Definitions and covariates

We investigated the development of 4 types of GI cancer (stomach, colon, liver, and pancreatic cancers), which were identified based on the corresponding International Classification of Diseases, 10th Revision, Clinical Modification codes (stomach cancer: C16, colon cancer: C18–C21, liver cancer: C22, and pancreatic cancer: C25). Cases were considered to have developed these cancers if their claim included the associated code and was submitted between January 2005 and December 2014. The follow-up durations were calculated from the date of study inclusion (the year in which the patient was diagnosed with DM or started antidiabetic medication) to the date of GI cancer diagnosis, death, or the end of the study.

Economic status was divided into quartiles. The region of residence was divided into 3 groups: Seoul (the capital city of Korea), metropolitan cities (Busan, Daegu, Daejeon, Gwangju, Incheon, and Ulsan), and rural areas. Other antidiabetic medications included sulfonylureas, thiazolidinediones, dipeptidyl peptidase-4 inhibitors, alpha-glucosidase inhibitors, and meglitinide. Use of these nonmetformin antidiabetic medications (a categorical variable) was identified using claims data.

Statistical analyses

All data were analyzed using SAS software (version 9.4; SAS Institute, Cary, NC). Metformin users and nonusers were compared using Cox proportional hazard regression models, which were adjusted for age, sex, year of enrollment, economic status, region of residence, and the use of nonmetformin antidiabetic medications. The results of the analyses were reported as hazard ratios (HRs) and 95% confidence intervals (CIs). Differences were considered statistically significant at P-values of <0.05.

RESULTS

Baseline characteristics

The final cohort included a matched sample of 131,877 metformin users and 131,877 metformin nonusers. Table 1 shows the subjects' age at diagnosis, sex, year of study enrollment, region of residency, and nonmetformin antidiabetic drugs. Metformin users were more likely to be older (mean age: 60.9 ± 13.9 years vs 60.7 ± 15.4 years), female subjects (50.1% vs 49.1%), and to have a higher income. Metformin nonusers accounted for a higher proportion of the subjects during the early study period. There were no significant differences in the region of residency and the use of nonmetformin antidiabetic drugs (with the exception of alpha-glucosidase inhibitor use). The mean daily dose of metformin was 531 ± 161 mg, the mean cumulative dose was 860,982 ± 562,409 mg, and the mean cumulative treatment duration was 1,618 ± 911 days. Stomach, colon, and liver cancers were less common among metformin users than among metformin nonusers. However, pancreatic cancer was more common among metformin users than among metformin nonusers. Among 19,458 subjects diagnosed with GI cancers, there were 197 subjects (1.01%) aged younger than 40 years (see Table 1, Supplemental Digital Content 1, http://links.lww.com/CTG/A410).

Table 1.

Baseline characteristics of the study population

[Figure omitted. See PDF]

[Figure omitted. See PDF]

	Metformin nonusers		Metformin users
	N	%	N	%
	131,877		131,877
Age (yr)a
Mean (SD)	60.9 (±13.9)		60.7 (±15.4)
0–14	536	0.4	438	0.3
15–20	344	0.3	469	0.4
20–24	543	0.4	449	0.3
25–29	1,640	1.2	740	0.6
30–34	2,721	2.1	1,566	1.2
35–39	3,989	3.0	3,903	3.0
40–44	7,784	5.9	6,682	5.1
45–49	12,252	9.3	11,655	8.8
50–54	14,136	10.7	15,061	11.4
55–59	15,366	11.7	13,925	10.6
60–64	16,275	12.4	20,061	15.2
65–60	19,031	14.4	20,123	15.3
70–74	14,177	10.8	18,320	13.9
75–79	11,365	8.6	11,283	8.6
≥80	11,618	8.8	7,202	5.5
Sexa
Male	67,121	50.9	65,864	49.9
Female	64,756	49.1	66,013	50.1
Year of study enrollmenta
2005	32,338	24.5	30,969	23.5
2006	28,897	21.9	23,513	17.8
2007	25,700	19.5	24,672	18.7
2008	24,100	18.3	25,964	19.7
2009	20,842	15.8	26,759	20.3
Economic statusa
1st quartile	39,870	30.2	38,399	29.1
2nd quartile	24,320	18.4	23,832	18.1
3rd quartile	30,628	23.2	30,950	23.5
4th quartile	37,059	28.1	38,696	29.3
Region of residence
Seoul (capital city)	28,803	21.8	29,229	22.2
Metropolitan cities	32,893	24.9	33,180	25.2
Others	70,181	53.2	69,468	52.7
Antidiabetic drugs
Metformin
Yes	0	0	131,877	100
No	131,877	100	0
Sulfonylurea
Yes	61,341	46.5	61,399	46.6
No	70,536	53.5	70,478	53.4
TZD
Yes	5,261	4.0	5,288	4.0
No	126,616	96.0	126,589	96.0
DPP4i
Yes	2,384	1.8	2,383	1.8
No	129,493	98.2	129,494	98.2
AGIa
Yes	11,455	8.7	10,994	8.3
No	120,422	91.3	120,883	91.7
Meglitinide
Yes	3,974	3.0	4,004	3.0
No	127,903	97.0	127,873	97.0
Daily metformin dose (mg)
Mean (range)	127,903	97.0	531 (161)	—
Cumulative metformin dose (mg)
Mean (range)			860,982 (562,409)	—
None	131,877	100	0	0
0–239,999	—	—	19,775	15.0
240,000–1,200,000	—	—	79,614	60.4
≥1,200,000	—	—	32,488	24.6
Cumulative duration of therapy (d)
Mean (range)			1,618 (911)	—
None	131,877	100	0	0
0–469	—	—	19,424	14.7
470–1,999	—	—	62,857	47.7
≥2,000	—	—	49,596	37.6
Cancer cases
Stomach	2,872		2,459
Colon	3,222		2,862
Liver	3,770		2,678
Pancreas	677		918

AGI, alpha-glucosidase inhibitors; DPP4i, dipeptidyl peptidase-4 inhibitors; TZD, thiazolidinediones.

a

P < 0.001.

Cancer risks according to metformin exposure

Metformin users had lower incidence rates of stomach, colon, and liver cancers, although they had a higher incidence rate of pancreatic cancer (Table 2). The multivariate analyses revealed that metformin use was associated with significantly lower risks of developing stomach cancer (HR: 0.841, 95% CI: 0.797–0.887; P < 0.001), colon cancer (HR: 0.865, 95% CI: 0.822–0.91; P < 0.001), and liver cancer (HR: 0.709, 95% CI: 0.675–0.746; P < 0.001). However, metformin use was associated with a significantly increased risk of pancreatic cancer (HR: 1.335, 95% CI: 1.209–1.475; P < 0.001).

Table 2.

Risks of developing gastrointestinal cancers according to metformin use

[Figure omitted. See PDF]

	Cancer cases		Person-yr		Incidence rate of cancer (%)		Incidence of cancer (per 100,000 person-yr)		HR (95% CI)
	Metformin nonusers	Metformin users	Metformin nonusers	Metformin users	Metformin nonusers	Metformin users	Metformin nonusers	Metformin users	Univariate	Multivariatea
Stomach	2,872	2,459	899,702	903,861	2.18	1.86	319.2	272.1	0.860	0.841
									(0.815–0.908)b	(0.797–0.887)b
Colon	3,222	2,862	899,702	903,861	2.44	2.17	358.1	316.6	0.891	0.865
									(0.847–0.937)b	(0.822–0.910)b
Liver	3,770	2,678	899,702	903,861	2.86	2.03	419.0	296.3	0.708	0.709
									(0.673–0.744)b	(0.675–0.746)b
Pancreas	677	918	899,702	903,861	0.51	0.70	75.2	101.6	1.369	1.335
									(1.239–1.511)b	(1.209–1.475)b

CI, confidence interval; HR, hazard ratio.

a

The HRs were adjusted for age, sex, economic status, region of residency, and antidiabetic medications (sulfonylureas, thiazolidinediones, dipeptidyl peptidase 4 inhibitors, alpha-glucosidase inhibitors, and meglitinide). The results of the univariate and multivariate analyses were significant, relative to the metformin nonuser group.

b

P < 0.001.

Among subjects who were younger than 50 years of age at diagnosis, metformin use was associated with significantly lower risks of stomach (HR: 0.728, 95% CI: 0.605–0.877; P < 0.001) and liver cancers (HR: 0.634, 95% CI: 0.570–0.705; P < 0.001) (see Table 2, Supplemental Digital Content 1, http://links.lww.com/CTG/A410). Similarly, among subjects who were 50 years of age or older at diagnosis, metformin use was associated with the decreased risks of stomach (HR: 0.857; 95% CI: 0.810–0.906), liver (HR: 0.734, 95% CI: 0.694–0.777), and colon cancers (HR: 0.871, 95% CI: 0.826–0.918), although metformin use in this group was also associated with an increased risk of pancreatic cancer (HR: 1.451, 95% CI: 1.304–1.615; P < 0.001).

Among both men and women, metformin users had lower risks of stomach, colon, and liver cancers, although not of pancreatic cancer (see Table 2, Supplemental Digital Content 1, http://links.lww.com/CTG/A410). Furthermore, the cumulative dose and duration of metformin exposure seemed to influence its effects on the development of GI cancers, with increasing exposure generally associated with decreasing risks of developing all 4 types of GI cancers (see Table 3, Supplemental Digital Content 1, http://links.lww.com/CTG/A410). The risks of stomach and liver cancers were reduced at a cumulative metformin dose of 240,000–1,200,000 mg, whereas the risks of colon and pancreatic cancers were reduced at a cumulative dose of ≥1,200,000 mg. The risks of all 4 cancers were reduced when the exposure duration was ≥2,000 days.

DISCUSSION

This study revealed that metformin use for newly diagnosed, drug-naïve type 2 DM was associated with a reduced risk of developing GI cancer in the Korean population. Although metformin use was associated with an overall reduced risk of developing stomach, colon, and liver cancers, it was not associated with a reduced risk of pancreatic cancer. However, subgroup analyses revealed that the highest cumulative dose (≥1,200,000 mg) and longest exposure duration (≥2,000 days) were associated with reduced risks of all 4 GI cancers, with increasing cumulative dose and duration tending to be associated with decreased cancer risks.

Findings from preclinical studies have indicated that metformin has anticancer effects (20), which are mainly mediated by the activation of AMPK. This activation downregulates the mTOR complex and the IGF-1/Akt pathway, leading to p53-mediated cell cycle arrest (21). Metformin inhibits the IGF-1 pathway through AMPK-dependent phosphorylation of insulin receptor substrate-1, which transmits signals from insulin and IGF-1 receptors to the phosphatidylinositide 3-kinase/Akt pathway and subsequently downregulates mTOR signaling. Metformin also affects p53-controlled pathways through AMPK-mediated p53 phosphorylation. Furthermore, metformin induces and alters the expression of microRNAs which are associated with antitumor mechanisms. These pathways help drive suppressive mechanisms that contribute to cancer cell death. Several preclinical studies have evaluated this issue using different cancer cell lines and models. Kato et al. (22) reported that metformin inhibited the proliferation of human gastric cancer cell lines and demonstrated marked alteration of various microRNA expressions in vitro and in vivo (7), which might also contribute to gastric cancer development. Miyoshi et al. (9) reported that metformin inhibited the proliferation of hepatocellular carcinoma cells and blocked the cell cycle in G0/G1.

Based on these preclinical findings, clinical studies have evaluated the association between metformin use and GI cancer development, although the results have conflicted. A meta-analysis of 7 cohort studies revealed that metformin use for type 2 DM was associated with a reduced risk of gastric cancer (HR: 0.763, 95% CI: 0.642–0.905; P = 0.0001) (23), whereas a separate meta-analysis of 4 studies revealed that metformin use for type 2 DM was associated with a reduced risk of colorectal cancer (relative ratio: 0.63, 95% CI: 0.47–0.84; P = 0.002) (13). Another meta-analysis of 5 studies revealed that metformin use for type 2 DM was associated with a reduced risk of liver cancer (odds ratio: 0.38, 95% CI: 0.24–0.59; P < 0.001) (10), and a meta-analysis of 4 studies reported that metformin use for type 2 diabetes was associated with a reduced risk of pancreatic cancer (relative ratio: 0.63, 95% CI: 0.46–0.86; P = 0.003) (11). Therefore, the results from our study are consistent with the findings of these meta-analyses.

By contrast, studies which were deemed incompatible and therefore not included in these meta-analyses indicated that metformin use was not associated with a reduced risk of developing GI cancer. A population-based cohort study in the Netherlands revealed that metformin use was not associated with a decreased risk of stomach, liver, pancreatic, and colorectal cancers (HR: 0.97, 95% CI: 0.82–1.15) (14) and also failed to detect changes in the risk of cancer according to an increase in the cumulative dose of metformin. This study included 35,654 subjects (62.4%) with at least 1 year of antidiabetic medication-free follow-up and 21,460 subjects (37.6%) with prevalent use of antidiabetic medications. By contrast, this study included only antidiabetes drug-naïve subjects to allow us to clearly determine their effect. The Dutch study was also based on a smaller subgroup of metformin users (37,125 metformin users vs 19,899 metformin nonusers), whereas our study, with 263,754 subjects, compared a larger number of metformin users and nonusers. Another nested case-control study from the United Kingdom revealed that based on conditional logistic regression analyses, extensive use (≥50 prescriptions) of metformin increased the risk of colorectal cancer (odds ratio: 1.43, 95% CI: 1.08–1.90; P < 0.05) and was actually associated with an increased risk among men (15). In that study, 920 subjects in the case group were patients who had an incident diagnosis of colorectal cancer and had a history of type 2 diabetes before the cancer diagnosis. By contrast, this study evaluated the incidence of cancer during a 5-year period in a matched sample of 263,754 patients with newly diagnosed DM. Therefore, we conclude that these methodological differences are likely the source of the disagreement between our findings and those of the studies described above.

This study revealed that a clear association between metformin use and the pancreatic cancer development among patients with newly diagnosed type 2 DM was lacking. However, metformin use was associated with a reduced risk of pancreatic cancer in the subgroups with the highest cumulative dose and the longest exposure duration. Early detection of pancreatic cancer is difficult, and it is more likely to be diagnosed at an advanced stage (vs stomach, colon, and liver cancers) because most patients do not experience symptoms until they have advanced pancreatic cancer (24). In addition, the association between DM and pancreatic cancer is peculiar. Long-term diabetes can be a risk factor for pancreatic cancer (25), and patients with pancreatic cancer can rapidly develop diabetes (26). The sequential association between these is not clear, and they have become risk factors for each other. The effects of metformin use might not be reflected among patients with newly diagnosed type 2 diabetes who already have pancreatic cancer. The complexity of this association may explain the varying findings on the association between metformin use and pancreatic cancer. Furthermore, Lee et al. (27) reported that dipeptidyl peptidase-4 inhibitors increased the risk of pancreatic cancer among patients with newly diagnosed type 2 diabetes. However, there was no increasing trend according to duration of exposure. We propose that the reason for the increased risk of pancreatic cancer that was observed in our study could be the peculiar concurrence of diabetes and pancreatic cancer, rather than the influence of antidiabetes drugs.

This study evaluated the association between metformin use and GI cancer in an Asian population because previous studies have generally focused on European or American populations. Asian populations have relatively high incidences of stomach, colon, liver, and pancreatic cancers, with Eastern Asia having the highest global incidence of stomach and liver cancers (2). Regional differences in the prevalence of Helicobacter pylori infection and dietary components may influence the high incidence of stomach cancer in Eastern Asia (2), whereas the high prevalence of chronic hepatitis B and C virus infections may influence the high incidence of liver cancer (2). This study evaluated an Asian population with newly diagnosed type 2 diabetes to clarify the association between metformin use and GI cancers. This study also analyzed subjects with newly diagnosed, drug-naïve type 2 diabetes. Previous studies that analyzed subjects who were already diagnosed with diabetes or took antidiabetes medications were somewhat limited in their ability to evaluate the effect of metformin. Therefore, our results better reflect the effect of metformin on GI cancer.

This study has some limitations. First, we retrospectively evaluated claims data from the NHIS database, which does not include laboratory test results. Thus, it was impossible to consider the magnitude of DM control because we did not have available data on hemoglobin A1c levels. Second, the claims do not contain important GI cancer-related information, such as family medical history, obesity, drinking, and smoking (27–30), which may have introduced some bias into our findings. Third, this study had a relatively short follow-up duration because it takes an average of 10–20 years for normal colonic crypts to develop into overt adenocarcinoma (31). Thus, a study with a longer follow-up period might be needed to more accurately define the association between metformin use and GI cancer development. Fourth, this study did not consider the duration of use for other antidiabetic medications, which were only considered categorical variables (any use or nonuse) for the adjustment of our analyses.

In conclusion, our findings suggest that metformin use could be associated with a reduced risks of stomach, colon, and liver cancers and a reduced risk of pancreatic cancer in some subgroups. Therefore, metformin has benefit as a first-line treatment for type 2 DM, based on its anticancer and glucose-lowering effects. A further role in cancer risk reduction could be studied in controlled trials.

CONFLICTS OF INTEREST

Guarantor of the article: Sun Ok Song, MD, PhD.

Specific author contributions: All authors contributed to the study design and were involved in all stages of manuscript development. J.H.Y. analyzed and interpreted data and drafted the manuscript. M.J.K. statistically analyzed and interpreted data. Y.Y.C., S.W.K., S.H.S., S.L., and Y.L. contributed to discussion. B.L. conceptualized and designed the study and contributed to discussion. S.O.S. conceptualized and designed the study, acquired data, analyzed and interpreted data, critically revised the manuscript for important intellectual content, and supervised the study.

Financial support: This work was supported by the National Health Insurance Ilsan Hospital grant (2016-20-011).

Potential competing interests: None to report.

Study Highlights

WHAT IS KNOWN

• ✓ Metformin has had anticancer effects in preclinical testing.

• ✓ Clinical studies have failed to confirm this effect in GI cancer.

WHAT IS NEW HERE

• ✓ Metformin use was associated with reduced stomach, colon, and liver cancer risks among newly diagnosed type 2 diabetes patients.

• ✓ Metformin use at the highest cumulative dose (≥1,200,000 mg) and the longest duration (≥2,000 days) reduced pancreatic cancer risk among newly diagnosed type 2 diabetes patients.

TRANSLATIONAL IMPACT

• ✓ Metformin might provide added values that reduced risks of GI cancers for managing newly diagnosed type 2 diabetes.

Author Notes

Correspondence: Sun Ok Song, MD, PhD. E-mail: [email protected].

SUPPLEMENTARY MATERIAL accompanies this paper at https://links.lww.com/CTG/A410