Process evaluation of a multicentre randomised clinical trial of substituting surgical excisions of low-risk basal cell carcinomas from secondary to primary care

Abstract

Objectives

In 2016, the SKINCATCH Trial, a clustered multi-centre randomised trial, was initiated to assess whether low-risk basal cell carcinomas (BCCs) can be treated by general practitioners (GPs) without loss of quality of care. The trial intervention consisted of a tailored 2-day educational course on skin cancer management. The aim of this process evaluation was to investigate GPs’ exposure to the intervention, implementation of the intervention and experiences with the intervention and trial.

Research design and methods

Data on exposure to the intervention, implementation and experiences were obtained at several points during the trial. Complementary quantitative components (ie, surveys, database analysis, medical record analysis) and qualitative components (ie, interviews and focus groups) were used. Quantitative data were analysed using descriptive statistics; qualitative data were summarised (barrier interviews) or audiorecorded, transcribed verbatim and thematically analysed using Atlas.Ti (focus groups).

Results

Following a 100% intervention exposure, results concerning the implementation of the trial showed that aside from the low inclusion rate of patients with low-risk BCCs (n=54), even less excisions of low-risk BCCs were performed (n=40). Although the intervention was experienced as highly positive, several barriers were mentioned regarding the trial including administrative challenges, lack of time and high workload of GPs, low volume of BCC patients and patients declining to participate or requesting a referral to a dermatologist.

Conclusions

Although GPs’ participation in the highly valued training was optimal, several barriers may have contributed to the low inclusion and excision rate of low-risk BCCs. While some of the issues were trial-related, other barriers such as low patient-volume and patients requesting referrals are applicable outside the trial setting as well. This may question the feasibility of substitution of surgical excisions of low-risks BCCs from secondary to primary care in the current Dutch setting.

Trial registration number

Trial NL5631 (NTR5746).

Correspondence to Dr Renate R van den Bos; [email protected]

Strengths and limitations of this study

• A strength is that this process evaluation uses complementary descriptive quantitative measures as well as qualitative measures at different time points during the course of the trial.

• It provides essential in-depth insight into general practitioners’ exposure to the intervention, implementation of the intervention, and their experiences with the intervention and trial.

• A limitation of our study is the late conduction of a barrier analysis instead of addressing identified barriers prior to the onset of the trial.

Background

Healthcare is becoming increasingly expensive with rising percentages of the gross domestic product spent on healthcare.^1–3 Since research has shown health systems with stronger primary care tend to have lower healthcare costs, initiatives such as substitution of hospital care towards primary care are increasingly developed and experimented with worldwide.^4–13 The main goal of these initiatives is to maintain the affordability, and thus sustainability, of healthcare. Furthermore, it is a means to provide more easily accessible care closer to the patients’ home. However, not every type of care may be suitable for substitution towards primary care. Whether a particular type of care is deemed appropriate for substitution depends on various disease and care specific factors, such as high-volume and being low-complex care, and the support of different stakeholders including general practitioners (GPs), medical specialists and patients.⁵

One type of care that has been conceived as a potential candidate for substitution of hospital care towards primary care is low-risk skin cancer care.^{5 14} In the Netherlands, as in several other countries such as the UK and Australia, GPs have a gatekeeper function.^{5 15 16} Consultations are mainly patient driven, and GPs, who until recently did not have a related primary care guideline, determine whether patients need access to secondary and tertiary healthcare.¹⁷ A substantial proportion of patients with a basal cell carcinomas (BCC) (60% in a comprehensive Dutch primary care database analysis) are referred to the dermatologist.^18–21 The idea of substituting low-risk skin cancer care to GPs is reflected in the recently published guideline ‘suspicious cutaneous lesions’ of the Dutch College of General Practitioners, which includes recommendations for GPs on the diagnosis and treatment of low-risk BCCs.¹⁷ Particularly, low-risk BCCs (ie, non-aggressive histological subtypes, low-risk locations and size <2 cm) are relatively easy to diagnose and treat. Minor surgery can be performed in primary care offices, and innovations such as teledermatology can support GPs.^{22 23}

In 2016, the SKINCATCH Trial (SKIN Cancer And Tumour Healthcare) was initiated to assess whether low-risk BCCs can be treated by GPs without loss of quality of care. The study design was a multicentre cluster randomised non-inferiority trial, in which the intervention included a tailored 2-day educational course on skin cancer management. Participating GPs showed great enthusiasm and interest at the start of the trial,¹⁴ and although the patient inclusion rate of all skin tumours suspicious for skin cancer was consistent with the researchers’ expectations, the inclusion rate of low-risk BCCs (primary outcome) lagged far behind.

Therefore, a process evaluation was conducted alongside the trial. A process evaluation is crucial for providing insight in to what extent the trial intervention was actually implemented, how it was experienced by study participants and whether the intervention is feasible in daily practice.^{24 25} The results can be used to guide the implementation of similar care substitution initiatives.²⁴ The aim of our process evaluation was, therefore, to assess GPs’ exposure to the intervention, implementation of the intervention and experiences with the intervention and trial.

Methods

Description of SKINCATCH trial

The SKINCATCH Trial (see figure 1) was initiated based on the hypothesis that conventional excision of low-risk BCC could be performed by GPs in a primary care setting while maintaining the same quality of care. The study design was a multicentre cluster randomised non-inferiority trial, with GP practices (including group practices) being included as clusters. These clusters were randomised into two parallel arms: the intervention group, which was trained before starting the trial, and the care-as-usual group. Main outcomes included the histological completeness rate of low-risk BCC excisions by GPs in the intervention group compared with dermatologist (primary outcome), diagnostic accuracy of GPs regarding skin tumours, cost-effectiveness of the intervention and treatment and patient reported outcomes regarding preferences and cosmetics (secondary outcomes) (see table 1).

Figure 1. Overview of SKINCATCH trial design. BCC, basal cell carcinoma; CEA, cost-effectiveness analysis; GP, general practitioner; PROMs, patient-reported outcome measures.

Table 1

Interventions, recommendations and outcome measures of the SKINCATCH trial

Main components of interventions for intervention group	A tailored 2-day educational course regarding the diagnosis and management of skin cancer with a focus on BCCs including hands-on surgical training (cadaveric workshops)
	An interactive 20 min e-learning for GPs, which was available at all times during the trial
Main recommendations for low-risk BCC care to be performed by GPs in intervention group	When a skin tumour is suspicious for a malignancy, a biopsy should be performed
	If the histopathological examination confirms a low-risk BCC, the GP should perform the excision with adequate margins
	If the histopathological examination shows a high-risk BCC or other type of skin cancer, the GP should refer the patient to the dermatologist
Main outcome measures	Histopathological completeness rate of low-risk BCC excisions by GPs in the intervention group compared with dermatologists
	Diagnostic accuracy of skin tumours
	Patient reported outcome measures concerning preferences on treating physician and cosmetic results of the received treatment
	Cost-effectiveness analysis

BCC, basal cell carcinoma; GP, general practitioner.

The GPs in the intervention group were offered an extensive training in BCC (and skin tumour) management consisting of a tailored 2-day educational course including hands-on surgical training in cadaveric workshops. The GPs in the care-as-usual group did not receive the 2-day educational intervention and were asked to provide skin cancer care the way they were used to. As compensation, they were offered the same BCC management training after completion of the trial. Eligible patients (ie, all patients with a skin tumour suspicious for malignancy) were to be included in the trial during the period February 2016 to May 2018. The first patient was enrolled on Feb 23 2016. Included patients were asked to complete questionnaires at start of their treatment, and 3 and 6 months post-treatment.

The power analysis for the primary outcome was based on a t-test of the proportion of histological completeness of the physicians (GPs and dermatologists), where the physician is the unit of analysis. We expected five eligible patients in the non-inferiority part of the trial per GP per year, which was based on national incidence rates and a prior GP survey.^{26 27} Using a non-inferiority margin of 5% (based on a clinically accepted margin) and a one-sided significance level of 2.5%,²⁸ a sample size of 45 GPs per group (90 GPs total) was required to obtain a power of 80%. This sample size was increased to 129 GPs to account for (1) the possibility of drop-outs of GPs, and (2) the effect of within-practice correlations of the GPs.

A total of 600 patients with a suspicious skin tumour were included in the trial; 316 patients were included by the GPs in the intervention group and contained 54 patients with a low-risk BCC (9% of the needed sample size for sufficient statistical power (n=600)). As recruitment of removed BCCs was so low, we are unable to report on the primary outcome of the trial (histological completeness rate of low-risk BCC excisions by GPs in the intervention group compared with dermatologists). The process evaluation presented in this paper was based on this low inclusion rate of low-risk BCCs.

Consent and permissions

As this process evaluation is an evaluation among trial participants, conducted as integral part of the trial, we did not obtain separate ethical approval, except for the focus groups. The Standards for Reporting Qualitative Research (SRQR) guidelines were applied, as far as applicable. These guidelines provide a tool for the transparent reporting of qualitative studies.²⁹

Design process evaluation

In designing this process evaluation, we used the framework of Hulscher et al²⁴ to gain insight into the processes responsible for the (variation in) results in the target group. Data on exposure to the intervention, implementation of the intervention and experiences with the intervention and trial were obtained. We used both quantitative and qualitative components, which are described in detail below.

Data collection, outcome measures and analyses

Surveys

Two types of surveys were conducted among participating GPs during the course of the trial to assess their exposure to the intervention and their experiences with the intervention and trial: a training evaluation survey and an online trial evaluation survey. Participation in each of the surveys was voluntary.

Training evaluation survey: After completing the prestudy training, all GPs were asked to complete a survey to evaluate the training. With this survey, both their exposure to and experiences with the training were assessed. The survey consisted of eight statements (seven statements on the content of the training, and one statement on the organisation of the training) using a five-point Likert-scale ranging from strongly disagree to strongly agree (online supplemental appendix A).

Trial evaluation survey: Ten months after the start of the trial, an online survey was sent to all participating GPs to further explore their experiences with the trial. The survey consisted of four multiple-choice questions, focusing on experiences with the trial and assessing the perceived barriers (online supplemental appendix B).

Training and trial evaluation surveys were analysed separately using SPSS V.24.0 statistical software.

Database analysis

To gain insight into the implementation of the intervention and more specifically the low inclusion rate of BCC patients, a database analysis at the end of the inclusion period was performed investigating the number of inclusions for the primary outcome measure of the trial (ie, histological completeness of low-risk BCC excisions) based on the paper or digital case report forms (CRF) (ie, OpenClinica).³⁰ The CRF included (among others) information on tumour characteristics (eg, size and location), the histopathological diagnosis of the skin tumour and whether or not the GP performed a surgical excision. The CRFs in OpenClinica were exported to and analysed with SPSS V.24.0 statistical software. Descriptive statistics were used to assess the number of performed low-risk BCC surgical excisions as compared with the number of included low-risk BCCs.

Medical record analysis

A medical record analysis was performed to further explore the implementation of the intervention by obtaining quantitative information regarding the number of potential eligible patients and potential eligible excisions. This analysis was performed among seven randomly selected GPs in two primary care practices, participating in the intervention group of the trial. All GP records from February 2016 to February 2017 were screened for eligible patients by a GP practice healthcare assistant using International Classification of Primary Care (ICPC) codes for skin tumours (online supplemental appendix C). Information was obtained on number of patients, clinical diagnosis of the GP, size of the tumour, localisation of the tumour and choice of treatment. In case of histopathological examination additional information was obtained on histopathological diagnosis from the biopsy and/or excision, and histological completeness in case of surgical excision. If the patient was referred to secondary care information was obtained on clinical or histopathological diagnosis. Descriptive statistics were used to assess the GPs’ management of eligible patients.

Telephonic ‘barrier’ interview

Six months after the initiation of the trial, telephonic interviews were conducted by one of the researchers (EN) to identify GPs’ experiences with the trial in terms of perceived barriers regarding the inclusion of patients. We invited GPs from both arms either with no inclusions or one or more inclusions to participate. After 12 interviews with GPs in the intervention group and 10 GPs in the care-as-usual group no new barriers emerged. The semistructured interviews were conducted between August and November 2016. The datawere analysed by the researcher conducting the telephonic interview (EN), noting reported elements during the interview and descriptively summarising the main barriers afterwards.

Focus groups

Three focus groups were conducted between December 2017 and March 2018 to gain an in-depth understanding of GPs’ experiences with the intervention and the trial. Focus groups were chosen as these facilitate interaction between participants, enabling us to identify the GPs’ views on substitution of care, and their experiences with the trial.^31–33 All GPs participating in the trial were invited by email, containing an information leaflet about the qualitative evaluation study. GPs could register for one of the three organised focus groups by contacting one of the researchers.

The sessions were moderated by an experienced independent qualitative researcher (ML) and an assistant, both not being involved in the trial. One of the SKINCATCH Trial researchers (EN) was present during the focus groups, but only to answer substantive questions regarding the trial.

In each focus group, the discussion was semistructured using a predefined topic list consisting of two separate parts: general views on substitution of care (part 1) and GPs’ experiences with the trial (part 2). The current study focuses on the latter part (online supplemental appendix D). Results on their general views on substitution of care have been described elsewhere.¹⁴

All focus groups were audiorecorded with consent of participants. Subsequently, the audio tapes were transcribed verbatim and imported to Atlas.ti (V.8 for Windows) for analysis.

Two researchers (EN and ML) independently openly coded the first transcript after which the obtained codes were discussed and a preliminary coding scheme was developed. Next, all transcripts were coded by one researcher (EN or ML) and subsequently checked by a second researcher (EN or ML). Differences were discussed and refined until agreement was reached, and new codes were added when needed. The initial coding phase was followed by the phase of constant comparison.³¹ Different codes were compared and the relationship between codes were explored to detect emerging themes.

Results

Participants

A total of 128 GPs from 90 different primary care practices were included for randomisation (table 2). One GP in the intervention group, and 22 GPs in the care-as-usual group dropped out. Most drop outs occurred within 3 months after the start of the trial. Reported reasons mostly concerned lack of time and personal illness. All 128 GPs were included for the database analysis, and a subgroup of 7 GPs (12%) of the intervention group were included for the medical record analysis. See table 2 for more information on the participants of the different quantitative and qualitative components. For further details regarding the focus groups, see online supplemental table S1.

Table 2

Participants (GPs) of the SKINCATCH trial and each of the components of the process evaluation

SKINCATCH trial	Intervention group (n=58)	Care as usual group (n=70)
Male, n (%)	32 (54)	33 (47)
Drop outs, n (%)	1 (2)	22 (31)
Quantitative components, n (%)
Database analysis	58 (100)	70 (100)
Medical record analysis	7 (12)	N/A
Training evaluation survey	57 (98)	N/A
Trial evaluation survey	24 (41)	36 (51)
Qualitative components, n (%)
Telephonic ‘barrier’ interview	12 (21)	10 (14)
Focus groups	9 (16)	8 (11)
Focus group 1 (n=8)	4 (50)	4 (50)
Focus group 2 (n=5)	2 (40)	3 (60)
Focus group 3 (n=4)	3 (75)	1 (25)

GP, general practitioner; N/A, not available.

Exposure to the intervention

All GPs in the intervention group (n=58) completed the extensive 2-day training programme. Regarding the e-learning, it was not possible to measure the exposure quantitatively; it could be openly accessed by GPs at all times. The focus groups suggested that a wide variation existed regarding the exposure to the e-learning. Whereas some GPs stated to have gone through the files, others reported not remembering it have been offered or not to have opened it due to time restrictions.

Implementation of the intervention

Only 54 patients with low-risk BCC (9% of needed sample size) of the total of 600 patients with suspicious skin tumours were included in the trial. Furthermore, the GPs in the intervention group performed 95 surgical excisions of skin tumours in total, of which 40 concerned a low-risk BCC. In the care as usual group 29 of the 284 included patients concerned patients with histopathological confirmed low-risk BCCs.

The medical record analysis of potentially eligible BCCs patients in 1 year among 7 GPs resulted in 448 potential patients. After manual extraction by two of the authors (EN, KHRR), 35 confirmed BCC patients remained of which 16 were low-risk BCC. Three BCCs (19%) were excised by two of the seven GPs; the remaining 13 tumours were not excised by the GP. Reported reasons in the medical records were: preference for topical treatment (n=2), patient preference for dermatologist (n=1), referral due to melanoma in differential diagnosis (n=1), coinciding melanoma (n=1), not reported in medical record (n=8).

Experiences with the intervention and trial

Experiences with the intervention

Training evaluation survey: The training was generally evaluated positively by the GPs (figure 2); almost all (n=54) indicated to have found the training useful and almost all (n=53) indicated they would recommend the training among colleagues. All GPs (strongly) agreed with the statement the training would change the way they manage skin cancer, and the vast majority (n=47) confirmed that it was clear to them what was expected regarding their participation in the trial. For further details on the training evaluation survey see online supplemental figure 1.

Figure 2. Results from the training evaluation survey.

Focus groups: The focus groups confirmed that the GPs were highly positive about the training. Some reported it to be the best training they have ever had. According to the GPs it offered them guidance in managing skin tumours in general, and it was particularly useful to learn techniques for minor surgery hands-on. GPs indicated to feel more empowered to extend their services regarding skin tumour management in daily practice. However, some GPs did mention that with time passing they returned to old patterns. According to the GPs, the training may not have been enough for all GPs to change their role in the management of skin tumours. Furthermore, according to some GPs the participation in the trial caused them to diminish their role in skin cancer management as they were used to performing minor surgery on high(-er) risk skin cancers (eg, BCCs located in the face), which was restricted by the study protocol. Regarding the e-learning, the few GPs who used the e-learning were generally positive and reported it was fun to do.

Experiences with the trial

Trial evaluation survey: Reported reasons for the low number of included (BCC) patients in the trial concerned lack of time (n=34) and realising the patients’ eligibility afterwards (n=27), patients rejected participation (n=11), not understanding the different study forms (n=5), the trial restricts me on performing excisions due to trial recommendations (n=3), the GP being afraid to perform minor surgery (n=1) and having to treat the patient different from what they were used to (n=1). A smaller group of GPs (n=13) agreed with the statement that it would make it easier for them to only include patients with a low-risk BCC rather than all skin cancers, and the largest part (n=44) disagreed with the option of clustering consultation hours for skin cancer patients for GPs individually to make patient recruitment more easy.

Telephonic ‘barrier’ interview: During the telephonic interview six barriers were identified. Main perceived barriers reported by the GPs concerned ambiguity regarding eligibility criteria of patients, and lack of clarity regarding the trials’ CRFs. GPs indicated that they expected one of the researchers to visit their practices for one-on-one explanation on the forms. Further perceived barriers included the trial not being a priority, the inclusion process being too time-consuming, difficulty retaining information over time, and discouragement due to refusal of patients or skin tumours appearing high risk.

Focus groups: GPs’ experiences regarding the trial varied. Whereas some GPs were positive about the trial and managed to include patients (up to 53), others reported rather negative experiences. Several barriers were identified which may have contributed to the relatively low inclusion rate (both in general as well as concerning low-risk BCCs). First, administrative challenges related to the inclusion of patients to the trial were reported as a barrier. According to the GPs, the inclusion procedure (informed consent procedure and CRF) was difficult to integrate in daily practice with several study forms needed to be completed at different times during the treatment course of the patient. GPs reported this to be difficult and too time-consuming. However, GPs lacked suggestions on how to improve these administrative challenges as they know it is crucial for data collection. Some GPs reported to have experienced the start of the trial as rather confusing; they stated study forms were not immediately present, and that both the start-date for inclusion as well as the eligibility criteria were not clear. Others were more positive and reported to have found a way of structuring it for themselves, and commented that inaccuracies were picked up well by the researchers. The online CRF application (ie, OpenClinica) was variably received by the GPs, though it was specifically designed for the trial in an attempt to facilitate the GPs in data registration. Some GPs reported it to be not user-friendly and continued using the paper forms, while others stated it to be of great help. Suggestions on reducing the administrative challenges included having researchers collect the data themselves by visiting the GPs’ practices and using an automated digital data collection programme.

Another reported barrier related to the administrative barrier, was a perceived lack of time and high workload to include patients. According to the GPs, this was related to cramped consultation hours, being behind schedule and patients presenting multiple problems during consultation with their GP in which the skin tumour was not perceived as the main issue. As a result of the lack of time and high workload, GPs were more hesitant to recruit patients as this would consume additional time.

A third barrier as reported by the GPs was the low volume of eligible patients seen in practice. GPs reported to only see a small number of low-risk BCC annually. Some also stated to have seen less BCC patients during the course of the trial than anticipated, for reasons not clear.

A fourth barrier reported were patients declining or refusing to participate in the trial. According to the GPs, some patients did not want to participate due to the difficulty and large amount of information they had to read on participation request, and things needed from them after inclusion (ie, questionnaires). The GPs further mentioned that especially older patients and patients less intelligent often declined to participate.

In addition to the low inclusion rate, the GPs were also asked for possible explanations for the low rate of excisions performed by GPs during the trial. Whereas some GPs indeed reported to have only performed few excisions, others were rather surprised hearing this as it did not align with their own experiences. Reported reasons for the low number of excisions were the low number of BCC patients seen in daily practice, patients requesting a referral to the dermatologist, a lack of time and high workload, having a colleague who performs all the excisions, and the training course not being sufficient to change GPs’ behaviour, particularly considering the reported already high workload.

Discussion

This evaluation study showed that, although GPs initially showed great enthusiasm towards the concept of substitution,¹⁴ and all GPs participated in the highly valued training, several barriers may have contributed to the low inclusion and excision rate of low-risk BCC patients. Some of these barriers seem to be attributable to the trial setting (eg, administrative challenges, patient recruitment issues), complicating its implementation in daily practice. However, other reported barriers such as high workload, low volume of low-risk BCC patients and patients requesting a referral, apply outside the trial setting as well.

Although several trial-related barriers, such as clear study forms and inclusion criteria, should have been adequately addressed in the current trial, other practical issues such as patient recruitment challenges are commonly reported problems within (multicentre) randomised controlled trials and are difficult to prevent completely.^34–38 Similarly, the reported barrier of lack of time/high workload of GPs seems to be inherently related to GP practices,^38–40 and may have further impeded study implementation. To tackle these barriers, targeted interventions to enhance recruitments skills of GPs may be valuable to optimise the feasibility of trial interventions in clinical medical care.³⁸

In addition to the trial-related barriers, other reported barriers also apply outside the trial setting and concern the topic of substituting low-risk BCC care towards primary care. Despite high and rising incidence rates of BCCs reported in the literature,^{27 41} we found that only a small proportion of BCCs can be considered ‘low-risk’ when taking into account body site, diameter and histological subtype,^41–43 which was recently confirmed by Fremlin et al.⁴² Aside from the low volume, the number of excisions performed by GPs in the intervention group was even lower. According to the GPs this may have been partly related to the training being insufficient to change GPs’ practices. Also, GPs were less inclined to perform a surgical excision when patients requested a referral to a dermatologist, which has been found in previous studies as well.^{14 15 44–48} These barriers, related to feasibility, need to be addressed, where possible, before assessing whether low-risk BCCs can be treated by GPs without a loss of quality of care.

Indeed, with the patient volume being this low (based on the medical record analysis approximately 2 patients with low-risk BCC per GP per year), it will be challenging, if not impossible, for GPs to obtain and maintain their competencies in low-risk BCC management.^{14 42} Particularly in the context of this low patient volume, a 1-day training may not be sufficient to acquire the relevant competencies. Offering adequate training in a repetitive setting tailored to the specific needs of each GP may, therefore, contribute to a better integration of what is learnt into daily practice.^{49 50} Although this was attempted by offering an e-learning module, the uptake (although variable) seemed to be only minimal. Furthermore, the cost-effectiveness of such interventions may be questioned. Other solutions may focus on organisational changes in primary care such as concentrated substitution.¹⁴ Within this concept GPs refer patients to a colleague GP with noted interest, experience and competence in skin cancer care, thereby clustering these patients within or between practices.¹⁴

A limitation of our study includes the late conduction of a barrier analysis. Implementation of change is a complex process, and a preceding barrier analysis among all involved stakeholder groups is advocated to increase the success of interventions.⁵¹ By addressing identified barriers prior to the onset of this trial, failure may have been prevented. In addition, such input can serve to promote awareness and stimulate involvement among the target groups, incentivising more successful adoption at a later stage.⁵² However, it is also important to elicit views of stakeholders who already have some experience with the intervention at hand, as this often elicits different types of barriers.¹⁴ Performing a barrier analysis both before the onset of the trial as well as during the trial as part of a process evaluation is therefore advised.

A strength of this study is that we used several complementary evaluation methods, combining both quantitative and qualitative data at different time points during the course of the trial, focusing on both the intervention and care-as-usual group. Although only a low number of GPs was included in the medical record analysis and data on the use of the e-learning module was lacking, by using triangulation of data we were able to capture different dimensions of the observed phenomena. As such, our process evaluation provides essential in-depth insight into the trial and the observed outcomes.

Conclusions

This process evaluation has identified some trial-related as well as more general topic-related barriers that may be responsible for the low inclusion and excision rate of low-risk BCC patients by GPs within the trial. Based on the results of this study, without being able to measure the surgical effectiveness of GPs, the feasibility of substituting low-risk BCC care from secondary to primary care in the current setting should be questioned. Future trials on care substitution may benefit from thorough qualitative barrier analyses among all involved stakeholders, before onset as well as during the course of the trial, to increase the likelihood of successful implementation.

The authors would like to thank the patients and GPs who participated in the study. We thank Mirjam Droger and Sven van Egmond for their assistance in conducting the focus groups, and primary care practice assistants Claudia Baetsleer (GP practice Pallion) and Shirley Ipskamp (GP practice Doktershuis) for their indispensable and proficient help in conducting the Medical Record Analysis.

Data availability statement

Data are available on reasonable request. All data relevant to the study are included in the article or uploaded as online supplemental information. The data that supports the findings of this study are available from the medical ethics committee of the Erasmus University Medical Center in Rotterdam (Contact: Interview study reference number MEC-2016-204 and Focus group study reference number MEC-2015-492), but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of the medical ethics committee of the Erasmus University Medical Center in Rotterdam.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

This study involves human participants and was approved by medical ethics committee of the Erasmus University Medical Center in Rotterdam (MEC-2016-204 and Focus group study reference number MEC-2015-492). Participants gave informed consent to participate in the study before taking part.

Footnote

Contributors EN, ML, MW, KHRR, RRvdB, PJEB and TN involved in the concept and design of the quantitative and qualitative study. EN and ML performed the focus groups. EN, ML and KHRR involved in quantitative and qualitative data acquisition. EN, ML, MW, RRvdB, PJEB and TN involved in data analysis and interpretation. The paper was written by EN and ML, and was critically revised by all authors. All authors read and approved the final manuscript. RRvdB is the responsible author acting as guarantor.

Funding Funding for this research was provided by the Foundation Achmea Healthcare (in Dutch: Stichting Achmea Gezondheidszorg) which had no role in the design, conduct, analysis or interpretation of the study. Award/grant number is not applicable.

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

References

¹ Alberto M, David M, Luca L. Future trends in health care expenditure 2017.

² OECD. Better ways to pay for health care 2016.

³ Mackenbach JP. [What are the health benefits of the most expensive healthcare system in Europe?]. Ned Tijdschr Geneeskd 2015; 159: A9433. http://www.ncbi.nlm.nih.gov/pubmed/26306488

⁴ de Jong J, Korevaar J MK. Substitution potential between primary and secondary health care, 2016. Available: www.nivel.nl

⁵ Van Dijk C, Korevaar J, De Jong J. Room for substitution? shift from secondary to primary care. Nivel 2013; 109.

⁶ van Hoof SJM, Kroese MEAL, Spreeuwenberg MD, et al. Substitution of hospital care with primary care: defining the conditions of primary care plus. Int J Integr Care 2016; 16: 12. doi:10.5334/ijic.2446 http://www.ncbi.nlm.nih.gov/pubmed/27616956

⁷ van Hoof SJM, Spreeuwenberg MD, Kroese MEAL, et al. Substitution of outpatient care with primary care: a feasibility study on the experiences among general practitioners, medical specialists and patients. BMC Fam Pract 2016; 17: 108. doi:10.1186/s12875-016-0498-8 http://www.ncbi.nlm.nih.gov/pubmed/27506455

⁸ Fortney JC, Steffick DE, Burgess JF, et al. Are primary care services a substitute or complement for specialty and inpatient services? Health Serv Res 2005; 40: 1422–42. doi:10.1111/j.1475-6773.2005.00424.x http://www.ncbi.nlm.nih.gov/pubmed/16174141

⁹ Laurant M, Reeves D, Hermens R. Substitution of doctors by nurses in primary care. Cochrane database Syst rev 2005; 2: CD001271.

¹⁰ Martínez-González NA, Djalali S, Tandjung R, et al. Substitution of physicians by nurses in primary care: a systematic review and meta-analysis. BMC Health Serv Res 2014; 14: 214. doi:10.1186/1472-6963-14-214 http://www.ncbi.nlm.nih.gov/pubmed/24884763

¹¹ Weller DP. Workforce substitution and primary care. Med J Aust 2006; 185: 8–9. doi:10.5694/j.1326-5377.2006.tb00439.x http://www.ncbi.nlm.nih.gov/pubmed/16813538

¹² Yong C-S. Task substitution: the view of the Australian medical association. Med J Aust 2006; 185: 27–8. doi:10.5694/j.1326-5377.2006.tb00446.x http://www.ncbi.nlm.nih.gov/pubmed/16813545

¹³ Starfield B, Shi L. Policy relevant determinants of health: an international perspective. Health Policy 2002; 60: 201–18. doi:10.1016/s0168-8510(01)00208-1 http://www.ncbi.nlm.nih.gov/pubmed/11965331

¹⁴ Noels EC, Wakkee M, van den Bos RR, et al. Substitution of low-risk skin cancer hospital care towards primary care: a qualitative study on views of general practitioners and dermatologists. PLoS One 2019; 14: e0213595. doi:10.1371/journal.pone.0213595 http://www.ncbi.nlm.nih.gov/pubmed/30889211

¹⁵ Greenfield G, Foley K, Majeed A. Rethinking primary care's gatekeeper role. BMJ 2016; 354: i4803. doi:10.1136/bmj.i4803 http://www.ncbi.nlm.nih.gov/pubmed/27662893

¹⁶ Moreno G, Tran H, Chia ALK, et al. Prospective study to assess general practitioners’ dermatological diagnostic skills in a referral setting. Australas J Dermatol 2007; 48: 77–82. doi:10.1111/j.1440-0960.2007.00340.x

¹⁷ Baaten GGG, Buis PAJ, Damen Z. NHG-Standaard Verdachte Huidafwijkingen, 2017. Available: https://www.nhg.org/standaarden/volledig/nhg-standaard-verdachte-huidafwijkingen

¹⁸ van Dijk CE, Verheij RA, Spreeuwenberg P, et al. Minor surgery in general practice and effects on referrals to hospital care: observational study. BMC Health Serv Res 2011; 11: 2. doi:10.1186/1472-6963-11-2 http://www.ncbi.nlm.nih.gov/pubmed/21205305

¹⁹ Lowell BA, Froelich CW, Federman DG, et al. Dermatology in primary care: prevalence and patient disposition. J Am Acad Dermatol 2001; 45: 250–5. doi:10.1067/mjd.2001.114598 http://www.ncbi.nlm.nih.gov/pubmed/11464187

²⁰ Boersma-van Dam E, Weesie Y, Hek K, et al. Zorg door de huisarts. Jaarcijfers 2017 en trendcijfers 2011-2017 2018.

²¹ Wakkee M, van Egmond S, Louwman M, et al. Opportunities for improving the efficiency of keratinocyte carcinoma care in primary and specialist care: results from population-based Dutch cohort studies. Eur J Cancer 2019; 117: 32–40. doi:10.1016/j.ejca.2019.05.010 http://www.ncbi.nlm.nih.gov/pubmed/31229947

²² Eminović N, Dijkgraaf MG, Berghout RM, et al. A cost minimisation analysis in teledermatology: model-based approach. BMC Health Serv Res 2010; 10: 251. doi:10.1186/1472-6963-10-251 http://www.ncbi.nlm.nih.gov/pubmed/20738871

²³ van der Heijden JP, de Keizer NF, Bos JD, et al. Teledermatology applied following patient selection by general practitioners in daily practice improves efficiency and quality of care at lower cost. Br J Dermatol 2011; 165: 1058–65. doi:10.1111/j.1365-2133.2011.10509.x http://www.ncbi.nlm.nih.gov/pubmed/21729026

²⁴ Hulscher MEJL, Laurant MGH, Grol RPTM. Process evaluation on quality improvement interventions. Qual Saf Health Care 2003; 12: 40–6. doi:10.1136/qhc.12.1.40 http://www.ncbi.nlm.nih.gov/pubmed/12571344

²⁵ Moore GF, Audrey S, Barker M, et al. Process evaluation of complex interventions: medical Research Council guidance. BMJ 2015; 350: h1258. doi:10.1136/bmj.h1258 http://www.ncbi.nlm.nih.gov/pubmed/25791983

²⁶ de Vries E, Nijsten T, Louwman MWJ, et al. [Skin cancer epidemic in the Netherlands]. Ned Tijdschr Geneeskd 2009; 153: A768. http://www.ncbi.nlm.nih.gov/pubmed/20025791

²⁷ Flohil SC, de Vries E, Neumann HAM, et al. Incidence, prevalence and future trends of primary basal cell carcinoma in the Netherlands. Acta Derm Venereol 2011; 91: 24–30. doi:10.2340/00015555-1009 http://www.ncbi.nlm.nih.gov/pubmed/21264452

²⁸ Head SJ, Kaul S, Bogers AJJC, et al. Non-Inferiority study design: lessons to be learned from cardiovascular trials. Eur Heart J 2012; 33: 1318–24. doi:10.1093/eurheartj/ehs099 http://www.ncbi.nlm.nih.gov/pubmed/22564354

²⁹ O'Brien BC, Harris IB, Beckman TJ, et al. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med 2014; 89: 1245–51. doi:10.1097/ACM.0000000000000388 http://www.ncbi.nlm.nih.gov/pubmed/24979285

³⁰ Cavelaars M, Rousseau J, Parlayan C, et al. OpenClinica. J Clin Bioinforma 2015; 5: S2. doi:10.1186/2043-9113-5-S1-S2

³¹ Green J, Thorogood N. Qualitative methods for health research. 3rd revised edition ed. Sage Publications Ltd, 2014: 233–54.

³² Krueger RA, Casey MA. Focus groups: a practical guide for applied research.: SAGE, 2000.

³³ Willis K, Green J, Daly J, et al. Perils and possibilities: achieving best evidence from focus groups in public health research. Aust N Z J Public Health 2009; 33: 131–6. doi:10.1111/j.1753-6405.2009.00358.x http://www.ncbi.nlm.nih.gov/pubmed/19413855

³⁴ McDonald AM, Knight RC, Campbell MK, et al. What influences recruitment to randomised controlled trials? A review of trials funded by two UK funding agencies. Trials 2006; 7: 9. doi:10.1186/1745-6215-7-9 http://www.ncbi.nlm.nih.gov/pubmed/16603070

³⁵ de Wit NJ, Quartero AO, Zuithoff AP, et al. Participation and successful patient recruitment in primary care. J Fam Pract 2001; 50: 976. http://www.ncbi.nlm.nih.gov/pubmed/11711015

³⁶ van der Gaag WH, van den Berg R, Koes BW, et al. Discontinuation of a randomised controlled trial in general practice due to unsuccessful patient recruitment. BJGP Open 2017; 1: bjgpopen17X101085. doi:10.3399/bjgpopen17X101085 http://www.ncbi.nlm.nih.gov/pubmed/30564680

³⁷ Bower P, Wilson S, Mathers N. Short report: how often do UK primary care trials face recruitment delays? Fam Pract 2007; 24: 601–3. doi:10.1093/fampra/cmm051 http://www.ncbi.nlm.nih.gov/pubmed/17872907

³⁸ Foster JM, Sawyer SM, Smith L, et al. Barriers and facilitators to patient recruitment to a cluster randomized controlled trial in primary care: lessons for future trials. BMC Med Res Methodol 2015; 15: 18. doi:10.1186/s12874-015-0012-3 http://www.ncbi.nlm.nih.gov/pubmed/25887970

³⁹ Croxson CH, Ashdown HF, Hobbs FR. Gps' perceptions of workload in England: a qualitative interview study. Br J Gen Pract 2017; 67: e138-e47. doi:10.3399/bjgp17X688849 http://www.ncbi.nlm.nih.gov/pubmed/28093422

⁴⁰ van den Hombergh P, Künzi B, Elwyn G, et al. High workload and job stress are associated with lower practice performance in general practice: an observational study in 239 general practices in the Netherlands. BMC Health Serv Res 2009; 9: 118. doi:10.1186/1472-6963-9-118 http://www.ncbi.nlm.nih.gov/pubmed/19604386

⁴¹ Verkouteren JAC, Ramdas KHR, Wakkee M, et al. Epidemiology of basal cell carcinoma: scholarly review. Br J Dermatol 2017; 177: 359–72. doi:10.1111/bjd.15321 http://www.ncbi.nlm.nih.gov/pubmed/28220485

⁴² Fremlin GA, Gomez P, Halpern J. Are there sufficient numbers of low-risk basal cell carcinomas to justify general practitioners (family physicians) carrying out basal cell carcinoma surgery? Clin Exp Dermatol 2016; 41: 138–41. doi:10.1111/ced.12718 http://www.ncbi.nlm.nih.gov/pubmed/26189609

⁴³ Lomas A, Leonardi-Bee J, Bath-Hextall F. A systematic review of worldwide incidence of nonmelanoma skin cancer. Br J Dermatol 2012; 166: 1069–80. doi:10.1111/j.1365-2133.2012.10830.x http://www.ncbi.nlm.nih.gov/pubmed/22251204

⁴⁴ Bath-Hextall F, Nalubega S, Evans C. The needs and experiences of patients with skin cancer: a qualitative systematic review with metasynthesis. Br J Dermatol 2017; 177: 666–87. doi:10.1111/bjd.15148 http://www.ncbi.nlm.nih.gov/pubmed/27775838

⁴⁵ van Egmond S, Wakkee M, Droger M, et al. Needs and preferences of patients regarding basal cell carcinoma and cutaneous squamous cell carcinoma care: a qualitative focus group study. Br J Dermatol 2019; 180: 122-129. doi:10.1111/bjd.16900 http://www.ncbi.nlm.nih.gov/pubmed/29927480

⁴⁶ Wammes JJG, Jeurissen PPT, Verhoef LM, et al. Is the role as gatekeeper still feasible? A survey among Dutch general practitioners. Fam Pract 2014; 31: 538–44. doi:10.1093/fampra/cmu046 http://www.ncbi.nlm.nih.gov/pubmed/25135953

⁴⁷ Westert GP, Jeurissen PPT, Assendelft WJJ. Why Dutch general practitioners do not put the squeeze on access to hospital care? Fam Pract 2014; 31: 499–501. doi:10.1093/fampra/cmu027 http://www.ncbi.nlm.nih.gov/pubmed/24919783

⁴⁸ Rotar AM, Van Den Berg MJ, Schäfer W, et al. Shared decision making between patient and GP about referrals from primary care: does gatekeeping make a difference? PLoS One 2018; 13: e0198729. doi:10.1371/journal.pone.0198729 http://www.ncbi.nlm.nih.gov/pubmed/29889861

⁴⁹ Bluestone J, Johnson P, Fullerton J, et al. Effective in-service training design and delivery: evidence from an integrative literature review. Hum Resour Health 2013; 11: 51. doi:10.1186/1478-4491-11-51 http://www.ncbi.nlm.nih.gov/pubmed/24083659

⁵⁰ Lugtenberg M, Pasveer D, van der Weijden T, et al. Exposure to and experiences with a computerized decision support intervention in primary care: results from a process evaluation. BMC Fam Pract 2015; 16: 141. doi:10.1186/s12875-015-0364-0 http://www.ncbi.nlm.nih.gov/pubmed/26474603

⁵¹ Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients' care. Lancet 2003; 362: 1225–30. doi:10.1016/S0140-6736(03)14546-1 http://www.ncbi.nlm.nih.gov/pubmed/14568747

⁵² Grol R, Wensing M. What drives change? barriers to and incentives for achieving evidence-based practice. Med J Aust 2004; 180: S57–60. doi:10.5694/j.1326-5377.2004.tb05948.x http://www.ncbi.nlm.nih.gov/pubmed/15012583