Evaluating clinician acceptability of the prototype CanRisk tool for predicting risk of breast and ovarian cancer: A multi-methods study

Abstract

Background

There is a growing focus on the development of multi-factorial cancer risk prediction algorithms alongside tools that operationalise them for clinical use. BOADICEA is a breast and ovarian cancer risk prediction model incorporating genetic and other risk factors. A new user-friendly Web-based tool (CanRisk.org) has been developed to apply BOADICEA. This study aimed to explore the acceptability of the prototype CanRisk tool among two healthcare professional groups to inform further development, evaluation and implementation.

Method

A multi-methods approach was used. Clinicians from primary care and specialist genetics clinics in England, France and Germany were invited to use the CanRisk prototype with two test cases (either face-to-face with a simulated patient or via a written vignette). Their views about the tool were examined via a semi-structured interview or equivalent open-ended questionnaire. Qualitative data were subjected to thematic analysis and organised around Sekhon’s Theoretical Framework of Acceptability.

Results

Seventy-five clinicians participated, 21 from primary care and 54 from specialist genetics clinics. Participants were from England (n = 37), France (n = 23) and Germany (n = 15). The prototype CanRisk tool was generally acceptable to most participants due to its intuitive design. Primary care clinicians were concerned about the amount of time needed to complete, interpret and communicate risk information. Clinicians from both settings were apprehensive about the impact of the CanRisk tool on their consultations and lack of opportunities to interpret risk scores before sharing them with their patients.

Conclusions

The findings highlight the challenges associated with developing a complex tool for use in different clinical settings; they also helped refine the tool. This prototype may not have been versatile enough for clinical use in both primary care and specialist genetics clinics where the needs of clinicians are different, emphasising the importance of understanding the clinical context when developing cancer risk assessment tools.

About the Authors:

Stephanie Archer

Roles Data curation, Formal analysis, Methodology, Project administration, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliation: Primary Care Unit, Department of Public Health and Primary Care, University of Cambridge, United Kingdom

ORCID logo http://orcid.org/0000-0003-1349-7178

Chantal Babb de Villiers

Roles Conceptualization, Investigation, Methodology, Project administration, Writing – review & editing

Affiliation: Primary Care Unit, Department of Public Health and Primary Care, University of Cambridge, United Kingdom

ORCID logo http://orcid.org/0000-0003-1334-1819

Fiona Scheibl

Roles Formal analysis

Affiliation: Primary Care Unit, Department of Public Health and Primary Care, University of Cambridge, United Kingdom

Tim Carver

Roles Software, Writing – review & editing

Affiliation: Centre for Cancer Genetic Epidemiology, Department of Public Health and Primary Care, University of Cambridge, United Kingdom

Simon Hartley

Roles Software, Writing – review & editing

Affiliation: Centre for Computational Biology, University of Birmingham, United Kingdom

ORCID logo http://orcid.org/0000-0001-7991-0581

Andrew Lee

Roles Software, Writing – review & editing

Affiliation: Centre for Cancer Genetic Epidemiology, Department of Public Health and Primary Care, University of Cambridge, United Kingdom

ORCID logo http://orcid.org/0000-0003-0677-0252

Alex P. Cunningham

Roles Software, Writing – review & editing

Affiliation: Centre for Cancer Genetic Epidemiology, Department of Public Health and Primary Care, University of Cambridge, United Kingdom

Douglas F. Easton

Roles Conceptualization, Software, Supervision, Writing – review & editing

Affiliation: Centre for Cancer Genetic Epidemiology, Department of Public Health and Primary Care, University of Cambridge, United Kingdom

Jennifer G. McIntosh

Roles Conceptualization, Methodology, Writing – review & editing

Affiliation: Centre for Cancer Research and Department of General Practice, University of Melbourne, Australia

Jon Emery

Roles Conceptualization, Methodology, Supervision, Writing – review & editing

Affiliations Primary Care Unit, Department of Public Health and Primary Care, University of Cambridge, United Kingdom, Centre for Cancer Research and Department of General Practice, University of Melbourne, Australia

Marc Tischkowitz

Roles Conceptualization, Funding acquisition, Methodology, Supervision, Writing – review & editing

Affiliation: Academic Department of Medical Genetics, University of Cambridge, United Kingdom

Antonis C. Antoniou

Roles Conceptualization, Funding acquisition, Methodology, Software, Supervision, Writing – review & editing

Affiliation: Centre for Cancer Genetic Epidemiology, Department of Public Health and Primary Care, University of Cambridge, United Kingdom

Fiona M. Walter

Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Methodology, Project administration, Supervision, Writing – original draft, Writing – review & editing

Affiliations Primary Care Unit, Department of Public Health and Primary Care, University of Cambridge, United Kingdom, Centre for Cancer Research and Department of General Practice, University of Melbourne, Australia

Introduction

In order to maximise opportunities to both prevent and detect cancer early, there is a need to identify people at higher risk, who may benefit from tailored screening and prevention [1–3]. There has been a growing focus on the development of multi-factorial cancer risk prediction algorithms, such as the Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm (BOADICEA [4–6]). The most recent version of BOADICEA combines genetic (e.g. rare genetic susceptibility variants and common genetic susceptibility variants in terms of a Polygenic Risk Score), family history, lifestyle and hormonal risk factors to calculate the future risk of developing cancer and the risk of being a carrier of a pathogenic variant [7]. The complex calculations included in BOADICEA have been validated in independent national and international prospective studies demonstrating good discrimination and accuracy [8–10].

Alongside the development of cancer risk prediction algorithms, researchers must also consider the design of graphical user-interfaces (GUIs) that enable the data entry required to populate the algorithm, the presentation of the risks calculated by the algorithm, and guidance on management of prevention strategies for that risk. These are fundamental to the successful implementation of risk prediction models into clinical practice. During the development stages of BOADICEA, a simple web-tool collected information to populate the algorithm. However, as the algorithm became more complex, there was a growing need for a more intuitive interface suitable for use in the clinical setting. As such, a new tool—CanRisk (www.canrisk.org), has recently been developed for healthcare professional use in a variety of clinical settings. Driven by the core principles of user-centred design [11], the core requirements of the CanRisk tool were initially influenced by clinicians from a wide range of clinical settings including primary, secondary and specialist care.

Acceptability is a multi-faceted construct that reflects the extent to which users consider an intervention appropriate, based on anticipated or experienced cognitive and emotional responses [12]. Whilst acceptability is a core facet of user-centred design, it is also an increasingly important component of evaluating the effectiveness of complex interventions [13, 14]. Furthermore, Sekhon et al. [12] suggest that successful implementation depends on the acceptability of the intervention to both intervention deliverers (healthcare professionals in the case of CanRisk) and receivers (patients). Assessing acceptability of any electronic risk assessment tool is particularly important during the beta stage of tool development as it enables further modifications to be made prior to evaluating the clinical utility of risk tools and how best to implement them [3]. Therefore, the aim of this study was to explore the acceptability of the prototype CanRisk tool among the healthcare professionals for whom it has been designed, in order to optimise further development, evaluation and, ultimately, implementation.

Methods

Design(s)

This study used a multi-methods approach. Participants completed a demographics questionnaire, two test cases (either face-to-face with a simulated patient or via a written vignette) and a semi-structured interview or equivalent open-ended questionnaire.

Participants

General Practitioners and Practice Nurses from primary care and Clinical Geneticists and Genetic Counsellors from specialist genetics clinics in London and the East of England were identified using an initial purposive sampling approach to account for clinical training and region, and enhanced by a snowball approach. Clinicians from specialist genetics clinics in France and Germany who were existing users of the previous simple web-based BOADICEA interface were also invited.

Ethical considerations

Following the receipt of an invitation to participate and a participant information sheet, participants gave full informed written consent. Participants were given the opportunity to withdraw up to and during study completion. As part of the study debrief, participants were reminded about the process for withdrawing their data. Ethical approval was sought and granted from the Cambridge Psychology Research Ethics Committee to cover all aspects of the study (PRE.2018.011). Additional approval for the study in England was granted by the Health Research Authority (Ref No: 18/HRA/1094); no additional approvals were required for the study in France and Germany.

CanRisk tool

The CanRisk Tool is a Web-based computer program developed by the Centre for Cancer Genetic Epidemiology at the University of Cambridge, and hosted on a University of Cambridge virtual server. The software is comprised of a number of components that form a series of graphical user interfaces (GUIs) to collect data required to run BOADICEA. This version of the CanRisk Tool was designed for use by healthcare professionals in a range of clinical settings. The tool prompts clinicians to enter the woman's risk factors (e.g. age, lifestyle, hormonal), genetic test results (if known) and family history data, and uses this information to calculate the woman's lifetime risk of breast and ovarian cancer and their risk of being a carrier of a pathogenic variant. Risk calculations are displayed in a variety of formats, including combinations of written text, graphs and icon arrays. For users in the UK, the tool also classifies the woman’s risk in accordance with the NICE guidelines [15]. No recommendations for risk management are provided. See Figs 1, 2 and 3 for screenshots of the prototype.

[Figure omitted. See PDF.]

Fig 1. CanRisk tool initial page displaying the accordion menu of clinical topic areas included in the risk prediction calculation*.

* image corresponds to the revised version of the tool (not the beta version assessed).

https://doi.org/10.1371/journal.pone.0229999.g001

[Figure omitted. See PDF.]

Fig 2. CanRisk tool detailed view of one topic area: Personal Details*.

* image corresponds to the revised version of the tool (not the beta version assessed).

https://doi.org/10.1371/journal.pone.0229999.g002

[Figure omitted. See PDF.]

Fig 3. CanRisk tool detailed view of the interactive graphical pedigree editor [16]*.

* image corresponds to the revised version of the tool (not the beta version assessed).

https://doi.org/10.1371/journal.pone.0229999.g003

Procedure

The research design aimed to standardise the content and quality of data collection across countries, so we used two methods of data collection for the study (face-to-face or on-line), determined by proximity to the participants. Participants were identified through local/national/international research networks (e.g. the Clinical Research Network in the UK) and were initially contacted by email. Participants were eligible for inclusion if they worked in a specialist genetics clinic or general practice and could give informed consent. Data were collected between April 2018 and August 2018, and the study was undertaken in participant’s clinical workplaces.

England.

Those based in England participated face-to-face with the researcher. They completed an initial questionnaire collecting demographic data (age, gender, years in role, number of hours worked, any research qualifications) and information on their current use of computerised risk assessment tools (use of: web browsers, computer in clinic, electronic risk assessment tools, electronic tools to draw family tree). Participants then completed two simulated patient encounters (SPEs) using the CanRisk tool to produce a risk calculation. The SPEs were designed to emulate a typical appointment and were conducted in the participant’s normal clinical room with their normal computer. Two female professional medical role players took the part of the patients in the SPEs. Each role player was given a script (see Table 1 for an example) and rehearsed with the research team prior to the study. To standardise the SPEs across multiple participants, the actors were provided with the opening and closing lines of the consultation, information about the patient and their family and a list of comments to include (Table 1). Following completion of the SPE, participants took part in an audio-recorded face-to-face semi-structured interview (the interview schedule is available in S1 File).

[Figure omitted. See PDF.]

Table 1. Example role player scenario and script used in England.

https://doi.org/10.1371/journal.pone.0229999.t001

International.

Participants from specialist genetics clinics in France and Germany took part in an online version of the study via the Web-based survey tool Qualtrics [17]. They completed the same initial questionnaire, followed by two vignette-based test cases, adapted from the scenarios used in the English setting but with the same risk factors as in the SPE cases. Demographic information pertaining to a fictitious patient was provided for each case alongside details of their family history (see Table 2 for an example). Participants were asked to enter the information into the CanRisk tool and evaluate how it would work in a real-world setting. Participants were also prompted to take notes as they used the CanRisk tool in order to complete a detailed open-ended questionnaire (again drawn closely from the interview schedule) exploring their user experiences.

[Figure omitted. See PDF.]

Table 2. Example vignette based case used in France and Germany.

https://doi.org/10.1371/journal.pone.0229999.t002

Analysis

Demographic data and information about current use of computerised risk tools were summarised. Qualitative data from the open-ended questionnaires and verbatim transcriptions from the semi-structured interviews were analysed inductively using thematic analysis [18]. In order to become familiar with the data, two authors (SA and FS) repeatedly read the interview transcripts, and notes were made about potential codes; these were subject to ongoing discussion and refinement. Following this, the same authors performed formal coding using NVivo [19], working through the transcripts and writing notes to support data patterns–these patterns were extracted as codes. Consistency of coding between SA and FS was discussed regularly through team meetings, but formal analysis of the concordance between reviewers was not completed. The codes were drawn together into a broad set of initial themes; these were refined with guidance of the senior author (FW). The final themes were mapped onto Sekhon et al.’s Theoretical Framework of Acceptability [12] which was developed to guide the assessment of acceptability from the perspectives of intervention deliverers and recipients both prospectively and retrospectively. The framework proposed seven component constructs: affective attitude, burden, perceived effectiveness, ethicality, intervention coherence, opportunity costs, and self-efficacy.

Results

Demographics

In England, 37 healthcare professionals participated in the study, including 11 General Practitioners (GPs), 10 Practice Nurses (PNs), 6 Clinical Geneticists (CGs) and 10 Genetic Counsellors (GCs) (Table 3). In their patient consultations, most (31/37, 84%) were already using computerised risk tools, relating to cancer (n = 14; GP = 2; CG = 4; GC = 8), cardiovascular (n = 18; GP = 10, PN = 8), and diabetes (n = 4; GP = 1, PN = 3) risk.

[Figure omitted. See PDF.]

Table 3. Participant demographics.

https://doi.org/10.1371/journal.pone.0229999.t003

In France and Germany, 68 people from specialist genetics clinics were approached, and 42 (62%) participated in the study. Four respondents were excluded from the analysis as they were not practising clinicians. The final sample of 38 healthcare professionals comprised 26 Genetic Counsellors (GCs) and 12 Clinical Geneticists (CGs) from specialist genetics services in France (n = 23) and Germany (n = 15). Most (36/38, 95%) were already using computerised risk tools in their consultations to calculate cancer risks.

For the purposes of this analysis, data from GCs and CGs in England, France and Germany were combined into the specialist clinical genetics sample (n = 54). The primary care sample was from GPs and PNs in England alone (n = 21). Detailed participant demographics for each of these groups are displayed in Table 3.

Analysis

The thematic analysis generated 17 themes—these were organised around the seven theoretical constructs outlined by Sekhon et al., [12]; Fig 4 shows a thematic map.

[Figure omitted. See PDF.]

Fig 4. Inductive themes mapped onto Sekhon et al’s Theoretical Framework of Acceptability [12].

The total number of times the theme occurred is presented after each theme name (e.g. n = x). The yellow (primary care) and green (specialist genetics clinic) colouring shows the proportional contribution of each sample to the theme.

https://doi.org/10.1371/journal.pone.0229999.g004

‘Affective attitude’: How clinicians felt about the CanRisk tool.

Affective attitude was directly influenced by participants’ first impressions. For specialist genetics clinicians, the majority of whom had used the previous simple web-based BOADICEA interface, there was a clear improvement:

“The CanRisk tool is an improvement to any other risk assessment tool based on BOADICEA. It is straightforward and has a simplistic design, and is therefore intuitive to use” (GC, Germany).

Clinicians from the specialist genetics setting anticipated that the CanRisk tool would be used frequently before and/or during their clinics, and would influence further testing and treatment decisions. In contrast, whilst some primary care clinicians had a positive outlook on the appropriateness of using the CanRisk tool in their setting, they perceived that the CanRisk tool would not be used frequently:

“I thought it was completely something reasonable for a GP to engage in or carry out. So perfectly reasonable consultations and appropriate and relevant. So it's the sort of thing that would happen, not frequently, but I would see these sorts of cases certainly” (GP, England).

Furthermore, several primary care clinicians were concerned that the CanRisk tool was not contributing to a management plan for their patient, and was simply providing more data on a risk they had already calculated personally:

“I’m not sure how useful it is in the primary care setting because all I’m doing is, I’m using that to collect data, and I’m just thinking it’s going to calculate my risk and it’s not using it to make any clinical decision” (GP, England).

Participants from both settings expressed views on whether the CanRisk tool could be, and should be, used in face-to-face consultation with patients. Some felt that the CanRisk tool was an asset for structuring complex conversations around factors contributing to cancer risk and involving patients in the consultation:

“It’s a good way of introducing conversation about risk and about risk perception. And as long as you’re doing it with them looking at the screen, so there’s that involvement” (GC, England).

Others were more concerned about using the CanRisk tool alongside patients due to the ‘real time’ nature of risk communication. Some clinicians described their anxieties about the lack of opportunity to interpret the risk scores before sharing the relevant information with the patient:

“I wasn’t sure what the risk was going to be calculated as, so I didn’t know what to expect, and we were going to get that at the same time, and I didn’t know how to convey that information when you can both see the numbers there and then” (GC, England).

In order to provide the best experience of care for patients in this time pressured environment, participants highlighted that they would need to be confident in their understanding, interpretation and communication of multi-factorial cancer risk; this is discussed further in a later theme on self-efficacy.

‘Ethicality’: How the CanRisk tool fits with the clinician’s value systems.

Ethicality encompassed participants’ concerns about how the CanRisk tool changed the dynamic of the consultation. Clinicians from both settings were worried about issues such as reduced eye contact, reduced time spent building rapport, and an altered flow to their normal consultation.

“I think my interaction and communication with the patient was terrible. I didn't get to interact with them, I didn't get to deal with their anxieties around the issue. I was more focusing on inputting data into the tool” (GP, England).

Further to this, participants from the primary care setting were concerned about the depth and nature of some of the questions, particularly around pregnancy and baby loss, describing them to be intrusive and insensitive:

Participant: “That’s a bit horrible, isn’t it? Please include stillbirths, but it’s not necessary to include miscarriages”.

Interviewer: “How could we improve that do you think?”

Participant: “I don’t know. You want it to be clear and it’s better to be upfront. It’s easy for me in that I can see what you want, but then it’s a little bit… to ask that question. I didn’t ask either of those women that question” (PN England).

Clinicians in the specialist genetic setting were also concerned that the patients may feel judged for engaging in particular behaviours (i.e. alcohol), being overweight or for not breastfeeding their child:

“I’m thinking if I ask questions about breastfeeding and alcohol, it’s going to make no difference to the screening recommendations we make. It’s actually just going to make someone go away feeling a bit shit because they were told they drink too much and they didn’t breastfeed” (GC, England).

In order to allay these concerns, participants discussed the need for clear explanation about the CanRisk tool, how it works, and why particular types of information were needed in order to achieve the best possible risk estimation. To this end, time in the consultation to achieve this (or a lack of it in many cases) was seen as a major limitation, particularly in primary care, but not a complete barrier to use.

‘Burden’: The effort required to use the CanRisk tool.

All participants discussed burden, or the alleviation of it. Some primary care participants described how the CanRisk tool could improve their clinical practice, and that ease of use was an important factor:

“It’s quite simple to use and I like the fact that it’s got the links, so I can link straight over to the NICE guidelines, I can go straight to the public health and look at the breast screening programme” (PN, England).

Other primary care participants felt that the CanRisk tool was confusing, particularly the aspects that were unfamiliar in their normal clinical practice (e.g. the completion of extended and detailed family histories); in some cases, they were concerned that this could impact on the risk calculation:

“It’s [family history] quite difficult to put in and then if we’re guessing, how accurate is the result? So, potentially, I could be putting someone in a much higher category than they actually are and then they’re going to go away and worry, and then they go to the hospital and actually they say, no you’re low risk” (PN, England).

Participants from both settings thought that using the CanRisk tool in clinical practice could complicate the consultation, particularly when patients might not have all of the information readily available. Clinicians from the specialist genetics setting felt that the CanRisk tool was unnecessarily complicated for a simple risk stratification as most of their patients have significant family histories that would not be moderated by other risk factors:

“So, this tool, in a way, made things a little more complicated for me than it needed to be, I suppose: there are certain categories of women that automatically will fall into that high risk, for me” (CG, England).

Primary care clinicians were clearly concerned about the amount of time needed to complete the CanRisk tool. They were apprehensive about their ability to complete the tool, achieve a good patient experience and initiate future management planning within a short consultation:

“It’s just whether you have time to do it. I don’t think you will have time to go through the whole thing without sort of establishing rapport initially, and finishing and closing at the end” (GP, England).

Specialist genetics clinicians rarely described the time needed to complete the tool as a limiting factor. Indeed, they reported having fewer time pressures due to the nature of their clinical set-up, which generally features longer appointments (up to 45 minutes in England) and often affords greater opportunities to prepare for each consultation.

‘Intervention coherence’: Understanding the CanRisk tool and how it works.

Intervention coherence focused predominantly on participants’ beliefs about the accuracy of the risk assessment and the trust they had in the model. For some participants, endorsement of the CanRisk tool by NICE or other trusted authorities instilled greater trust:

“So, from the cancer risk point of view, obviously, Macmillan or anything like that, or Cancer Research I’d be thinking oh yes, or if it’s got NICE approval as well, then I think yes, okay, that’s good” (PN, England).

Adoption of the CanRisk tool within a clinical system (i.e. a hospital or GP practice) would also give some participants a feeling of assurance that it was appropriate for use in that clinical setting:

“If [Clinical IT Specialist] has deemed, and the doctors have deemed, it safe to embed into our templates, then I would trust that that's okay for me to use” (PN, England).

Whilst some participants were happy to rely on the risk calculation given by the CanRisk tool because of its reputation in their clinical community, others expressed concerns about receiving a risk calculation that did not fit with their clinical judgement:

“I have a figure that I’m expecting, and I find that very handy because if it’s wildly different, it alerts me that there could be concerns” (CG, England).

Specialist genetics clinicians found a discrepancy between the CanRisk calculation and their clinical judgement particularly challenging, as many were used to calculating risk based solely on family history information:

“So, you see, when we calculate risk, we don’t take that into account, and we don’t take HRT into account, or contraceptive pill into account, at the moment. So, it’s very much based on the family history. So, actually, having that be a bit higher, it just makes you think, actually, lifestyle factors may play a part, yes” (GC, England).

Some participants were also concerned about the impact of incomplete information about any of the risk factors, particularly lifestyle and hormonal factors, might have on the risk calculation:

“A lot of personal information is asked. What about patients [for] who we don't have all these items—the cancer risk is not as precise?” (GC, France).

Many of these concerns were underpinned by a lack of knowledge about how the calculation was made, leaving some participants feeling uncomfortable about the reliability and specificity of the risk calculation:

“[We’d] say, “This is one model we’ve got, one tool we’ve got for estimating your risk and it’s coming out saying your risk is about two percent.” We’d say that all datasets are incomplete, there may be new data and all sorts of things, but this is the current model. So I think in terms of regulation, it’s really important that that uncertainty is somehow captured when it’s being used in different clinical situations” (GC, England).

Some participants felt that the calculated risk figure needed to include a caveat to better represent the uncertainty. Participants wanted greater transparency to ensure that healthcare professionals and their patients were fully informed about the limits of both the BOADICEA model and the CanRisk tool.

‘Opportunity costs’: The sacrifice of benefits, profits and values.

Opportunity costs were seldom discussed by participants in this study. Only data security, in terms of the safety of data entered into the CanRisk tool, was seen a potential problem. For some, there were concerns about data protection, specifically around that integration of risk prediction tools into existing IT systems:

“I think that’s a legitimate concern, we have electronic records and some people are worried about that, and so having online tools where you’re putting in all of this personal information, putting in your family members, that might spark some patient concerns” (GC, England).

Other concerns focused on the security of the Web more generally, the chances for data to be hacked, and whether any secondary use of the data was planned. None of the participants reported that these concerns would stop them from using the CanRisk tool, but they acknowledged that these points should be addressed in any guidance documentation.

‘Perceived effectiveness’: The ability of the CanRisk tool to achieve its purpose.

Perceived effectiveness was a central tenet of acceptability for most participants in this study, with views on efficacy being linked to positive attitudes towards the CanRisk tool. Some participants believed that the use of the CanRisk tool promoted more thorough data collection and therefore improved the quality of information:

“So that's good, because you don't miss information and it's quite tidy. And, in your mind, you can understand where you're going with the information. And, actually, it's not even very alarming to the patient, because they can see where you're headed with the questions” (GP, England).

Following completion of the risk calculation, many participants talked about the benefits of having varied and versatile formats to help them communicate the patient’s risk:

“It was really helpful offering various versions [of risk communication]. At least one should fit for all of the patients—it is sometime helpful explaining the same message using different paths” (CG, Germany).

However, several PNs and GCs felt that the range of options for communicating risk were overwhelming. In some cases, the additional information was felt to be unnecessary as the risk calculation forms just one part of the wider consultation:

“Sometimes you won’t need to go into any more detail than moderately increased risk. Sometimes that’s absolutely fine. Moderately increased risk, you know, and actually the rest of the consultation might be about their feelings surrounding coming up to the age at which their mum was diagnosed and their thoughts about that” (GC, England).

In general, clinicians from the specialist genetics setting were confident about the next steps following a risk calculation; in some cases the tool was used to support a decision for a particular type of treatment (e.g. risk-reducing mastectomy). Conversely, primary care participants talked about their uncertainty about the next steps following the risk calculation and there was a general frustration about the lack of information on clinical management:

“I would know the risk stratification off the top of my head and I would know what to do next with it. But I'd know there is some guidelines somewhere that I need to follow in terms of whether or not they need a referral or not. But if we are going to use a risk stratification tool, then I would want some sort of, and you do this next, it's this is the risk, deal with it, GP” (GP, England).

Following this, some PNs were concerned about how and when to pass the case over to a GP colleague, particularly if the patient was identified as being at moderate or high risk. Some PNs were conscious of their lack of knowledge of the next steps, and this affected their confidence in being able to provide appropriate information to their patient:

“I don’t even know what, how quickly, people would get assessed for BRCA, I wouldn’t know if I send them here, if they’d have to go to Place 1, I really don’t know, so, because it’s not something I’ve done” (PN, England).

Whilst some participants believed that the CanRisk tool could help regulate the collection of data and structure conversations around risk, for the majority, the desire to develop confidence and competence when using the CanRisk tool was integral to its acceptability.

‘Self-efficacy’: Clinician’s confidence in using the CanRisk tool.

Self-efficacy was predicated upon a need for additional training. Participants felt that training should focus on how to best use the CanRisk tool, promoting a sense of proficiency:

“We should be familiar with the tools that we’re using, so that we use them properly. And therefore, I think there should be a training component to it. Otherwise, you make mistakes” (CG, England).

All participants wanted to be able to feel comfortable about using the CanRisk tool in a consultation. Those from the specialist genetics clinics felt more confident due to their experience of using the simple web-based BOADICEA interface. In addition, their specialist training left them comfortable communicating cancer risk to their patients, but some struggled to describe the impact of non-genetic factors. Conversely, whilst most primary care participants were confident in their communication skills around multi-factorial risk, some were concerned about their limited genetics knowledge:

“Genetics, not part of my normal knowledge. Even in medical school you do a tiny, weeny bit of it, it's not, that is an absolute specialist area” (GP, England).

Whilst only one PN said they were ‘not so sure this would be an appropriate thing for somebody of my level of nursing to do’ (PN, England), many other primary care participants stated that they would need additional training on interpreting genetic risk and its communication:

“So you've got to have an education package that goes with it, to tell people what it's about, why all the things are on there and actually how to interpret it. I think most of us, as doctors, my sort of age, never had any consultations since training, we'd have never been taught how to convey risk” (GP, England).

Whilst the training needs of participants varied depending of the setting and/or the clinical training and experience of the participant, there was commonality in terms of a desire to provide the best care for their patients.

Discussion

Our findings suggest that the prototype CanRisk tool was broadly acceptable to the majority of participants. Most clinicians found the CanRisk tool easy to use due to its intuitive design. For geneticists and genetic counsellors who were users of the simple web-based BOADICEA interface, CanRisk offered a more user-friendly experience and the standardised format for data collection was of benefit to many. The foremost concern for clinicians from the primary care setting was the amount of time needed to complete and interpret a risk calculation, highlighting a potential barrier to the use of the CanRisk tool in this setting. Clinicians from both settings were also apprehensive about the impact that the CanRisk tool had on their consultation and the lack of opportunity to interpret the risk score before sharing it with the patient.

Many participants found the CanRisk tool to be intuitive; however, the majority were from specialist genetics clinics with previous experience of the simple web-based BOADICEA interface. All participants were concerned about potentially unhelpful changes to their consultation style influenced by the inclusion of detailed questions in the prototype that may appear to be unrelated to cancer risk, and increased time spent looking at the computer screen, consistent with the ‘third party effect’ [20]. In order to allay these concerns, a period of familiarisation and training prior to using the CanRisk tool for the first time in the clinical setting is advisable. This would allow healthcare professionals to navigate the CanRisk tool confidently, select which risk factors are important for that patient, and collect information on these in a way that is consistent with their consultation style [20, 21].

Many participants were also concerned about collecting the extra information needed for this multi-factorial cancer risk prediction tool. This was particularly evident when these topics are not routinely explored such as lifestyle factors by clinical geneticists and genetic counsellors and extended detailed family histories for GPs and practice nurses. Primary care clinicians were particularly concerned about the amount of information required for risk prediction as they had limited time available during their consultations [22]. In order to improve the acceptability of the CanRisk tool, the burden of data collection must be addressed. Some areas of the CanRisk tool could be pre-populated from existing clinical records; this would require the CanRisk tool to be embedded in electronic medical records (e.g. EMIS or SystmOne for primary care, or EPIC for hospital care). Another approach could be for patients to complete an electronic pro-forma. Whilst these two potential solutions may sound simple, the technical and regulatory requirements may make it difficult to navigate on a local and national level [23, 24], and may add further concerns around data security.

The study findings suggest that endorsement from appropriate clinical bodies is essential for confidence in adopting new technologies into the clinical setting. Whilst BOADICEA is referenced in the NICE clinical guidance for familial breast cancer (CG164) [15], clinicians may not be aware of the association between BOADICEA and the CanRisk tool. Including a reference or link to the CanRisk tool specifically within the guideline may increase visibility of the tool, strengthen the association between BOADICEA and the CanRisk tool, and increase adoption. The addition of CE marking to the CanRisk tool, indicating conformity with health, safety and environment protection standards for products within the European Economic Area [25], is also likely to support acceptability across the clinical settings.

Although some clinicians regarded the variety of risk presentation formats as a useful feature, others found them to be overwhelming. Further consideration of the risk presentation formats for each job role and/or clinical setting may bolster the clinician’s confidence, assist in quick interpretation and improve patients’ understanding and perception of risk [26]. In order to achieve shared understanding, incorporating layers of information, where details about the risk and its parameters are displayed in an incremental and controlled fashion, may be of benefit to both patients and staff [27]. Additional learning resources focusing on the communication of multifactorial risk for those in the specialist genetics setting and genetic risk for those in the primary care setting may also support clinicians to communicate the results more effectively. The development and testing of these resources should be considered prior to implementation.

The analysis highlighted a number of factors influencing participants’ views on the perceived effectiveness of the final CanRisk tool. Specifically, there were concerns about the validity of the output when there is missing information; more detailed information regarding the complex calculations performed in BOADICEA and the contribution of each risk factor to the model may be required. Information on the amount of error within each calculation should be included to better inform users about the level of uncertainty. This may help develop patient centred understanding of the risk score and cancer risk more broadly and what role this information may have in future care decisions [28]. The analysis also emphasised a need for the CanRisk tool to include clinical guidance on managing patients following a risk calculation. This was particularly important in the primary care setting, where most management of women found to be at population and moderate risk for breast cancer is likely to take place.

Limitations

This study collected data on acceptability from a variety of clinician groups, using a range of methods; whilst this allowed us to achieve both breadth and depth, it also resulted in a number of limitations. Firstly, this analysis was generated using data from both semi-structured interviews and questionnaires; the use of questionnaires to collect detailed experiential information was pragmatic, but the opportunity to explore some issues in more detail was limited. Secondly, no GPs contributed data in the international sample; this restricted the feedback from an integral group of clinicians and limits the generalisability of this research to the international primary care community. Thirdly, as the majority of clinicians from the specialist genetics clinics had previous experience of using BOADICEA in their clinical practice, some aspects of the CanRisk tool were already familiar; this may have amplified the differences between clinicians from primary care and specialist genetics clinics.

Next steps

Further development and testing of the CanRisk tool is required. The findings presented here have already underpinned a number of changes to the prototype CanRisk tool (see Table 4). Whilst these changes may go some way to improving the tool for clinical users in a variety of settings, further research exploring the patient experience of the tool and its use in clinical practice is necessary. Data collected in these studies would allow us to understand the acceptability of the CanRisk tool in this important stakeholder group and may inform future patient centred design.

[Figure omitted. See PDF.]

Table 4. Summary of changes made to the prototype CanRisk tool.

https://doi.org/10.1371/journal.pone.0229999.t004

Conclusion

The prototype CanRisk tool was generally acceptable to our participating clinicians. However, the findings highlight the challenges associated with developing a complex tool for use in different clinical settings. The next version of the CanRisk tool is likely to benefit from having core and clinical setting-specific components to increase acceptability across the clinical settings and, ultimately, future implementation.

Supporting information

[Figure omitted. See PDF.]

S1 File.

https://doi.org/10.1371/journal.pone.0229999.s001

(DOCX)

Acknowledgments

We would like to thank Maureen Rae, Rita Schmutzler, Jon Roberts, Antoine de Pauw and Hugh Wilson for their contributions to aspects of completing this study.

Citation: Archer S, Babb de Villiers C, Scheibl F, Carver T, Hartley S, Lee A, et al. (2020) Evaluating clinician acceptability of the prototype CanRisk tool for predicting risk of breast and ovarian cancer: A multi-methods study. PLoS ONE 15(3): e0229999. https://doi.org/10.1371/journal.pone.0229999

References

1. Eccles SA, Aboagye EO, Ali S, Anderson AS, Armes J, Berditchevski F, et al. Critical research gaps and translational priorities for the successful prevention and treatment of breast cancer. Breast cancer research: BCR. 2013;15(5):R92. Epub 2013/11/30. pmid:24286369; PubMed Central PMCID: PMC3907091.

2. Howell A, Anderson AS, Clarke RB, Duffy SW, Evans DG, Garcia-Closas M, et al. Risk determination and prevention of breast cancer. Breast Cancer Research. 2014;16(5):446. pmid:25467785

3. Usher-Smith J, Emery J, Hamilton W, Griffin SJ, Walter FM. Risk prediction tools for cancer in primary care. British Journal Of Cancer. 2015;113:1645. pmid:26633558

4. Antoniou AC, Cunningham AP, Peto J, Evans DG, Lalloo F, Narod SA, et al. The BOADICEA model of genetic susceptibility to breast and ovarian cancers: updates and extensions. Br J Cancer. 2008;98(8):1457–66. Epub 2008/03/20. pmid:18349832; PubMed Central PMCID: PMC2361716.

5. Antoniou AC, Pharoah P, Smith P, Easton DF. The BOADICEA model of genetic susceptibility to breast and ovarian cancer. British journal of cancer. 2004;91(8):1580. pmid:15381934

6. Lee AJ, Cunningham AP, Tischkowitz M, Simard J, Pharoah PD, Easton DF, et al. Incorporating truncating variants in PALB2, CHEK2, and ATM into the BOADICEA breast cancer risk model. Genetics In Medicine. 2016;18:1190. https://www.nature.com/articles/gim201631#supplementary-information. pmid:27464310

7. Lee A, Mavaddat N, Wilcox AN, Cunningham AP, Carver T, Hartley S, et al. BOADICEA: a comprehensive breast cancer risk prediction modelincorporating genetic and nongenetic risk factors. Genetics in Medicine. 2019. pmid:30643217

8. Fischer C, Kuchenbäcker K, Engel C, Zachariae S, Rhiem K, Meindl A, et al. Evaluating the performance of the breast cancer genetic risk models BOADICEA, IBIS, BRCAPRO and Claus for predicting BRCA1/2 mutation carrier probabilities: a study based on 7352 families from the German Hereditary Breast and Ovarian Cancer Consortium. Journal of Medical Genetics. 2013;50(6):360–7. pmid:23564750

9. MacInnis RJ, Bickerstaffe A, Apicella C, Dite GS, Dowty JG, Aujard K, et al. Prospective validation of the breast cancer risk prediction model BOADICEA and a batch-mode version BOADICEACentre. Br J Cancer. 2013;109(5):1296–301. Epub 2013/08/15. pmid:23942072; PubMed Central PMCID: PMC3778274.

10. Terry MB, Liao Y, Whittemore AS, Leoce N, Buchsbaum R, Zeinomar N, et al. 10-year performance of four models of breast cancer risk: a validation study. The Lancet Oncology. 2019;20(4):504–17. Epub 2019/02/26. pmid:30799262.

11. Johnson CM, Johnson TR, Zhang J. A user-centered framework for redesigning health care interfaces. Journal of biomedical informatics. 2005;38(1):75–87. pmid:15694887

12. Sekhon M, Cartwright M, Francis JJ. Acceptability of healthcare interventions: an overview of reviews and development of a theoretical framework. BMC health services research. 2017;17(1):88. pmid:28126032

13. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ (Clinical research ed). 2008;337:a1655. pmid:18824488

14. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ (Clinical research ed). 2015;350:h1258. Epub 2015/03/21. pmid:25791983; PubMed Central PMCID: PMC4366184.

15. NICE. Familial breast cancer: classification, care and managing breast cancer and related risks in people with a family history of breast cancer. In: Excellence NIfHaC, editor. London2013 (Updated 2017).

16. Carver T, Cunningham AP, Babb de Villiers C, Lee A, Hartley S, Tischkowitz M, et al. pedigreejs: a web-based graphical pedigree editor. Bioinformatics. 2017;34(6):1069–71. pmid:29095980

17. Qualitrics. Qualtrics XM. Provo, UT, USA2018.

18. Braun V, Clarke V. Using thematic analysis in psychology. Qualitative research in psychology. 2006;3(2):77–101.

19. QSR International. NVivo. 2018.

20. Emery J, Walton R, Coulson A, Glasspool D, Ziebland S, Fox J. Computer support for recording and interpreting family histories of breast and ovarian cancer in primary care (RAGs): qualitative evaluation with simulated patients. BMJ (Clinical research ed). 1999;319(7201):32–6. pmid:10390458

21. Short D, Frischer M, Bashford J. Barriers to the adoption of computerised decision support systems in general practice consultations: a qualitative study of GPs’ perspectives. International Journal of Medical Informatics. 2004;73(4):357–62. pmid:15135754

22. Deveugele M, Derese A, van den Brink-Muinen A, Bensing J, De Maeseneer J. Consultation length in general practice: cross sectional study in six European countries. BMJ (Clinical research ed). 2002;325(7362):472. Epub 2002/08/31. pmid:12202329; PubMed Central PMCID: PMC119444.

23. Shah N, Martin G, Archer S, Arora S, King D, Darzi A. Exploring mobile working in healthcare: Clinical perspectives on transitioning to a mobile first culture of work. International Journal of Medical Informatics. 2019;125:96–101. pmid:30914187

24. Trivedi MH, Daly EJ, Kern JK, Grannemann BD, Sunderajan P, Claassen CA. Barriers to implementation of a computerized decision support system for depression: an observational report on lessons learned in "real world" clinical settings. BMC medical informatics and decision making. 2009;9(1):6. pmid:19159458

25. Department for Business EIS. CE marking 2012 [23/8/2019]. Available from: https://www.gov.uk/guidance/ce-marking.

26. Edwards AG, Naik G, Ahmed H, Elwyn GJ, Pickles T, Hood K, et al. Personalised risk communication for informed decision making about taking screening tests. The Cochrane database of systematic reviews. 2013;(2):Cd001865. Epub 2013/03/02. pmid:23450534; PubMed Central PMCID: PMC6464864.

27. Spiegelhalter D, Pearson M, Short I. Visualizing uncertainty about the future. Science (New York, NY). 2011;333(6048):1393–400. Epub 2011/09/10. pmid:21903802.

28. Domchek SM, Eisen A, Calzone K, Stopfer J, Blackwood A, Weber BL. Application of breast cancer risk prediction models in clinical practice. Journal of clinical oncology: official journal of the American Society of Clinical Oncology. 2003;21(4):593–601. Epub 2003/02/15. pmid:12586794.