The prescribing safety assessment: Looking to the future

Abstract

[...]it was highlighted that while PSA practice was seen as a positive activity, there was concern that too much focus on specific question performance may detract from teaching the ‘skill’ of prescribing to merely learning to pass an assessment. real tension between learning how to pass an exam and how to be good at something Attendees who worked within universities expressed the view that the opportunity to use the PSA platform for teaching purposes would be valuable and that their students had a desire to practice prescribing on the platform ahead of their PSA. [...]there was the sense that the PSA should be promoting clinical reasoning skills, rather than encouraging reliance on looking all the answers up. Quantifying the impact that the PSA has had on improving prescribing competency is complex: research in this field requires the ability to design a study that eliminates confounders such as e-prescribing, addresses the significant limitations of error reporting, and has sufficient data to demonstrate statistical significance. Suggestions from authors to address this included analyzing pre- and post-assessment prescribing of cohorts of students, or utilizing qualitative measures such as student confidence.

INTRODUCTION

The Prescribing Safety Assessment (PSA) is a high-stakes exam sat by approximately 8000 final-year UK medical students each year.¹ All doctors joining the NHS in foundation year 1 (FY1) are required to pass this assessment to progress through to their second year of foundation training.² The PSA has been a very successful initiative; since being first piloted in 2012, it has been adopted by every UK medical school, won awards, and attracted attention internationally. However, with the planned introduction of the Medical Licensing Assessment (MLA) for all UK final-year medical undergraduates, the PSA needs to ensure its continued position as a valuable assessment across UK medical programs. To investigate this, the British Pharmacological Society and the Medical Schools Council have commissioned the Dacre Review which aims to consider how the PSA can best continue to meet the prescribing training needs of health professionals.³

To explore this issue, the opinions of PSA authors who attended the annual ‘PSA peer review conference’ were gathered through a series of scenario-based discussions. Attendees were made aware of the audio recording during the discussion for research purposes and gave their implied consent for their anonymized responses to be used. Their responses are discussed within this commentary.

DEMOGRAPHIC BREAKDOWN

A total of 39 question authors were present at the 2022 conference. Author backgrounds reflected a variety of professions including medicine (54%, N = 21) and pharmacy (38%, N = 15). The remaining attendees were PSA administrative team members. Among medics, there was a range of training grades reflected in the attendees with the majority being consultants (57%); 23% were doctors in specialty training and 20% were junior doctors.

18 attendees listed a university as their primary place of work, with 13 different institutions represented. 11 attendees were clinical academics working at UK medical schools.

FINDINGS

When looking at the responses from across the activities, four main themes emerged:

How to keep the PSA relevant

a way that PSA can be embedded throughout a student's clinical journey

A key focus was how to integrate the PSA within the medical school curriculum and whether this could be extended beyond a single, final-year high-stakes assessment. Authors proposed the exposure of students to the assessment in earlier years of their training, through formal, but formative, timed assessments. To ensure relevance, an emphasis on feedback for specific areas of poor performance was suggested. However, it was highlighted that while PSA practice was seen as a positive activity, there was concern that too much focus on specific question performance may detract from teaching the ‘skill’ of prescribing to merely learning to pass an assessment.

real tension between learning how to pass an exam and how to be good at something

Attendees who worked within universities expressed the view that the opportunity to use the PSA platform for teaching purposes would be valuable and that their students had a desire to practice prescribing on the platform ahead of their PSA. Authors felt the opportunity to use the PSA platform for local teaching would make the PSA relevant to all years of the medical school, not only those in final year.

When considering how to keep the PSA relevant within a wider context, non-medical prescribers were identified as a key group with whom engagement was required. It was suggested that this was an opportunity to set the PSA as the benchmark assessment for all prescribers and that it could be integrated into other training programs.

I would suggest the fact that all pharmacist graduates are going to come out as prescribers in 2026…it's a golden opportunity

The British Pharmacological Society Assessments (BPSA) group has been working with a small number of institutions to integrate the platform that the PSA uses into their undergraduate curriculum. Looking to the future, this opportunity should be available in all institutions so that the platform is widely adopted as the teaching tool and means of assessment for prescribing competency.

How to make the PSA representative

While many agreed with the principle of ensuring that the assessment is representative of the scenarios that foundation doctors may encounter, the question was raised as to whether this reflects the main aim of the assessment, which is to assess safe prescribing practice.

It was suggested that if the assessment was limited to the formulary commonly used by foundation doctors for independent initiation of medications, the range of drugs would be very limited and exclude drugs with important patient safety considerations. It was considered important to assess knowledge of other medications that students will be exposed to during practice and to promote the idea of developing knowledge beyond their current level of practice.

FY1s will be prescribing complex drugs…they may not be initiating them, but it will be their signature

Another issue discussed was whether to increase the resources candidates have access to during the assessment from just the BNF, to include other resources that prescribers access within a clinical environment. Concerns were raised about the difficulties that may pose in setting valid assessment questions, as even recognized resources can contradict each other, making decisions about the ‘correctness’ of answers challenging. Relying on a single source for the purposes of the assessment helps to limit that ambiguity. There were also concerns around how a wider list of permitted resources could negatively affect candidates during the assessment; the PSA is time pressured and consulting multiple resources would hinder progress. Finally, there was the sense that the PSA should be promoting clinical reasoning skills, rather than encouraging reliance on looking all the answers up.

I think it is kinder to the students to restrict them…I don't think they would recognise this

In order to ensure that the PSA moves with practice and continues to represent the scenarios faced by FY1 doctors, recent PSA recruitment initiatives have encouraged applications from individuals with current or recent experience of being a foundation doctor and with diverse backgrounds that reflect the population of candidates as a whole. This group would have a better understanding of the common tasks required of a foundation doctor. Future developments such as including new question templates and scenarios about patient clerking were suggested, but it was recognized that this would require significant resource to make fundamental changes to an established assessment.

How to promote engagement with the PSA

we don't have a contact in every medical school…we have contacts with those who actively contribute…then there are some black holes…

Ensuring support of the assessment from stakeholders was expressed to be important in ensuring the longevity of the assessment. Universities were identified as a key stakeholder group and the concern was that there were some institutions which are not represented in any PSA groups. These institutions needed to be engaged with to understand the barriers which inhibit participation and to identify means of promoting involvement. It was noted that new medical schools had opened since the inception of the PSA, and maintaining an up-to-date network of representatives from each medical school will be important in promoting involvement.

if we provided more for them…it was less of a chore

It was acknowledged that there will always be some institutions that are less motivated to be involved in the PSA. Possible incentives to help address this could include opening more of the question bank for formative use, so that the benefits of the collaboration would be more visible and tangible.

One possible future development suggested would be for the PSA to be moved to be a requirement for entering, rather than at the end of FY1, which could result in more medical schools engaging in the process.

How to ensure the PSA is valued

how to prove we have made a difference…when everything else around you is dynamically changing

An overarching theme in many of the discussions was the importance of ensuring the PSA remains valuable, valid, and has integrity. It was noted that since the creation of the PSA, there has been a shift to integrate the concept of safe prescribing into curricula.

Issues of collusion discovered during the pandemic highlighted the need to strengthen processes that ensure integrity and security of the assessment. Stronger governance around the assessment will be an important outcome of the Dacre Review. Involvement of stakeholders in this evolution will help maintain trust in the assessment. While the Dacre Review findings are yet to be published, steps have already been taken to address many of the challenges mentioned in this piece.

Quantifying the impact that the PSA has had on improving prescribing competency is complex: research in this field requires the ability to design a study that eliminates confounders such as e-prescribing, addresses the significant limitations of error reporting, and has sufficient data to demonstrate statistical significance. Suggestions from authors to address this included analyzing pre- and post-assessment prescribing of cohorts of students, or utilizing qualitative measures such as student confidence. However, investment in such research would require a significant amount of funding and participation across institutions over a prolonged period of study.

engagement relies on trust

CONCLUSION

The PSA has successfully evolved to become the benchmark for prescribing safety across the UK. The challenge for such a large initiative, which is run by a small team, on a limited budget, is to remain current and at the cutting edge of assessment in this field.

To address the issues discussed, the next steps include working with more institutions to integrate the platform into undergraduate teaching and assessment in earlier years of the curriculum. The sharing of success and best practice of early adopter institutions with those not yet using the platform will help ensure that the most effective initiatives can be widely adopted. In addition, greater university engagement will help create the investment needed to support high-quality research, performed at scale, to demonstrate the impact of the PSA on prescribing performance. Lastly, aligning the PSA governance models with other national assessments, such as the MLA, will ensure that assessment has active oversight and robust quality assurance methods moving forward.

AUTHOR CONTRIBUTIONS

Dr. Ellen Haslam has made substantial contributions to conception and design, acquisition of data and analysis. Has been involved in drafting the manuscript and revising it critically for important intellectual content; and given final approval of the version to be published.

Professor Kurt Wilson has made substantial contributions to conception and design, acquisition of data and analysis. Has been involved in drafting the manuscript and revising it critically for important intellectual content; and given final approval of the version to be published.

Dr Lynne Bollington has made substantial contributions to conception and design, acquisition of data and analysis. Has been involved in drafting the manuscript and revising it critically for important intellectual content; and given final approval of the version to be published.

Professor Simon Maxwell has made substantial contributions to conception and design, acquisition of data and analysis. Has been involved in drafting the manuscript and revising it critically for important intellectual content; and given final approval of the version to be published.

ACKNOWLEDGMENTS

University of Manchester provided funding for the publication of this work.

CONFLICT OF INTEREST STATEMENT

Simon Maxwell, Lynne Bollington, and Kurt Wilson are all current members of PSA executive committee. Ellen Haslam is a PSA question author.

DATA AVAILABILITY STATEMENT

The data is not publicly available as consent was not gained for sharing of raw audio recordings or transcripts. Consent was gained for themes and anonymized comments to be used and published.

ETHICS STATEMENT

Consent was gained from all participants for their contributions to be audio recorded and their anonymized responses to be used for research purposes.