Does performance at the intercollegiate Membership of the Royal Colleges of Surgeons (MRCS) examination vary according to UK medical school and course type? A retrospective cohort study

Abstract

Objectives

The knowledge, skills and behaviours required of new UK medical graduates are the same but how these are achieved differs given medical schools vary in their mission, curricula and pedagogy. Medical school differences seem to influence performance on postgraduate assessments. To date, the relationship between medical schools, course types and performance at the Membership of the Royal Colleges of Surgeons examination (MRCS) has not been investigated. Understanding this relationship is vital to achieving alignment across undergraduate and postgraduate training, learning and assessment values.

Design and participants

A retrospective longitudinal cohort study of UK medical graduates who attempted MRCS Part A (n=9730) and MRCS Part B (n=4645) between 2007 and 2017, using individual-level linked sociodemographic and prior academic attainment data from the UK Medical Education Database.

Methods

We studied MRCS performance across all UK medical schools and examined relationships between potential predictors and MRCS performance using χ² analysis. Multivariate logistic regression models identified independent predictors of MRCS success at first attempt.

Results

MRCS pass rates differed significantly between individual medical schools (p<0.001) but not after adjusting for prior A-Level performance. Candidates from courses other than those described as problem-based learning (PBL) were 53% more likely to pass MRCS Part A (OR 1.53 (95% CI 1.25 to 1.87) and 54% more likely to pass Part B (OR 1.54 (1.05 to 2.25)) at first attempt after adjusting for prior academic performance. Attending a Standard-Entry 5-year medicine programme, having no prior degree and attending a Russell Group university were independent predictors of MRCS success in regression models (p<0.05).

Conclusions

There are significant differences in MRCS performance between medical schools. However, this variation is largely due to individual factors such as academic ability, rather than medical school factors. This study also highlights group level attainment differences that warrant further investigation to ensure equity within medical training.

Correspondence to Mr Ricky Ellis; [email protected]

Strengths and limitations of this study

• This is the first study to explore differences in Membership of the Royal Colleges of Surgeons (MRCS) performance between medical school course types, pedagogy and indicators of institutional esteem.

• It is a large-scale longitudinal cohort study using the UK Medical Education Database.

• The outcome measure of pass/fail at the MRCS examination may hide institutional differences in performance at the question level.

• A-Levels were used as a marker of prior academic attainment in this study, which does not represent the full range of school-leaving examinations used across the UK.

• A larger sample would enable a more granular look at group-level differential attainment.

Background

Medical schools vary significantly in their teaching methodology, curriculum, course structure, assessment methods and standards.^1–4 In the UK, the General Medical Council (GMC) acknowledged that these differences between medical schools exist and that it is ‘inevitable’ that this variation can influence a graduate’s ‘interests, abilities and career progression’ but that it is not a ‘cause for concern’,⁵ presumably because all new medical graduates must meet the same GMC standards. This view can be debated given that medical school seems to influence career progression, direction and success. For example, the number of graduates choosing each specialty differs significantly across medical schools.^6–8 There is significant variation in preparedness for practice, progression through Annual Reviews of Competency Progression in UK training programmes and fitness to practice sanctions according to the medical school of primary qualification.^{5 9} There are also significant differences in the performance of graduates from different medical schools on high-stakes postgraduate examinations such as the Fellowship of the Royal College of Anaesthetists (FRCA),¹⁰ Membership of the Royal College of Obstetricians and Gynaecologists (MRCOG),¹¹ Membership of the Royal College of Paediatrics and Child Health,¹² Membership of the Royal College of General Practitioners (MRCGP)^{13 14} and Membership of the Royal College of Physician (MRCP).^14–16 This variation in performance is far from unique to the UK, with significant differences in performance according to medical school also found in postgraduate assessments in other countries such as the USA.^{17 18} However, to our knowledge, no studies have yet demonstrated whether success at postgraduate surgical examinations differs according to medical school, course type or medical school indicators of esteem (eg, institutional ranking) in the UK.

Understanding the relationship between medical school, course type and pedagogy with markers of postgraduate success is vital for the optimisation of undergraduate teaching by enabling the alignment of undergraduate and postgraduate curricula and assessment values. This alignment ensures best educational practices and the optimisation of training to produce safe and prepared doctors.

The Intercollegiate Membership of the Royal Colleges of Surgeons examination (MRCS) is a high-stakes postgraduate examination, highly valued in the UK as a gatekeeper to the surgical profession.¹⁹ Success at MRCS is associated with success in surgical training, national selection for higher specialty training and first attempt success in the Fellowship of the Royal College of Surgeons examinations (FRCS) and can therefore be used as an indicative marker of future outcomes in a surgical career.^20–22 Success in this examination can be used by medical schools in the alignment of training and assessment values, and students who wish to pursue surgery as a specialty may want to know which medical school will ‘best’ prepare them for a surgical career.²³

In this study, we aimed to evaluate whether medical school of primary qualification or medical course type influence MRCS success. We aimed to establish this by the comparison of first attempt pass rates for MRCS across all UK medical schools and understanding the likelihood of passing MRCS based on university, course type and course pedagogy. Additionally, we aimed to investigate whether indicators of esteem such as Russell Group membership and institutional national ranking predict MRCS success.

Moreover, in order to understand the true impact of medical school differences on MRCS performance we adjusted analyses for prior academic attainment and sociodemographic factors that are known to predict MRCS success.^{24 25} Previous studies have found that after adjusting for these demographic factors (gender, maturity and ethnicity), variation in early surgical training experiences in the UK (Foundation and Core Surgical Training) has little impact on MRCS success.^{26 27} Prior academic attainment is known to be the strongest predictor of later success in medical education,^{20 28 29} and at MRCS.^{24 25 30} Given that some universities are more competitive at entry than others,^{30 31} it is likely that some medical schools recruit the highest performing candidates. As such, both factors are, adjusted for in our analyses.

Methods

This was a longitudinal retrospective cohort study. Individual-level linked data was obtained from the UK Medical Education Database (UKMED)³² and the four Royal Colleges of Surgeons of the UK and Ireland (Edinburgh, Glasgow, England and Ireland). The UKMED database contains background sociodemographic details and assessment results from school to postgraduate examinations and career progression data from combined sources linked at an individual level for all UK medical students and doctors in training.³² This novel database enables powerful multicentre longitudinal cohort studies by including large study populations with minimal missing data. Anonymised data were extracted from UKMED for all UK medical graduates who had attempted either the Part A or the Part B MRCS examination between 2007 and 2017.

The following data were extracted: place of primary medical qualification, course pedagogy and type, MRCS Part A and B first attempt result, gender, self-declared ethnicity and graduation status at the time of entry to medical school. Gender, ethnicity and graduate status were extracted as these are known predictors of MRCS success.^{24 25} Candidate first attempt results were used as they have been shown to be the best predictor of future performance in postgraduate examinations.^{24 33} These variables are described in more detail below.

Except for place of primary qualification, all variables were dichotomised. Part A and B MRCS performance was categorised as ‘pass’ or ‘fail’ at first attempt. Graduation status was defined as ‘no’ if candidates had not obtained a degree prior to entering medicine and ‘yes’ if they entered as a graduate. Self-declared ethnicity was coded as ‘white’ or ‘non-white’ as per similar studies to enable powered analysis of smaller cohorts, rather than this being an ethical or social decision.^{20 21 34} Course pedagogy was classified as ‘problem-based learning’ or ‘not problem-based learning’. Course type was classified as ‘Graduate-Entry’ (GEM: 4-year accelerated Graduate-Entry Medicine programmes) or ‘Undergraduate’ which was later further classified into ‘Standard-Entry’ programme (SEM) or ‘Medicine with a Gateway Year’ (5 years plus one preparatory year). Note that foundation year students were combined with gateway students for this last category, as both approaches have the aim of widening access to medicine; that is, providing alternative ways into medicine for those who do not meet the academic criteria for SEM courses because of socioeconomic or personal disadvantage.³⁵

Finally, there are a significant number of graduates who choose to do an SEM programme,³⁶ so candidates who undertook SEM courses were further defined as ‘Graduate on entry’ or ‘Not graduate on entry’.

Medical schools

At the time of this study, there were 35 medical schools in the UK recognised by the GMC, including a combined University of London awarding body. Most are undergraduate courses, offering a 5-year programme, plus 16 accelerated graduate entry programmes. Eleven medical schools offer gateway/foundation courses. The study-specific dataset included values for 31 medical schools: newer medical schools (eg, Lancaster, Anglia Ruskin and The University of Buckingham) were not represented in the dataset as very few if any of their graduates had attempted MRCS within the study period. Several GEM courses included in the analysis have since ceased to exist (such as Leicester and Bristol), additionally, new GEM and Gateway courses were not included if graduates of these courses had not attempted the MRCS within the study period.

Within the UK, a number of universities combine to create linked medical schools such as Leicester-Warwick Medical School (a combination of the Universities of Leicester and Warwick) and Peninsula Medical School (a combination of Plymouth and Exeter Universities). Many later cease their partnership, creating two independent medical schools. To represent this in the data analysis candidates who studied at either Leicester-Warwick or Peninsula Medical Schools were categorised according to the university from which they graduated (ie, Leicester, Warwick, Plymouth or Exeter). Graduates of Hull-York Medical School and Brighton and Sussex Medical School remain under the combined title as they were still combined institutions at the time of data analysis. Within the study period certain medical schools were also linked (eg, Keele students were awarded degrees by the University of Manchester until 2012). To acknowledge this, students were categorised by the place of graduation for their primary medical qualification, including London graduates.

Indicators of esteem: rankings

In this study, universities were ordered according to their ranking by ‘The Complete University Guide’ as of August 2020. ‘The Complete University Guide’ is the most well recognised independent university ranking system in the UK and uses the following data annually to create an overall score (100 points being the most a university can be awarded): entry standards, student satisfaction, research quality and intensity, graduate prospects, student to staff ratio, spending, honours and degree completion. More information on how the ranking system is calculated is available on the complete university guide website.³¹ This ranking system provides a quantitative comparator between universities for this study and its use does not suggest that its value is greater than that of any other ranking systems that exist which are calculated using similar data. Note that Lancaster University (ranked 16th) was excluded having only opened in 2006 and having insufficient outcome data. St Andrews Medical School (ranked 25th) was also excluded as it offers only preclinical education: those who commenced their studies at St Andrews were therefore categorised by their place of graduation (eg, Manchester University, The University of Dundee, etc). The ranking table was adjusted accordingly, to create an ordinal variable.

Indicators of esteem: Russell Group

Russell Group Institutions are a collection of self-selected research-driven universities that have developed a reputation of excellence.³⁷ Most older medical schools are associated with the Russell Group. Whether these universities are truly the elite institutions within the UK is a highly debated topic^38–40 but they do graduate the majority (80%) of the UK medical students.

Pedagogy

Despite well-established definitions of what comprises PBL it can be challenging to identify which medical schools run PBL courses.^{41 42} We have aligned our definition with that of the British Medical Association as well as that used in recent studies to ensure consistency within the literature, enabling comparisons to be drawn between the results of these studies.^{1 15 43} PBL schools are: Liverpool, Manchester, Glasgow, Queen Mary, Cardiff, Plymouth, Exeter, Sheffield, Keele, Hull-York and East Anglia.

Markers of prior academic attainment

Individual-level linked performance data was extracted for A-Levels as a marker of prior academic attainment. A-Levels are taken as school exit examinations in the majority of schools in England and in some schools elsewhere in the UK. A-Level results are routinely used as a medical school selection metric.³⁰ Total A-Level scores used in data analyses are the sum of all A-Level scores achieved that is, A=10 (being the highest score achievable for each A-Level), B=8, C=6, D=4, E=2, U=0 (being the lowest score for each A-Level). A small minority of candidates in the dataset (n=30) undertook A-Levels after A* grades were implemented in 2010. These were subsequently excluded for cohort homogeneity.

MRCS examination background

The examination comprises two parts: Part A, the written component made up of two multiple-choice questionnaire tests and Part B, a clinical examination that includes 18 Objective Structured Clinical Examination stations.⁴⁴ Taken during Foundation and Core Surgical Training, both MRCS Part A and Part B must now be passed to enable the progression of trainees into higher surgical specialty (residency) training.⁴⁵

Statistical analysis

All analyses were conducted using SPSS V.22.0 (IBM). χ² tests were used to assess the relationship between two categorical factors such as medical school and first attempt MRCS pass/fail outcomes.

All counts have been rounded to the nearest five for illustration according to Higher Education Statistics Agency data standards.⁴⁶ Regression models were used to calculate the ORs and 95% CI for passing MRCS Parts A and B at first attempt according to place of primary medical qualification. The University of Keele was declared the reference category for construction of the logistic regression model for MRCS Part A, as the pass rate at this university (58.6%) most closely resembled the pass rate of the entire cohort of Part A candidates from all universities. The University of Birmingham was declared the reference category for Part B in the logistic regression model, as the pass rate at this university (71.1%) most closely resembled the pass rate of the entire cohort of Part B candidates from all universities.

Potential independent predictors of first attempt success at Part A and B MRCS were identified using multivariate logistic regression models. Regression models were constructed using backward stepwise regression with and without adjustment for prior academic attainment (A-Level performance) for direct comparison.⁴⁷ Any variable (sociodemographic factor, course type, teaching methodology or marker of institutional esteem) with an association with the outcome at a conservative p<0.10 on univariate analysis was entered into the logistic regression model. All potential predictors with p>0.05 in the full model were subsequently removed until only statistically significant predictors remained in the final model. Potential interactions between the remaining significant predictors were also examined.

Data management

The highest standards of security, governance and confidentiality were ensured when storing handling and analysing data. See later for details of ethics approval.

Patient and public involvement

No patients or members of the public were involved in this study.

Results

Medical school differences

Between 2007 and 2017 a total of 9730 UK medical graduates from 31 medical schools attempted the MRCS Part A, with 59% (SD 49) passing on the first attempt. A total of 4645 candidates attempted MRCS Part B and 71% (SD 45) passed at their first attempt. Of all Part A examination candidates 64% were male, 59% were white and 86% had no degree-level qualification prior to studying medicine. Similar demographics were seen in Part B applicants with 65% male candidates, 61% white candidates and 86% of candidates having no prior degree. χ² analysis revealed a significant difference in MRCS pass rates between medical schools for Part A (p<0.001) and Part B (p<0.001). Figure 1 shows MRCS Part A first attempt pass rates by medical school and figure 2 shows MRCS Part B first attempt pass rates by medical school. Raw data are presented in online supplemental appendix 1.

Figure 1. MRCS Part A first attempt pass rates by medical school with 95% CI. MRCS, Membership of the Royal Colleges of Surgeons.

Figure 2. MRCS Part B first attempt pass rates by medical school with 95% CI. Swansea University Part B results excluded due to small cohort (n=2). MRCS, Membership of the Royal Colleges of Surgeons.

Medical school ranking and position of esteem

ORs for passing MRCS Part A and B at the first attempt for each medical school can be found in table 1. Oxford and Cambridge University graduates (ranked first and second, respectively) performed significantly better in MRCS Part A than the mean with resulting OR of 9.11 (95% CI 4.77 to 17.39) and 5.82 (3.42 to 9.90), respectively. After adjusting for prior academic attainment, Oxford University graduates were still found to be more than three times more likely to pass MRCS Part A at first attempt (OR 3.18 (95% CI 1.15 to 8.81)) and Cambridge graduates were more than two times as likely to pass (OR 2.64 (95% CI 1.03 to 6.78)). After adjusting for prior academic attainment, no medical schools were found to be statistically significant predictors of MRCS Part B first-attempt success and there was no statistically significant difference in MRCS performance between most medical schools.

Table 1

OR for pass at first attempt at Part A and Part B MRCS across all UK medical schools ranked according to the Complete University Guide as of 2019–2020

Rank	Medical school	Part A			Part B
		MRCS candidates (n=9730)	Unadjusted OR (95% CI)	OR adjusted for prior academic attainment (95% CI)	MRCS candidates (n=4645)	Unadjusted OR (95% CI)	OR adjusted for prior academic attainment (95% CI)
1	University of Oxford	210	9.11 (4.77 to 17.39)	3.18 (1.15 to 8.81)	94	2.32 (1.23 to 4.40)	4.43 (0.51 to 38.58)
2	University of Cambridge	285	5.82 (3.42 to 9.90)	2.64 (1.03 to 6.78)	142	1.52 (0.92 to 2.50)	3.92 (0.77 to 19.82)
3	University of Glasgow	350	0.96 (0.62 to 1.47)	0.51 (0.18 to 1.39)	169	1.40 (0.88 to 2.23)	1.11 (0.20 to 6.09)
4	Swansea University	15	0.28 (0.08 to 0.96)	0.90 (0.83 to 0.97)	0	– –	– –
5	The University of Edinburgh	365	1.76 (1.13 to 2.74)	2.01 (0.81 to 5.00)	190	1.40 (0.89 to 2.19)	0.56 (0.19 to 1.62)
6	University of Dundee	215	0.56 (0.35 to 0.89)	0.73 (0.26 to 2.05)	105	0.95 (0.57 to 1.58)	0.42 (0.14 to 1.32)
7	Imperial College London	815	2.05 (1.36 to 3.08)	1.26 (0.58 to 2.75)	415	1.06 (0.73 to 1.52)	1.22 (0.47 to 3.20)
8	Queen Mary University of London	475	0.44 (0.29 to 0.67)	0.45 (0.19 to 1.04)	210	0.41 (0.28 to 0.61)	0.38 (0.14 to 1.01)
9	Keele University	110	– –	– –	70	1.13 (0.61 to 2.09)	0.61 (0.17 to 2.17)
10	University of Exeter	70	0.52 (0.28 to 0.95)	0.38 (0.13 to 1.07)	35	1.32 (0.57 to 3.08)	2.24 (0.25 to 20.12)
11	University of Aberdeen	230	0.68 (0.43 to 1.07)	0.39 (0.12 to 1.29)	105	0.94 (0.57 to 1.56)	0.28 (0.04 to 2.23)
12	University of Bristol	355	1.58 (1.02 to 2.46)	0.66 (0.28 to 1.52)	170	1.27 (0.80 to 2.01)	0.83 (0.24 to 2.86)
13	University College London	575	1.53 (1.02 to 2.33)	1.25 (0.55 to 2.82)	275	0.84 (0.57 to 1.24)	1.11 (0.37 to 3.31)
14	Newcastle University	390	0.81 (0.53 to 1.24)	0.59 (0.26 to 1.32)	200	1.01 (0.66 to 1.54)	1.44 (0.50 to 4.17)
15	Cardiff University	390	1.10 (0.72 to 1.69)	0.79 (0.35 to 1.78)	180	1.13 (0.72 to 1.75)	1.34 (0.44 to 4.14)
16	King’s College London	665	0.94 (0.62 to 1.41)	0.63 (0.29 to 1.38)	305	0.97 (0.66 to 1.42)	1.31 (0.45 to 3.84)
17	The University of Sheffield	285	0.62 (0.40 to 0.97)	0.82 (0.34 to 2.00)	145	0.74 (0.47 to 1.16)	0.43 (0.15 to 1.30)
18	University of Leeds	275	0.84 (0.54 to 1.32)	0.67 (0.28 to 1.64)	130	2.01 (1.17 to 3.47)	2.63 (0.51 to 13.58)
19	University of Plymouth	70	0.50 (0.27 to 0.92)	0.63 (0.23 to 1.70)	35	0.45 (0.22 to 0.93)	0.39 (0.10 to 1.50)
20	University of East Anglia	110	0.37 (0.22 to 0.64)	0.44 (0.17 to 1.14)	45	0.57 (0.29 to 1.11)	1.54 (0.27 to 8.73)
21	Brighton and Sussex Medical School	90	0.65 (0.37 to 1.13)	1.10 (0.35 to 3.44)	45	0.94 (0.46 to 1.92)	0.35 (0.08 to 1.57)
22	Queen’s University Belfast	245	0.84 (0.53 to 1.32)	0.49 (0.21 to 1.15)	115	0.88 (0.54 to 1.44)	0.80 (0.25 to 2.56)
23	University of Nottingham	465	1.44 (0.94 to 2.21)	0.92 (0.41 to 2.07)	235	1.40 (0.91 to 2.13)	2.03 (0.63 to 6.54)
24	The University of Manchester	580	0.72 (0.47 to 1.08)	0.58 (0.26 to 1.28)	275	0.96 (0.65 to 1.41)	0.78 (0.29 to 2.09)
25	Hull York Medical School	85	0.60 (0.34 to 1.06)	0.79 (0.25 to 2.50)	40	0.92 (0.44 to 1.92)	1.30 (0.11 to 16.01)
26	University of Birmingham	480	1.26 (0.83 to 1.93)	1.08 (0.48 to 2.41)	220	– –	– –
27	University of Warwick	160	0.78 (0.48 to 1.27)	2.08 (0.16 to 27.09)	70	0.66 (0.38 to 1.16)	0.80 (0.69 to 1.02)
28	University of Leicester	275	1.07 (0.68 to 1.67)	0.80 (0.33 to 1.94)	130	0.81 (0.51 to 1.30)	0.54 (0.17 to 1.72)
29	University of Southampton	310	0.81 (0.52 to 1.26)	0.63 (0.27 to 1.45)	140	0.76 (0.48 to 1.20)	0.56 (0.19 to 1.61)
30	University of Liverpool	365	0.60 (0.39 to 0.92)	0.66 (0.30 to 1.47)	160	1.02 (0.65 to 1.59)	1.01 (0.35 to 2.88)
31	St George’s University of London	430	0.73 (0.48 to 1.12)	0.73 (0.32 to 1.62)	200	0.84 (0.55 to 1.27)	0.46 (0.18 to 1.13)

Statistically significant ORs are shown in bold.

MRCS, Membership of the Royal Colleges of Surgeons.

There was a significant difference in MRCS Part A pass rates between candidates from Russell Group universities (60.7% (4970/8185)) and non-Russell Group universities (49.9% (770/1540)) p<0.001 (table 2). Similarly, a significant difference was seen in Part B of the examination with a pass rate of 71.4% (2790/3910) for Russell Group universities and 67.5% (495/735) for non-Russell Group universities p=0.038.

Table 2

MRCS first attempt pass rates by course type and prior degree status

Predictor	Part A (n=9730)	Part B (n=4645)
Russell Group
Yes	60.7% (4970/8185)	71.4% (2790/3910)
No	49.9% (770/1540)	67.5% (495/735)
Missing	n=0	n=0
P value	<0.001	0.038
Course
Undergraduate	59.3% (5305/8950)	71.0% (3050/4300)
Graduate-entry	54.6% (405/745)	69.3% (230/335)
Missing	n=35	n=10
P value	0.012	0.533
Undergraduate course classification
Standard-Entry Medicine	60.0% (5255/8755)	71.1% (3010/4230)
Medicine with Gateway Year	28.1% (55/190)	60.9% (40/70)
Missing	n=0	n=0
P value	<0.001	0.081
Prior degree status on undergraduate courses
Not graduate on entry	60.2% (4945/8220)	71.5% (2830/3960)
Graduate on entry	49.5% (360/730)	65.0% (220/335)
Missing	n=0	n=0
P value	<0.001	0.015
Graduate student outcomes
Graduate on Standard-Entry course	49.5% (360/730)	65.0% (220/335)
Graduate on Graduate-Entry course	54.6% (405/745)	69.3% (230/335)
Missing	n=0	n=0
P value	0.054	0.251
Teaching methodology
Not problem-based learning	63.1% (4560/7225)	72.2% (2505/3465)
Problem based learning	47.0% (1175/2500)	66.6% (785/1180)
Missing	n=0	n=0
P value	<0.001	<0.001

All p values presented are from χ² analysis.

MRCS, Membership of the Royal College of Surgeons.

Course type

Univariate analysis of pass rates by course type is displayed in table 2. The majority of all MRCS Part A candidates had studied a Standard-Entry Medicine (SEM) course (8950/9730): only 745 candidates had graduated from a GEM course. There was a significant difference between Part A pass rates of SEM (59.3%) and GEM graduates (54.6%) p=0.012. Of the 335 graduates who attempted Part B, 69.3% passed first time, and there was no statistically significant difference in MRCS Part B pass rates between SEM and GEM candidates (p=0.533).

A small proportion of the trainees attempting MRCS Part A who had studied an SEM course (n=8950) entered medicine as graduates (n=730). There was a significant difference in MRCS Part A success between those entering without a prior degree 60.2% (4945/8220) and graduates 49.5% (360/730) from SEM courses, p<0.001. Similar results were found for MRCS Part B (71.5% (2830/3960) vs 65.0% (220/335), respectively p<0.001).

Table 2 shows that of all candidates who attended an SEM, 190 entered their course via a ‘Gateway Year’. A statistically significant difference was seen in MRCS Part A pass rates between students who undertook a Gateway Year (28.1%) and those who entered directly into a Standard-Entry course (60.0%) p<0.001. There was a difference in MRCS Part B pass rates between Gateway students (60.9% (40/70)) and direct-entry students (71.1% (3010/4230)) but this was not statistically significant (p=0.081).

Of all graduates from SEM courses, 49.5% passed Part A first time compared with 54.6% of graduates from GEM courses (p=0.054). Similarly, 65% of SEM graduates passed Part B first time compared with 69.3% of GEM graduates (p=0.251).

Course pedagogy

A significant difference was observed in MRCS Part A first attempt pass rates between candidates who studied on a course described as PBL and those who studied at medical schools with other core pedagogies (47.0% (1175/2505) vs 63.1% (4560/7225) p<0.001 (table 2)). A similar difference was observed in Part B of the MRCS (PBL: 66.6% (785/1180) and non-PBL: 72.2% (2505/3465) p<0.001).

A comparison of MRCS pass rates between GEM courses can also be found in table 3. There was a significant difference in pass rates between GEM schools for MRCS Part A (p=0.028) but not for MRCS Part B (p=0.072). Drilling down further highlights that the aggregate data disguise variation. For example, graduates of the King’s College London GEM programme performed above average (eg, 76.7% Part A and 81.0% Part B pass rates; table 3) but the MRCS performance of candidates from their undergraduate programme was lower than average (57% Part A and 70.5% Part B, figure 1).

Table 3

MRCS first attempt pass rates by Graduate-Entry Medicine course

Medical school	Part A			Part B
	Total number of candidates (n=745)	Pass rate (%) (n=405)	95% CI	Total number of candidates (n=335)	Pass rate (%) (n=230)	95% CI
The University of Oxford	5	100.0	100.0 to 100.0	5	100.0	100.0 to 100.0
The University of Cambridge	25	80.0	63.1 to 96.9	10	40.0	0.31 to 76.9
The University of Swansea	15	28.6	1.5 to 55.6	0	–	–
Imperial College London	25	51.9	31.7 to 72.0	10	60.0	23.1 to 96.9
Queen Mary University of London	75	51.3	39.8 to 62.8	35	58.8	41.4 to 76.3
The University of Bristol	25	60.9	39.3 to 82.4	10	72.7	41.3 to 100.0
University of Newcastle-upon-Tyne	35	63.6%	46.3 to 81.0	20	85.7	69.4 to 100.0
Cardiff University	50	51.0%	36.8 to 65.2	20	84.2	66.2 to 100.0
King’s College London	30	76.7%	60.6 to 92.7	20	81.0	62.6 to 99.3
University of Nottingham	95	56.3%	46.1 to 66.4	45	59.6	45.0 to 74.1
The University of Birmingham	30	50.0%	31.0 to 69.0	15	88.2	71.2 to 100.0
The University of Warwick	160	52.5%	44.7 to 60.2	70	62.0	50.4 to 73.5
The University of Leicester	40	47.6%	31.9 to 63.4	15	78.6	54.0 to 100.0
The University of Southampton	25	52.0%	31.0 to 73.0	10	77.8	43.9 to 100.0
The University of Liverpool	20	45.0%	21.1 to 68.9	15	84.6	61.9 to 100.0
St George’s Medical School London	85	50.0%	39.2 to 60.8	35	66.7	49.7 to 83.6

All values presented from χ² analysis; Part A 27.12 p=0.028 and Part B 23.59 p=0.72.

MRCS, Membership of the Royal College of Surgeons.

Sociodemographic factors

Pass rates for MRCS Parts A and B by graduate on entry to medicine status, gender and ethnicity are shown in table 4. Non-graduates, males and individuals of white ethnicity had significantly higher pass rates for MRCS Parts A and B compared with their graduate, female and non-white ethnicity counterparts.

Table 4

MRCS first attempt pass rates by gender, ethnicity and graduation status for UK medical graduates

Predictor	Part A (n=9730)	Part B (n=4645)
Graduate on entry to medicine
No	60.2% (4945/8220)	71.5% (2830/3960)
Yes	52.4% (790/1510)	66.8% (455/685)
Missing	n=0	n=0
P value	<0.001	0.014
Gender
Male	62.5% (3865/6185)	72.1% (2185/3030)
Female	52.8% (1870/3545)	68.3% (1100/1615)
Missing	n=0	n=0
P value	<0.001	0.007
Ethnicity
White	63.7% (3580/5615)	76.6% (2130/2780)
Non-white	52.3% (2055/3930)	62.5% (1120/1790)
Missing	n=180	n=75
P value	<0.001	<0.001

All p values presented are from χ² analysis.

MRCS, Membership of the Royal College of Surgeons.

Multivariate analysis

The multivariate logistic regression models showing independent predictors of success at MRCS Part A and MRCS Part B can be found in table 5. After adjusting for prior academic attainment, white candidates, males and those who studied medicine without a prior degree-level qualification were all significantly more likely to pass MRCS Part A at the first attempt (p<0.05). After adjusting for prior attainment, white ethnicity remains a statistically significant predictor of Part B success (p<0.05), although gender and graduate status were not independent predictors of Part B success.

Table 5

Predictors of pass at first attempt at MRCS Part A (n=5735) and Part B (n=3285) for UK medical graduates

Predictor	Part A		Part B
	Unadjusted OR (95% CI)	Adjusted OR (95% CI)	Unadjusted OR (95% CI)	Adjusted OR (95% CI)
Graduate on entry into medicine
Non-graduates vs graduates	1.4 (1.19 to 1.64)	2.86 (1.00 to 8.16)	1.66 (1.24 to 2.24)	2.08 (0.74 to 5.88)
Gender
Males vs females	1.66 (1.48 to 1.88)	1.62 (1.34 to 1.95)	1.25 (1.09 to 1.44)	1.23 (0.86 to 1.77)
Ethnicity
White vs non-white	1.65 (1.46 to 1.87)	1.4 (1.17 to 1.68)	2.06 (1.80 to 2.36)	2.07 (1.46 to 2.93)
Russell Group
Russel Group vs non-Russell Group	1.79 (1.56 to 2.05)	1.14 (0.88 to 1.48)	1.24 (1.03 to 1.49)	1.81 (1.17 to 2.80)
Undergraduate course type
Standard-Entry vs Gateway Year	3.72 (2.69 to 5.15)	2.34 (1.21 to 4.52)	1.67 (1.02 to 2.76)	2.53 (0.89 to 7.17)
Teaching methodology
Not PBL vs PBL	1.99 (1.74 to 2.27)	1.53 (1.25 to 1.87)	1.49 (1.27 to 1.75)	1.54 (1.05 to 2.25)

P=0.034 for interaction between ethnicity and gender, P=0.001 for ethnicity and teaching methodology, P=0.001 for PBL and Russell group classification in unadjusted Part A regression model and P=0.031 for graduate status and Russell group classification in adjusted Part A model. *P=0.022 for interaction between graduate status and teaching methodology in unadjusted Part B regression model.

ORs (95% CI) given prior to and after adjustment for prior academic attainment.

MRCS, Membership of the Royal College of Surgeons.

Candidates who attended a non-PBL medical school were found to be 53% (OR 1.53 (95% CI 1.25 to 1.87)) more likely to pass Part A and 54% (OR 1.54 (95% CI 1.05 to 2.25)) more likely to pass Part B at the first attempt after adjusting for prior academic performance, compared with those who attended a PBL school. Candidates attending an SEM course were nearly four times more likely to pass Part A at first attempt (OR 3.72 (95% CI 2.69 to 5.15)) and 67% more likely to pass Part B (OR 1.67 (95% CI 1.02 to 2.76)) when compared with those entering SEM via a Gateway Year. After adjusting for prior attainment, SEM candidates were more than twice as likely to pass Part A (OR 2.34 (95% CI 1.21 to 4.52)) but attending an SEM course was not found to be a statistically significant predictor of Part B success.

Candidates who attended a Russell Group university were 79% more likely to pass Part A (OR 1.79 (95% CI 1.56 to 2.05)) and 24% more likely to pass Part B (OR 1.24 (95% CI 1.03 to 1.49)). However, after adjusting for prior academic attainment, attending a Russell Group university was found to predict success at MRCS Part B (OR 1.81 (95% CI 1.17 to 2.80)) but not Part A.

Discussion

This study, the first to examine the variation in pass rates for the MRCS examination across UK medical schools, identified significant differences in pass rates for both MRCS Part A and Part B across schools, course type and pedagogy.

Our most important finding is the lack of statistically significant difference in MRCS success between medical schools after adjusting for A-Levels as a measure of prior academic attainment. This indicates that prior attainment is a significant contributory factor to postgraduate performance between different schools. In other words, differences in postgraduate examination performance are more closely related to individual factors than medical school differences. This reflects patterns seen in other medical assessments.^{11 14 17 20 21 28 48–51}

Institutional esteem is a known pull factor for medical school applicants.^52–54

We found that even after adjusting for prior academic attainment and, by extension, the selection of the highest achieving applicants (see later), both Oxford and Cambridge universities performed significantly better than other academic institutions. These results suggest that the training and education offered by these schools does add value to the likelihood of their student’s later success, over and above the individual’s academic ability.

However, with the exceptions of Oxford and Cambridge, we found little association between MRCS pass rates and medical school rankings. This is perhaps unsurprising given that rankings are based on amalgamated scores,³¹ several of which are not relevant to vocational medical degrees with their high retention and employability rates. Additionally, earlier studies indicated that staff to student ratio and student feedback, two seemingly relevant measures used in university rankings, seem to have no effect on performance in medical graduates.^{15 16} In contrast, Russell Group (research-intensive/focused universities) medical graduates were far more likely to pass MRCS at the first attempt. The relationship between research intensity/focus and MRCS outcomes is unclear. However, it may be that higher entry requirements for Russell Group universities^{55 56} play a role given the strong message from our findings and those of the wider literature that prior academic performance is the strongest predictor of future success.^{14 17 20 21 25 28–30 48–51} Indeed, we would suggest that educational institutions that are self-selecting as an elite group have a self-interest in selecting the very best applicants who will continue to perform at a high level after graduating in order to perpetuate their status as the leading schools.

As per McManus et al’s MedDifs paper (2020),¹⁵ we found that pedagogic differences (PBL vs non-PBL) are related to variation in outcome measures on postgraduate examinations. Graduates from PBL courses perform less well on MRCS A and B. Other literature hints at possible reasons for this. PBL graduates have strengths compared with those from non-PBL courses in some areas,^{57 58} but PBL graduates have reported less surgical teaching than is offered at other medical schools¹⁵ and differences in time dedicated to undergraduate surgical training in UK medical schools has been found to correlate with preparedness for clinical practice in surgery.²³ PBL courses have also been criticised for neglecting basic science content,^{59 60} and this may be a contributing factor in the performance of PBL students at Part A of the MRCS, given that paper 1 (of 2) is dedicated to applied basic sciences.

Gateway courses provide a pathway to medicine for students from more diverse sociodemographic and academic backgrounds.^{61 62} Students from Gateway courses perform less well on assessments during medical school,^{61 63} at Foundation Programme Selection⁶⁴ and, as found in this study. the MRCS. However, there are two points to note. While increasing the diversity of the medical workforce is high on the workforce planning agenda,⁶⁵ the actual number of Gateway programme graduates in our analysis was very small (n=190). This suggests that surgery is not a common career pathway for these students. Why this is the case is unknown but it may be related to myriad factors including high competition for surgical training posts,⁶⁶ a lack of perceived ‘fit’ with surgery, few visible role models from similar backgrounds in senior surgical roles, and/or a greater preference to choose a medical career which enables them to give back to under-served communities.^{67 68} Future research is required to examine this further.

Despite the performance of those who entered medical school as graduates being comparable to those who entered as undergraduates throughout medical school^{69 70} and on graduation,⁶³ there remains a significant attainment difference between these groups on postgraduate specialty examinations.^{20 71 72} Our analysis suggests that this is not due to course type (GEM or SEM). Further work is required to ascertain whether graduates are disadvantaged in postgraduate training due to other factors, such as increased commitments on their time (eg, family, dependants and financial obligations)⁷² or whether this is a reflection of lower prior academic achievement.^{56 73}

Implications for research, policy and practice

Much literature indicates that medical school influences the progression, direction and performance of their graduates.^{5–7 9–13 15 16 74} However, it is reassuring to find that the majority of this variation in performance between schools on the MRCS at least can be accounted for by individual factors, namely prior academic attainment. There were, however, clear differences in performance by course pedagogy and markers of institutional esteem which can be used by medical schools to optimise the alignment between undergraduate and postgraduate teaching, learning and assessment values in surgery, and by individuals when considering where to apply to study medicine.

These findings are relevant to medical school selection. In the UK, the first and major hurdle to entry into medicine is achieving high grades on school exit examinations (such as A-Levels). This is usually coupled with an aptitude test and, if an applicant reaches the required standard on these measures, an interview to assess non-cognitive (personal) qualities.⁷⁵ There has been much debate in the selection literature as to the weight which should be placed on each of these selection components.⁷⁶ Our data suggest that if a medical school wants to graduate doctors who are good at passing postgraduate exams, then prior academic attainment should be heavily weighted at the point of selection.

However, if the mission of medical schools is to graduate doctors who will, for example, meet social accountability mandates, then more holistic selection criteria are required.⁷⁷ Moreover, there are other factors potentially influencing postgraduate success which we could not take into account: group factors (eg, factors related to the demographics of the student group)⁷⁸; individual career preferences¹⁶ and prior schooling⁷⁹; mentorship and research opportunities⁸⁰ and a student’s overall experience of a specialty.⁷⁴ We are unlikely to ever characterise all variables that contribute to postgraduate examination success, but this study goes some way to identifying key patterns.

In addition to variation in MRCS pass rates, there is also significant variation in the number of graduates from each medical school entering careers in surgery.^{6 52} Students who wish to pursue surgery as a specialty may want to know which medical school will ‘best’ prepare them for a surgical career.²³ Many students enter medicine with clear views as to which specialty they wish to pursue.^{52 81 82} Perceptions of how well an individual will be placed for a surgical career on graduation may be one factor that is taken into account at the time of application to medical school.⁸³ However, it will not be the only factor. Studies indicate that numerous factors are ‘traded-off’ when considering training location and these trade-offs differ for different groups (eg, on the basis of gender or socioeconomic background).^{84 85} Similarly, applicants may consider factors such as pedagogic approach (eg, PBL vs, for example, or a lecture-based course)^86–88; course length if a graduate (graduates have the choice between a traditional 5year programme or an accelerated GEM course⁸⁹); and/or the reputation and national ranking of a medical school when considering where to apply.^{52–54 90} In short, choosing which medical school to attend is a major decision and factors other than career preference may be important in this process.

Differential attainment

Group differences in performance by gender, maturity and ethnicity reflect those seen in previous studies.^{20 24} These attainment differences have also been identified in other high-stakes medical examinations, including FRCS, MRCP, MRCGP, MRCPsych and the USMLE.^{20 34 48 91–93} Research that aims to investigate this differential attainment at MRCS is currently ongoing. Bias and discrimination at the question level must be ruled out using techniques such as differential item functioning analysis,⁹⁴ as should the possibility of examiner bias.^{95 96} The wider literature also suggests the need to examine systemic inequities in the workplace learning environment.⁹⁷

Strengths and limitations

To our knowledge, this large cohort study is the first to assess the relationship between MRCS success and medical school choice, type and ranking after adjusting for measures of prior academic attainment. The UKMED dataset enabled a large-scale, multi-cohort analysis of medical school differences on MRCS first attempt outcomes. The dataset had very little missing data enabling detailed and accurate analyses, demonstrating the utility of national medical education databases. We used candidate first attempt scores despite candidates being able to take multiple attempts at both parts of the MRCS, as first attempt performance in postgraduate examinations has been shown to be the best predictor of future performance³³ and this outcome has been used in previous studies looking at factors which predict performance in the MRCS.^{20 24} The outcome measure of pass/fail was used as in previous studies since this is what is meaningful to those sitting MRCS.^{24 25 98} Data were not available for individual MRCS questions and stations potentially hiding institutional differences in performance.

A-Levels were used as a marker of prior academic attainment in this study. This does not represent the full range of school-leaving examinations used by all UK schools (others include Irish and Scottish Highers and the International Baccalaureate). However, A-Levels have been used previously as markers of prior academic attainment in seminal medical education papers and we have no reason to believe that other school-leaving examinations would show different results.^{28 29} The strengths and limitations of using markers of prior academic attainment such as A-Levels in high achieving cohorts such as doctors are discussed in these papers and in our previous work.³⁰

Despite a long study period and a large study population; stratification of the analysis by medical school results in smaller cohort numbers (and therefore reduced statistical power) for comparison. Smaller cohort numbers and lower numbers of actual observations in some subanalyses may result in overfitting, affecting the predictive ability of regression models. Larger cohort sizes would have enabled a more detailed analysis of group differences such as self-declared ethnicity data, avoiding the need for the binary categorisation used here which ensured maximum statistical power.^{97 99}

Stage of training is known to have an impact on MRCS performance, with those who attempt the examination earlier in their training generally performing better than their peers.²⁴ Without access to stage of training data for the first attempt at MRCS, we were unable to adjust for this variable in the analyses. Stage of training could be extrapolated using the date of graduation, however, given that over half of UK doctors take at least 1 year out of training after the Foundation programme, this would introduce a significant degree of inaccuracy to the analyses. Similarly, we were unable to adjust for degree intercalation. Those who undertake an intercalated degree are known to perform better in later medical school examinations, which is to be expected, given that entry to intercalation programmes is competitive.¹⁰⁰ It is therefore likely that this group will continue to be top performers in postgraduate assessments, given prior academic attainment is the best predictor of later success.²⁸ Additionally, very few intercalating students will be graduates on entry to medicine and are therefore unlikely to experience the same burden of time, financial and caring commitments as graduates. The impact of intercalating on markers of postgraduate performance across all specialties would be best assessed in a separate study. This would be particularly relevant given the recent removal of points scored for undergraduate degrees in UK Foundation Programme selection measures, which has started a debate regarding the future merit of intercalating.

Analysis that includes multiple sociodemographic and course factors inevitably includes a degree of multicollinearity, although every effort was made to minimise this. Interaction terms were explored and statistically significant interactions are listed in the footnote of table 5. These highlight differences in cohort sociodemographics between each teaching methodology and course type. Further exploration of these differences may be of interest to those in charge of selection and recruitment for medical school. Courses change over time and as such results and attainment differences may also have changed throughout the study period: future studies may wish to use a time-series analysis to look at this.⁷⁶

Conclusion

There are significant differences in MRCS performance between UK medical school course types and pedagogy. However, variation in MRCS pass rates between medical schools is largely due to individual factors, such as the academic ability of individuals, rather than medical school factors. This data has implications for those in charge of selection policy and curricula delivery. This study also highlights group level attainment differences that transcend training location and stage, warranting further investigation to ensure equity within medical training.

The authors would like to acknowledge Iain Targett at the Royal College of Surgeons of England, for his help with data collection and John Hines and Gregory Ayre from the Intercollegiate Committee for Basic Surgical Examinations for their support during this project. Our thanks to members of the UKMED Research Group who provided useful feedback on an earlier version of this manuscript, and whose comments helped refine the paper. The authors would also like to acknowledge Daniel Smith for his help with the UKMED database. Data source: UK Medical Education Database ('UKMED'). UKMEDP043 extract generated on 25/07/2018. We are grateful to UKMED for the use of these data. However, UKMED bears no responsibility for their analysis or interpretation the data includes information derived from that collected by the Higher Education Statistics Agency Limited ('HESA') and provided to the GMC ('HESA Data'). Source: HESA Student Records 2002/2003 to 2015/2016. Copyright Higher Education Statistics Agency Limited. The Higher Education Statistics Agency Limited makes no warranty as to the accuracy of the HESA Data, cannot accept responsibility for any inferences or conclusions derived by third parties from data or other Information supplied by it.

Data availability statement

Data may be obtained from a third party and are not publicly available. The dataset used in this study was acquired from the UK Medical Education Database and is held in Safe Haven. Data access requests must be made to UKMED. Full information for applications can be found at https://www.ukmed.ac.uk.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

UKMED has received ethics exemption for projects using exclusively UKMED data from Queen Marys University of London Ethics of Research Committee on behalf of all UK medical schools (https://www.ukmed.ac.uk/documents/UKMED_research_projects_ethics_exemption.pdf). The Intercollegiate Committee for Basic Surgical Examinations (ICBSE) and its Internal Quality Assurance Subcommittee, which monitors MRCS standards, research and quality, approved this study.

Footnote

Twitter @RickJEllis1, @dsgscrimgeour

Contributors RE wrote the first draft of the manuscript. RE performed statistical analyses with AL’s supervision. RE, PB, DSGS, AJL and JC all reviewed and edited the manuscript. JC led the study proposal for access to UKMED data. All authors approved the final draft of the manuscript. RE is study guarantor.

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

References

¹ Devine OP, Harborne AC, Horsfall HL, et al. The analysis of teaching of medical schools (atoms) survey: an analysis of 47,258 timetabled teaching events in 25 UK medical schools relating to timing, duration, teaching formats, teaching content, and problem-based learning. BMC Med 2020; 18: 126. doi:10.1186/s12916-020-01571-4 http://www.ncbi.nlm.nih.gov/pubmed/32404194

² Devine OP, Harborne AC, McManus IC. Assessment at UK medical schools varies substantially in volume, type and intensity and correlates with postgraduate attainment. BMC Med Educ 2015; 15: 146. doi:10.1186/s12909-015-0428-9 http://www.ncbi.nlm.nih.gov/pubmed/26362320

³ Taylor CA, Gurnell M, Melville CR, et al. Variation in passing standards for graduation-level knowledge items at UK medical schools. Med Educ 2017; 51: 612–20. doi:10.1111/medu.13240 http://www.ncbi.nlm.nih.gov/pubmed/28295495

⁴ McCrorie P, Boursicot KAM. Variations in medical school graduating examinations in the United Kingdom: are clinical competence standards comparable? Med Teach 2009; 31: 223–9. doi:10.1080/01421590802574581 http://www.ncbi.nlm.nih.gov/pubmed/19288309

⁵ General Medical Council. Be prepared: are new doctors safe to practise? Manchester: general medical Council, 2014. Available: https://www.gmc-uk.org/Be_prepared___are_new_doctors_safe_to_practise_Oct_2014.pdf_58044232.pdf

⁶ Twigg V, Aldridge K, McNally SA, et al. Does choice of medical school affect a student’s likelihood of becoming a surgeon? Bulletin 2018; 100: 90–5. doi:10.1308/rcsbull.2018.90

⁷ Alberti H, Randles HL, Harding A, et al. Exposure of undergraduates to authentic GP teaching and subsequent entry to GP training: a quantitative study of UK medical schools. Br J Gen Pract 2017; 67: e248–52. doi:10.3399/bjgp17X689881 http://www.ncbi.nlm.nih.gov/pubmed/28246097

⁸ Cleland JA, Johnston PW, Anthony M, et al. A survey of factors influencing career preference in new-entrant and exiting medical students from four UK medical schools. BMC Med Educ 2014; 14: 151. doi:10.1186/1472-6920-14-151 http://www.ncbi.nlm.nih.gov/pubmed/25056270

⁹ Goldacre MJ, Taylor K, Lambert TW. Views of junior doctors about whether their medical school prepared them well for work: questionnaire surveys. BMC Med Educ 2010; 10: 78. doi:10.1186/1472-6920-10-78 http://www.ncbi.nlm.nih.gov/pubmed/21070622

¹⁰ Bowhay AR, Watmough SD. An evaluation of the performance in the UK Royal College of anaesthetists primary examination by UK medical school and gender. BMC Med Educ 2009; 9: 38. doi:10.1186/1472-6920-9-38 http://www.ncbi.nlm.nih.gov/pubmed/19563655

¹¹ Rushd S, Landau AB, Khan JA, et al. An analysis of the performance of UK medical graduates in the MRCOG Part 1 and part 2 written examinations. Postgrad Med J 2012; 88: 249–54. doi:10.1136/postgradmedj-2011-130479 http://www.ncbi.nlm.nih.gov/pubmed/22331918

¹² Menzies L, Minson S, Brightwell A, et al. An evaluation of demographic factors affecting performance in a paediatric membership multiple-choice examination. Postgrad Med J 2015; 91: 72–6. doi:10.1136/postgradmedj-2014-132967 http://www.ncbi.nlm.nih.gov/pubmed/25617382

¹³ Wakeford R, Foulkes J, McManus C, et al. MRCGP pass rate by medical school and region of postgraduate training. Royal College of general practitioners. BMJ 1993; 307: 542–3. doi:10.1136/bmj.307.6903.542-a http://www.ncbi.nlm.nih.gov/pubmed/8400976

¹⁴ McManus IC, Wakeford R. PLAB and UK graduates' performance on MRCP(UK) and MRCGP examinations: data linkage study. BMJ 2014; 348: g2621. doi:10.1136/bmj.g2621 http://www.ncbi.nlm.nih.gov/pubmed/24742473

¹⁵ McManus IC, Harborne AC, Horsfall HL, et al. Exploring UK medical school differences: the MedDifs study of selection, teaching, student and F1 perceptions, postgraduate outcomes and fitness to practise. BMC Med 2020; 18: 136. doi:10.1186/s12916-020-01572-3 http://www.ncbi.nlm.nih.gov/pubmed/32404148

¹⁶ McManus IC, Elder AT, de Champlain A, et al. Graduates of different UK medical schools show substantial differences in performance on MRCP(UK) Part 1, Part 2 and PACES examinations. BMC Med 2008; 6: 5. doi:10.1186/1741-7015-6-5 http://www.ncbi.nlm.nih.gov/pubmed/18275598

¹⁷ Hecker K, Violato C. How much do differences in medical schools influence student performance? A longitudinal study employing hierarchical linear modeling. Teach Learn Med 2008; 20: 104–13. doi:10.1080/10401330801991915 http://www.ncbi.nlm.nih.gov/pubmed/18444195

¹⁸ Burk-Rafel J, Pulido RW, Elfanagely Y, et al. Institutional differences in USMLE step 1 and 2 CK performance: cross-sectional study of 89 us allopathic medical schools. PLoS One 2019; 14: e0224675. doi:10.1371/journal.pone.0224675 http://www.ncbi.nlm.nih.gov/pubmed/31682639

¹⁹ Ellis R, Cleland J, Scrimgeour D. Does the MRCS exam fulfil its purpose in surgical professions? Bull R Coll Surg Engl 2021; 103: 344–50. doi:10.1308/rcsbull.2021.127

²⁰ Scrimgeour DSG, Cleland J, Lee AJ, et al. Prediction of success at UK specialty board examinations using the mandatory postgraduate UK surgical examination. BJS Open 2019; 3: 865–71. doi:10.1002/bjs5.50212 http://www.ncbi.nlm.nih.gov/pubmed/31832594

²¹ Scrimgeour DSG, Brennan PA, Griffiths G, et al. Does the Intercollegiate Membership of the Royal College of Surgeons (MRCS) examination predict ‘on-the-job’ performance during UK higher specialty surgical training? Ann R Coll Surg Engl 2018; 100: 669–75. doi:10.1308/rcsann.2018.0153

²² Scrimgeour DSG, Cleland J, Lee AJ, et al. Impact of performance in a mandatory postgraduate surgical examination on selection into specialty training. BJS Open 2017; 1: 67–74. doi:10.1002/bjs5.7 http://www.ncbi.nlm.nih.gov/pubmed/29951608

²³ Lee MJ, Drake TM, Malik TAM, et al. Has the Bachelor of Surgery Left Medical School?-A National Undergraduate Assessment. J Surg Educ 2016; 73: 655–9. doi:10.1016/j.jsurg.2016.01.005 http://www.ncbi.nlm.nih.gov/pubmed/26908017

²⁴ Scrimgeour DSG, Cleland J, Lee AJ, et al. Which factors predict success in the mandatory UK postgraduate surgical exam: the Intercollegiate membership of the Royal College of surgeons (MRCS)? Surgeon 2018; 16: 220–6. doi:10.1016/j.surge.2017.10.001 http://www.ncbi.nlm.nih.gov/pubmed/29102295

²⁵ Ellis R, Scrimgeour DSG, Brennan PA, et al. Does performance at medical school predict success at the Intercollegiate membership of the Royal College of surgeons (MRCS) examination? A retrospective cohort study. BMJ Open 2021; 11: e046615. doi:10.1136/bmjopen-2020-046615 http://www.ncbi.nlm.nih.gov/pubmed/34400449

²⁶ Ellis R, Cleland J, Scrimgeour DSG, et al. A cross-sectional study examining the association between MRCS performance and surgeons receiving sanctions against their medical registration. Surgeon 2021: 00076–7. doi:10.1016/j.surge.2021.04.003 http://www.ncbi.nlm.nih.gov/pubmed/34030984

²⁷ Ellis R, Cleland J, Lee AJ, et al. A cross-sectional study examining MRCS performance by core surgical training location. Med Teach 2021: 1–6. doi:10.1080/0142159X.2021.1995599 http://www.ncbi.nlm.nih.gov/pubmed/34727832

²⁸ McManus IC, Woolf K, Dacre J, et al. The Academic Backbone: longitudinal continuities in educational achievement from secondary school and medical school to MRCP(UK) and the specialist register in UK medical students and doctors. BMC Med 2013; 11: 242. doi:10.1186/1741-7015-11-242 http://www.ncbi.nlm.nih.gov/pubmed/24229333

²⁹ Ferguson E, James D, Madeley L. Factors associated with success in medical school: systematic review of the literature. BMJ 2002; 324: 952–7. doi:10.1136/bmj.324.7343.952 http://www.ncbi.nlm.nih.gov/pubmed/11964342

³⁰ Ellis R, Brennan P, Scrimgeour DS, et al. Performance at medical school selection correlates with success in part A of the intercollegiate membership of the Royal College of surgeons (MRCS) examination. Postgrad Med J 2021. doi: doi:10.1136/postgradmedj-2021-139748. [Epub ahead of print: 10 Mar 2021 ]. http://www.ncbi.nlm.nih.gov/pubmed/33692157

³¹ The Complete University Guide. Medical school ranking 2020, 2020. Available: https://www.thecompleteuniversityguide.co.uk/

³² Dowell J, Cleland J, Fitzpatrick S, et al. The UK medical education database (UKMED) what is it? why and how might you use it? BMC Med Educ 2018; 18: 6. doi:10.1186/s12909-017-1115-9 http://www.ncbi.nlm.nih.gov/pubmed/29304801

³³ McManus IC, Ludka K. Resitting a high-stakes postgraduate medical examination on multiple occasions: nonlinear multilevel modelling of performance in the MRCP(UK) examinations. BMC Med 2012; 10: 60. doi:10.1186/1741-7015-10-60 http://www.ncbi.nlm.nih.gov/pubmed/22697599

³⁴ Woolf K, Potts HWW, McManus IC. Ethnicity and academic performance in UK trained doctors and medical students: systematic review and meta-analysis. BMJ 2011; 342: d901. doi:10.1136/bmj.d901 http://www.ncbi.nlm.nih.gov/pubmed/21385802

³⁵ Medical Schools Council. Medicine Course Types [Internet], 2018. Available: https://www.medschools.ac.uk/studying-medicine/course-types [Accessed 11 Nov 2021 ].

³⁶ Kumwenda B, Cleland J, Greatrix R, et al. Are efforts to attract graduate applicants to UK medical schools effective in increasing the participation of under-represented socioeconomic groups? a national cohort study. BMJ Open 2018; 8: e018946. doi:10.1136/bmjopen-2017-018946

³⁷ Russell Group. Profile of the Russell Group of Universities [Internet], 2016. Available: https://russellgroup.ac.uk/policy/publications/profile-of-the-russell-group-of-universities/ [Accessed 11 Nov 2021 ].

³⁸ Boliver V. Are there distinctive clusters of higher and lower status universities in the UK? Oxf Rev Educ 2015; 41: 608–27. doi:10.1080/03054985.2015.1082905

³⁹ Coughlan S. Is the Russell Group really an ‘oligarchy’? [Internet]. BBC News Education and Family, 2014. Available: https://www.bbc.co.uk/news/education-27399512 [Accessed 11 Nov 2021 ].

⁴⁰ Fazackerley A. Should students be encouraged to set their sights on Russell Group universities? [Internet]. The Guardian, 2013. Available: https://www.theguardian.com/education/2013/feb/18/russell-group-universities-students-ambitions [Accessed 11 Nov 2021 ].

⁴¹ Lloyd-Jones G, Margetson D, Bligh JG. Problem-Based learning: a coat of many colours. Med Educ 1998; 32: 492–4. doi:10.1046/j.1365-2923.1998.00248.x http://www.ncbi.nlm.nih.gov/pubmed/10211290

⁴² Maudsley G. Do we all mean the same thing by “problem-based learning”? A review of the concepts and a formulation of the ground rules. Academic Medicine 1999; 74: 178–85. doi:10.1097/00001888-199902000-00016

⁴³ British Medical Association. Course and teaching types at medical school. London, 2017. Available: https://www.bma.org.uk/advice/career/studying-medicine/becoming-a-doctor/course-types [Accessed 11 Nov 2021 ].

⁴⁴ Intercollegiate Committee for Basic Surgical Examinations. Guide to the intercollegiate MRCS examination [Internet], 2013. Available: https://www.intercollegiatemrcsexams.org.uk/mrcs/candidate-guidance/ [Accessed 11 Nov 2021 ].

⁴⁵ Joint Committee on Surgical Training. Surgical selection in the UK. [Internet]. Available: https://www.jcst.org/introduction-to-training/selection-and-recruitment/ [Accessed 11 Nov 2021 ].

⁴⁶ HESA. Rounding and suppression to anonymise statistics. Available: https://www.hesa.ac.uk/about/regulation/data-protection/rounding-and-suppression-anonymise-statistics [Accessed 11 Nov 2021 ].

⁴⁷ Harrell FE. Regression modeling strategies: with applications to linear models, logistic regression, and survival analysis. 2nd edition. New York: Springer, 2006: 600.

⁴⁸ Rubright JD, Jodoin M, Barone MA. Examining demographics, prior academic performance, and United States medical licensing examination scores. Academic Medicine 2019; 94: 364–70. doi:10.1097/ACM.0000000000002366

⁴⁹ Sutton E, Richardson JD, Ziegler C, et al. Is USMLE step 1 score a valid predictor of success in surgical residency? Am J Surg 2014; 208: 1029–34. doi:10.1016/j.amjsurg.2014.06.032 http://www.ncbi.nlm.nih.gov/pubmed/25440485

⁵⁰ Swanson DB, Sawhill A, Holtzman KZ, et al. Relationship between performance on Part I of the American Board of orthopaedic surgery certifying examination and scores on USMLE steps 1 and 2. Acad Med 2009; 84: S21–4. doi:10.1097/ACM.0b013e3181b37fd2 http://www.ncbi.nlm.nih.gov/pubmed/19907379

⁵¹ Ghaffari-Rafi A, Lee RE, Fang R, et al. Multivariable analysis of factors associated with USMLE scores across U.S. medical schools. BMC Med Educ 2019; 19: 154. doi:10.1186/s12909-019-1605-z

⁵² Twigg V, McNally SA, Eardley I. What are the differences between medical schools that graduate more aspiring surgeons than others? Bulletin 2020; 102: e009. doi:10.1308/rcsbull.2020.e009

⁵³ McMANUS IC, Winder BC, Sproston KA, et al. Why do medical school applicants apply to particular schools? Med Educ 1993; 27: 116–23. doi:10.1111/j.1365-2923.1993.tb00241.x

⁵⁴ Brown C. A qualitative study of medical school choice in the UK. Med Teach 2007; 29: 27–32. doi:10.1080/01421590601032419 http://www.ncbi.nlm.nih.gov/pubmed/17538829

⁵⁵ McManus IC, Dewberry C, Nicholson S, et al. Construct-level predictive validity of educational attainment and intellectual aptitude tests in medical student selection: meta-regression of six UK longitudinal studies. BMC Med 2013; 11: 243. doi:10.1186/1741-7015-11-243 http://www.ncbi.nlm.nih.gov/pubmed/24229353

⁵⁶ Garrud P, McManus IC. Impact of accelerated, graduate-entry medicine courses: a comparison of profile, success, and specialty destination between graduate entrants to accelerated or standard medicine courses in UK. BMC Med Educ 2018; 18: 250. doi:10.1186/s12909-018-1355-3 http://www.ncbi.nlm.nih.gov/pubmed/30400933

⁵⁷ Jones A, McArdle PJ, O'Neill PA. Perceptions of how well graduates are prepared for the role of PRE-REGISTRATION house officer: a comparison of outcomes from a traditional and an integrated PBL curriculum. Med Educ 2002; 36: 16–25. doi:10.1046/j.1365-2923.2002.01105.x http://www.ncbi.nlm.nih.gov/pubmed/11849520

⁵⁸ Miles S, Kellett J, Leinster SJ. Medical graduates’ preparedness to practice: a comparison of undergraduate medical school training. BMC Med Educ 2017; 17: 33. doi:10.1186/s12909-017-0859-6

⁵⁹ Williams G, Lau A. Reform of undergraduate medical teaching in the United Kingdom: a triumph of evangelism over common sense. BMJ 2004; 329: 92–4. doi:10.1136/bmj.329.7457.92 http://www.ncbi.nlm.nih.gov/pubmed/15242914

⁶⁰ Albanese MA, Mitchell S. Problem-Based learning: a review of literature on its outcomes and implementation issues. Acad Med 1993; 68: 52–81. doi:10.1097/00001888-199301000-00012 http://www.ncbi.nlm.nih.gov/pubmed/8447896

⁶¹ Garlick PB, Brown G. Widening participation in medicine. BMJ 2008; 336: 1111–3. doi:10.1136/bmj.39508.606157.BE

⁶² Curtis S, Blundell C, Platz C, et al. Successfully widening access to medicine. Part 2: curriculum design and student progression. J R Soc Med 2014; 107: 393–7. doi:10.1177/0141076814538787 http://www.ncbi.nlm.nih.gov/pubmed/25271274

⁶³ Mahesan N, Crichton S, Sewell H, et al. The effect of an intercalated BSc on subsequent academic performance. BMC Med Educ 2011; 11: 76. doi:10.1186/1472-6920-11-76 http://www.ncbi.nlm.nih.gov/pubmed/21967682

⁶⁴ Curtis S, Smith D. A comparison of undergraduate outcomes for students from gateway courses and standard entry medicine courses. BMC Med Educ 2020; 20: 4. doi:10.1186/s12909-019-1918-y http://www.ncbi.nlm.nih.gov/pubmed/31900151

⁶⁵ Health Education England. Widening Participation it Matters! Our strategy and initial action plan [Internet], 2014. Available: https://www.hee.nhs.uk/sites/default/files/documents/Widening%20Participation%20it%20Matters_0.pdf [Accessed 11 Nov 2021 ].

⁶⁶ Health Education England. Specialty Recruitment Competition Ratios [Internet], 2020. Available: https://specialtytraining.hee.nhs.uk/Competition-Ratios [Accessed 11 Nov 2021 ].

⁶⁷ Puddey IB, Playford DE, Mercer A. Impact of medical student origins on the likelihood of ultimately practicing in areas of low vs high socio-economic status. BMC Med Educ 2017; 17: 1. doi:10.1186/s12909-016-0842-7 http://www.ncbi.nlm.nih.gov/pubmed/28056975

⁶⁸ Dowell J, Norbury M, Steven K, et al. Widening access to medicine may improve general practitioner recruitment in deprived and rural communities: survey of GP origins and current place of work. BMC Med Educ 2015; 15: 165. doi:10.1186/s12909-015-0445-8 http://www.ncbi.nlm.nih.gov/pubmed/26428081

⁶⁹ Puddey IB, Mercer A, Carr SE. Relative progress and academic performance of graduate vs undergraduate entrants to an Australian medical school. BMC Med Educ 2019; 19: 159. doi:10.1186/s12909-019-1584-0 http://www.ncbi.nlm.nih.gov/pubmed/31113431

⁷⁰ Manning G, Garrud P. Comparative attainment of 5-year undergraduate and 4-year graduate entry medical students moving into Foundation training. BMC Med Educ 2009; 9: 76. doi:10.1186/1472-6920-9-76 http://www.ncbi.nlm.nih.gov/pubmed/20028543

⁷¹ General Medical Council. Reports on the progress of doctors in training split by postgraduate body. Available: https://www.gmc-uk.org/education/reports-and-reviews/progression-reports [Accessed 11 Nov 2021 ].

⁷² Pyne Y, Ben-Shlomo Y. Older doctors and progression through specialty training in the UK: a cohort analysis of general medical Council data. BMJ Open 2015; 5: e005658. doi:10.1136/bmjopen-2014-005658 http://www.ncbi.nlm.nih.gov/pubmed/25649208

⁷³ Garrud P. Who applies and who gets admitted to UK graduate entry medicine? - An analysis of UK admission statistics. BMC Med Educ 2011; 11: 71. doi:10.1186/1472-6920-11-71 http://www.ncbi.nlm.nih.gov/pubmed/21943332

⁷⁴ Goldacre MJ, Turner G, Lambert TW. Variation by medical school in career choices of UK graduates of 1999 and 2000. Med Educ 2004; 38: 249–58. doi:10.1046/j.1365-2923.2004.01763.x http://www.ncbi.nlm.nih.gov/pubmed/14996333

⁷⁵ Patterson F, Knight A, Dowell J, et al. How effective are selection methods in medical education? A systematic review. Med Educ 2016; 50: 36–60. doi:10.1111/medu.12817 http://www.ncbi.nlm.nih.gov/pubmed/26695465

⁷⁶ Fielding S, Tiffin PA, Greatrix R, et al. Do changing medical admissions practices in the UK impact on who is admitted? an interrupted time series analysis. BMJ Open 2018; 8: e023274. doi:10.1136/bmjopen-2018-023274 http://www.ncbi.nlm.nih.gov/pubmed/30297349

⁷⁷ Patterson F, Roberts C, Hanson MD, et al. 2018 Ottawa consensus statement: selection and recruitment to the healthcare professions. Med Teach 2018; 40: 1091–101. doi:10.1080/0142159X.2018.1498589 http://www.ncbi.nlm.nih.gov/pubmed/30251906

⁷⁸ Ferguson E, James D, Yates J, et al. Predicting who applies to study medicine: implication for diversity in UK medical schools. Med Teach 2012; 34: 382–91. doi:10.3109/0142159X.2012.652237 http://www.ncbi.nlm.nih.gov/pubmed/22515306

⁷⁹ Kumwenda B, Cleland JA, Walker K, et al. The relationship between school type and academic performance at medical school: a national, multi-cohort study. BMJ Open 2017; 7: e016291. doi:10.1136/bmjopen-2017-016291 http://www.ncbi.nlm.nih.gov/pubmed/28860227

⁸⁰ Berger AP, Giacalone JC, Barlow P, et al. Choosing surgery as a career: early results of a longitudinal study of medical students. Surgery 2017; 161: 1683–9. doi:10.1016/j.surg.2016.12.016 http://www.ncbi.nlm.nih.gov/pubmed/28161006

⁸¹ Cleland J, Johnston PW, French FH, et al. Associations between medical school and career preferences in year 1 medical students in Scotland. Med Educ 2012; 46: 473–84. doi:10.1111/j.1365-2923.2012.04218.x http://www.ncbi.nlm.nih.gov/pubmed/22515755

⁸² Goldacre MJ, Laxton L, Harrison EM, et al. Early career choices and successful career progression in surgery in the UK: prospective cohort studies. BMC Surg 2010; 10: 32. doi:10.1186/1471-2482-10-32 http://www.ncbi.nlm.nih.gov/pubmed/21044317

⁸³ Adams T, Garden A. What influences medical school choice? Med Teach 2006; 28: 83–5. doi:10.1080/01421590500313027 http://www.ncbi.nlm.nih.gov/pubmed/16627331

⁸⁴ Kumwenda B, Cleland JA, Prescott GJ, et al. Relationship between sociodemographic factors and selection into UK postgraduate medical training programmes: a national cohort study. BMJ Open 2018; 8: e021329. doi:10.1136/bmjopen-2017-021329 http://www.ncbi.nlm.nih.gov/pubmed/29961026

⁸⁵ Scanlan GM, Cleland J, Johnston P, et al. What factors are critical to attracting NHS Foundation doctors into specialty or core training? a discrete choice experiment. BMJ Open 2018; 8: e019911. doi:10.1136/bmjopen-2017-019911 http://www.ncbi.nlm.nih.gov/pubmed/29530910

⁸⁶ Cariaga-Lo LD, Richards BF, Hollingsworth MA, et al. Non-cognitive characteristics of medical students: entry to problem-based and lecture-based curricula. Med Educ 1996; 30: 179–86. doi:10.1111/j.1365-2923.1996.tb00740.x http://www.ncbi.nlm.nih.gov/pubmed/8949551

⁸⁷ Holen A, Manandhar K, Pant DS, et al. Medical students' preferences for problem-based learning in relation to culture and personality: a multicultural study. Int J Med Educ 2015; 6: 84–92. doi:10.5116/ijme.558e.6451 http://www.ncbi.nlm.nih.gov/pubmed/26188962

⁸⁸ Bigsby E, McManus IC, Sedgwick P, et al. Which medical students enjoy problem-based learning? EIMJ 2013; 5: e72–6. doi:10.5959/eimj.v5i1.28

⁸⁹ Carter YH, Peile E. Graduate entry medicine: high aspirations at birth. Clin Med 2007; 7: 143–7. doi:10.7861/clinmedicine.7-2-143 http://www.ncbi.nlm.nih.gov/pubmed/17491502

⁹⁰ Broecke S. University rankings: do they matter in the UK? Educ Econ 2015; 23: 137–61. doi:10.1080/09645292.2012.729328

⁹¹ Tiffin PA, Paton LW. Differential attainment in the MRCPsych according to ethnicity and place of qualification between 2013 and 2018: a UK cohort study. Postgrad Med J 2021; 97: 764–76. doi:10.1136/postgradmedj-2020-137913 http://www.ncbi.nlm.nih.gov/pubmed/32883769

⁹² Dewhurst NG, McManus C, Mollon J, et al. Performance in the MRCP(UK) Examination 2003-4: analysis of pass rates of UK graduates in relation to self-declared ethnicity and gender. BMC Med 2007; 5: 8. doi:10.1186/1741-7015-5-8 http://www.ncbi.nlm.nih.gov/pubmed/17477862

⁹³ Wakeford R, Denney M, Ludka-Stempien K, et al. Cross-comparison of MRCGP & MRCP(UK) in a database linkage study of 2,284 candidates taking both examinations: assessment of validity and differential performance by ethnicity. BMC Med Educ 2015; 15: 1. doi:10.1186/s12909-014-0281-2 http://www.ncbi.nlm.nih.gov/pubmed/25592199

⁹⁴ Hope D, Adamson K, McManus IC, et al. Using differential item functioning to evaluate potential bias in a high stakes postgraduate knowledge based assessment. BMC Med Educ 2018; 18: 64. doi:10.1186/s12909-018-1143-0 http://www.ncbi.nlm.nih.gov/pubmed/29615016

⁹⁵ McManus IC, Elder AT, Dacre J. Investigating possible ethnicity and sex bias in clinical examiners: an analysis of data from the MRCP(UK) PACES and nPACES examinations. BMC Med Educ 2013; 13: 103. doi:10.1186/1472-6920-13-103 http://www.ncbi.nlm.nih.gov/pubmed/23899223

⁹⁶ Yeates P, Woolf K, Benbow E, et al. A randomised trial of the influence of racial stereotype bias on examiners' scores, feedback and recollections in undergraduate clinical exams. BMC Med 2017; 15: 179. doi:10.1186/s12916-017-0943-0 http://www.ncbi.nlm.nih.gov/pubmed/29065875

⁹⁷ Fyfe M, Horsburgh J, Blitz J. Guidelines: The dos, don'ts and don't knows of remediation in medical education. Perspect Med Educ 2021; 8: 322–38.

⁹⁸ Ellis R, Cleland J, Scrimgeour D, et al. The impact of disability on performance in a high-stakes postgraduate surgical examination: a retrospective cohort study. J R Soc Med 2021: 1410768211032573. doi:10.1177/01410768211032573 http://www.ncbi.nlm.nih.gov/pubmed/34269623

⁹⁹ Ross PT, Hart-Johnson T, Santen SA, et al. Considerations for using race and ethnicity as quantitative variables in medical education research. Perspect Med Educ 2020; 9: 318–23. doi:10.1007/s40037-020-00602-3 http://www.ncbi.nlm.nih.gov/pubmed/32789666

¹⁰⁰ Cleland JA, Milne A, Sinclair H, et al. An intercalated BSC degree is associated with higher marks in subsequent medical school examinations. BMC Med Educ 2009; 9: 24. doi:10.1186/1472-6920-9-24 http://www.ncbi.nlm.nih.gov/pubmed/19454007