Protocol for a three-phase prospective study to develop educational resource of antimicrobial resistance and stewardship for medical programme

Abstract

Introduction

Antimicrobial resistance (AMR) is one of the critical medical issues of the 21st century. Medical professionals are the primary prescribers of antimicrobials; their undergraduate education of antimicrobial stewardship (AMS) is considered one of the fundamental approaches in combating the issue of AMR. This education level provides a platform to bridge any gaps in their knowledge and competency in AMS. This study aims to develop an educational resource on microbes, hygiene and prudent antimicrobial use for the undergraduate medical programme. The guideline produced will then be assimilated into the existing curriculum which will help to improve the quality of education which in turn will improve rationale as the use of antimicrobials in the future.

Methods and analysis

A three-step approach consensus approach will be adopted for this study for the development of a validated medical curriculum guideline on AMR. A preliminary curriculum for the programme will be drafted from reviews of published literature including syllabi as well as national and international guidelines. A total of 26 potential sources were found to be relevant, and selected for this study. Subsequently, the drafted curriculum will be subjected for validation via online surveys by various infectious disease experts. Finally, a Delphi technique will be employed to obtain consensus on heterogeneous findings to the revised curriculum. The quantitative and qualitative responses will be analysed and discussed among the panel of researchers.

Ethics and dissemination

This study protocol has been approved by the Institute of Health Sciences Research Ethics Committee of Universiti Brunei Darussalam (Reference: UBD/PAPRSBIHSREC/2020/124). Informed consent declaration will be collected prior to data collections as indication of agreement of participation in the study. Results will be made available to medical educators and also researchers on AMR and stewardship. The results also will be disseminated at feedback sessions to officers at Ministry of Health and Ministry of Education, Brunei Darussalam.

Correspondence to Dr Long Chiau Ming; [email protected] ; Dr Jagjit Singh Dhaliwal; [email protected]

Strengths and limitations of this study

• This protocol presents the first application of Delphi method in the antimicrobial resistance field for professional medical degree that could lead to full consensus curriculum alignment.

• Triphasic validated consensus method will be used to develop a future-proof antimicrobial resistance curriculum for professional medicine programme.

• To minimise this limitation, a comprehensive recruitment process involving a systematic review and snowball technique will be used, to ensure a representative range of international researchers and clinicians involved in exercise are invited to participate in the survey.

• Need-based educational resource on microbes and rational antimicrobial prescribing and content priorities related to medical clinical practice are produced and validated by international panels.

• The acceptance of the developed antimicrobial resistance curriculum need to be tested among wider international medical fraternity using further quantitative survey

Background

Antimicrobials are considered one of the greatest medical developments in the last century, however, the abuse of these seemingly miracle drugs have led them to become ineffective in treating infections¹ Overtime, these microbes have gained immunity such as through gene mutations and exchange of resistant genes² Ineluctably, common infectious diseases like pneumonia, tuberculosis and gonorrhoea are becoming harder, if not impossible, to treat.³ As consequences, a higher morbidity and mortality rate as well as an increased in healthcare cost are the price to pay.^{4 5} Therefore, the antimicrobial resistance (AMR) has become a global menace that ‘threatens the very core of modern medicine’.^{6 7}

One of the most leading cases on AMR involves multiple-drug resistant tuberculosis which has infected almost half a million people in 2016 alone.⁸ At current rates, AMR has contributed to at least 700 000 deaths annually, and projected to increase to 10 million lives annually by 2050.⁵ Some data are available on the current status of AMR in Brunei Darussalam⁹ but the nation has relatively one of the lowest antibiotics consumptions with an average of 5.9 defined daily dose per 1000 population per day.^{10 11} Paradoxically, an observational study found a marked increase in resistance of Neisseria Gonorrhoeae towards Quinolone with 93% in 2009¹² compared with only 21% in 2001.¹³

During COVID-19 pandemic, the use of antibiotics has been surprisingly escalated despite its viral origin.¹⁴ A review of studies found that 72% of the hospitalised COVID-19 cases received antibiotics despite less than 10% experienced bacterial or fungal co-infections.¹⁵ These inappropriate prescriptions can lead to the higher risk of AMR.¹⁵ Furthermore, the extensive use of biocidal agents in non-clinical settings such as for personal disinfection may also escalate this phenomena.¹⁵ However, no conclusion was made based on previous reports on the effects of COVID-19 on AMR.¹⁶ In this current pandemic, the WHO has recommended the use of antibiotics in only severe cases where risk of getting infections and death is increased.¹⁴

Various strategies have been employed to tackle the issue of AMR; Brunei Darussalam, in particular, has published the National Action Plan and the Good Antibiotic Prescribing Practice booklets.^{11 17} One of the main priorities realised is the education of the health professionals as well as undergraduate healthcare students including medicine, dentistry and pharmacy. The education of antimicrobial stewardship (AMS) has been recognised as one of the essential elements in addressing the issue of AMR¹⁸ A recent local study concluded that only 69% of the students have a good knowledge on antibiotics and AMR.¹¹ Another study in the UK also stated that only one-third of the undergraduate healthcare programmes includes all the recommended AMS principles.¹⁹

Objectives

The general objective of this study is to develop an educational resource on microbes, hygiene and prudent antimicrobial use for undergraduate medical programme. First, the educational resource developed as a result of this study will be assimilated into the existing curriculum in the local institution. Second, by reviewing various syllabi as well as national and international guidelines, we will be able to define the gaps in knowledge and improve the quality of education and the use of antimicrobials in the future.

Methods/design

Study overview

This study adopts a three-step approach in developing medical curriculum content guidelines on AMR by consensus as outlined by Cumyn and Harris²⁰:

• Initial delineation of curriculum content.

• Validation of curriculum content.

• Application of Delphi technique on debatable modification.

Phase 1: initial delineation of curriculum content

We will produce a preliminary curriculum guideline on antimicrobials and AMR by reviewing and analysing literature primarily from published syllabi and guidelines by medical councils such as Australian Medical Council and Malaysian Medical Council.

Data sources and search plan

We have extracted literatures through various methods (see table 1). Relevant resources were extracted from searching across several databases including PubMed, SCOPUS, Web of Science and EBSCOhost. Furthermore, we have also identified medical schools in the Oceania region accredited with respective medical councils such as Australian Medical Council and Malaysian Medical Council. A thorough search has been done at each university’s webpage for any published curriculum. There are 23 accredited medical schools in Australia by the Australian Medical Council as well as 13 and 34 accredited medical schools in Indonesia and Malaysia, respectively, by the Malaysian Medical Council. In addition to this, manual search was done using Google Advanced Search and cross reference search of selected articles.

Table 1

Summary of selected literatures detailing the search terms used, the search engine (PubMed, Scopus, Web of Science, EBSCOhost), the number of web results generated and the number selected from the results

Search term	Search engine	Results	Selected
(Bachelor medicine OR bachelor surgery OR medical OR mbbs) AND (syllabus OR curriculum OR learning objective OR learning aim) AND stewardship	PubMed	750	3
Antimicrobial resistance AND (syllabus OR curriculum OR learning objective OR learning aim) AND undergraduate	PubMed	76	3
(Bachelor medicine OR bachelor surgery OR medical OR mbbs) AND (syllabus OR curriculum OR learning objective OR learning aim) AND stewardship	Scopus	670	3
Antimicrobial resistance AND (syllabus OR curriculum OR learning objective OR learning aim) AND undergraduate	Scopus	147	4
(Bachelor medicine OR bachelor surgery OR medical OR mbbs) AND (syllabus OR curriculum OR learning objective OR learning aim) AND stewardship	Web of Science (WOS)	142	2
Antimicrobial resistance AND (syllabus OR curriculum OR learning objective OR learning aim) AND undergraduate	Web of Science (WOS)	29	3
(Bachelor medicine OR bachelor surgery OR medical OR mbbs) AND (syllabus OR curriculum OR learning objective OR learning aim) AND stewardship	EBSCOhost (Medline Complete	3675	1
Antimicrobial resistance AND (syllabus OR curriculum OR learning objective OR learning aim) AND undergraduate	EBSCOhost (Medline Complete)	372	1
N.B. Total selected: 20 Total duplicates: 17 (cross-checked with sources from each database and reference lists) Final total: 3

The search strategy was formulated by researcher (LCM) while the search itself was performed by lead researcher (MAMMY). The search was focused on undergraduate medicine programme curriculum combined with the research main theme ‘antimicrobials’ as to narrow down the search results. Overall, a total of 26 potential articles/resources will be included in the study (see table 2). The articles’ information also recorded the setting of the study and the year of publication.

Table 2

Details of potential literatures/syllabi from various sources (n=26)

Title	Setting	Year Published
A. databases (PubMed, Scopus, WOS and EBSCOhost) after duplicates removed (n=3)
An outcome-based approach for teaching prudent antimicrobial prescribing to undergraduate medical students: report of a Working Party of the British Society for Antimicrobial Chemotherapy	UK	2005
Development of consensus based national antimicrobial stewardship competencies for UK undergraduate healthcare professional education	UK	2018
How to educate prescribers in antimicrobial stewardship practices	France	2013
B. Medical school’s curriculum (n=2)
Universiti Kebangsaan Malaysia	Malaysia	2018–2019
Universiti Malaya	Malaysia	–
C. Manual search (Google Advanced Search) (n=16)
Medical Councils
Curriculum of MBBS by Pakistan Medical and Dental Council and Higher Education Commission Islamabad	Pakistan	2011
Nigeria Undergraduate Medical and Dental Curriculum Template	Nigeria	2012
Competency Based Undergraduate Curriculum for the Indian Medical Graduate	India	2018
Core Curriculum of Bachelor of Medicine and Bachelor of Surgery by The Rwanda Medical and Dental Council	Rwanda	2019
Universities
All India Institute of Medical Sciences India	India	2003
Sambalpur University	India	2012
University of Peradeniya	Sri Lanka	2013
Universiti of Dammam College of Medicine	Saudi Arabia	2014–2015
SRM University	India	2015
Universiti of Health Science Lahore	Pakistan	2015–2016
Dow University of Health Sciences	Pakistan	2016
Kerala University of Health Sciences	India	2016
Amna Inayat Medical College Pakistan	Pakistan	2017
NITTE (deemed to be university)	India	2018
National Universiti of Medical Sciences Pakistan	Pakistan	2019–2020
Others
Integrated Syllabus for MBBS Programme	USA	2014
D. Reference lists (n=5)
Antimicrobial prescribing and stewardship competencies	UK	2013
Antimicrobial Stewardship Curriculum by Wake Forest School of Medicine	USA	–
European Antibiotic Awareness Day 2017: training the next generation of healthcare professionals in antibiotic stewardship	Europe	2017
Health Workers’ Education and Training on Antimicrobial Resistance	UK	2019
WHO Competency Framework for Health Workers’ Education and Training on Antimicrobial Resistance	Geneva	2018

Inclusion and exclusion criteria

Inclusion criteria:

• All literatures pertaining to antimicrobials and AMR from any country at all time.

• English language only.

• Medical education only.

Exclusion Criteria:

• Non-English and non-antimicrobials related literatures.

• Postgraduate and undergraduate curriculums other than medicine or general healthcare.

Search terms

Various search terms and combinations were proposed to broaden the search results. The words “bachelor medicine,” “bachelor surgery,” “medical,” and “Bachelor of Medicine, Bachelor of Surgery OR mbbs”; “syllabus,” “curriculum,” “learning objective,” and “learning aim”; “antimicrobial resistance”; “undergraduate”; and “stewardship” were used interchangeably. Furthermore, these terms were combined into two common search term for simplicity and to optimise time: (bachelor medicine OR bachelor surgery OR medical OR mbbs) AND (syllabus OR curriculum OR learning objective OR learning aim) AND stewardship, as well as, Antimicrobial resistance AND (syllabus OR curriculum OR learning objective OR learning aim) AND undergraduate (see table 1).

Evaluation of curriculum content

All curriculum topics and learning points from all the selected resources will be identified using a previously validated broad-based search criterion (see box 1).²¹ Following the identification of learning points to be included in the curriculum, the variations between syllabi will be calculated using two methods; first, the percentage of learning points on AMR in the whole curriculum, second, the percentage of learning points in each determined level of achievement. A modified four-level Miller’s pyramid (see figure 1) will be applied to rate the level of achievement of each individual learning point; level 1, demonstrates the knowledge; level 2, demonstrates the ability to understand knowledge in a clinical context; level 3, demonstrates behaviour in a controlled environment; level 4, demonstrates behaviour in a free-working environment.²¹

Box 1

Curriculum search criteria and definitions²¹

Curriculum search criteria:

• Anti (wildcard search accepting antibiotic, antimicrobial or similar).

• Resist (wildcard search accepting resistant, resistance or similar).

• Infect (wildcard search accepting infection, infective, infected or similar).

• Stewardship.

AMS/AMR definitions:

AMS: ‘Optimising the indication, selection, dosing, route of administration and duration of antimicrobial therapy to maximise clinical cure or prevention of infection while limiting the collateral damage of antimicrobial use, including toxicity, selection of pathogenic organisms and emergence of resistance’.

AMR: ‘Resistance of an organism to an antimicrobial drug that was originally effective for the treatment of infections caused by it’.

AMR, antimicrobial resistance; AMS, antimicrobial stewardship.

Figure 1. Modified Miller’s pyramid to rate level of perceived output for each learning points. 21

Assessment of quality of included studies

Contents that are essentials will be included and compiled to form a draft curriculum. Any duplications in learning points will only be counted once. The selected resources will then be assessed by the panel of researchers for the style of the drafted curriculum as well as its appropriateness, accuracy, clarity and comprehensiveness. Researchers can also suggest for any additions, modifications and deletions of contents that is suitable. Any issues arise will be discussed within the team to reach consensus. Essentially, contents that are deemed important and agreed will then be proceeded to the second phase of the research study for validation. The process of selection of curriculum topics and learning points will be based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (see figure 2).²¹

Figure 2. PRISMA flow diagram. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

Phase 2: validation of the curriculum content

The preliminary curriculum produced from Phase 1 of this study will be validated by online quantitative and qualitative survey by a larger group of local and international experts on the field of antimicrobials and AMR.

Quantitative survey

An online questionnaire will be made based on the curriculum composing of various topics on antimicrobials with respective learning points (see table 3).

Table 3

Example of survey format to be used for phase 2

Please indicate with (✓) for each item: include/modify/delete. Suggestion(s) to include any additional items can be made in the comment section at the end of every topic.
Topic 1: Infection prevention and control	Include	Modify	Delete
Learning point 1: Understand the role of hand hygiene to prevent transmission of pathogens
Justification:
Learning point 2: Promote principles of healthcare-associated infections prevention and control
Justification:
Comment:

Qualitative survey

The second part of the survey includes online interviews in which experts will be for their feedback and queries on the quantitative survey. Any important modifications will be made based on the analysis of the two surveys and will be proceeded for consensus in phase 3.

If the outcome unanimously reached 100% rejection, the item will be removed from the curriculum. Items achieving an agreement level of 100% or less will be carried on to the phase 3.

Phase 3: application of Delphi technique

The modified curriculum will then be further debated via Delphi method which consists of rounds of questionnaire to obtain the experts perspectives and consensus on certain items of the topics.²² The results will then be collated and results will be shared within the group after each round.²³ The experts will be allowed to modify their answers in the subsequent rounds based on the projected results. This method is chosen due to its minimal potential bias, cost and time effectiveness as well as good acceptance by majority of user.^{24 25} The planned start and end dates for the study are 1 February 2022 and 30 January 2023.

Round 1 of survey

In this first round, the panel of experts will be instructed to rate on a 5-point Likert scale (1=strongly disagree; 2=disagree; 3=neutral; 4=agree; 5=strongly agree) on the level of their agreement on the proposed modifications (see table 4). A comment space will be provided on each item for the experts to explain their perspectives.

Table 4

Example of format to be used for Delphi survey in phase 3

Background Information: Here is a list of items as suggested by some respondents. We wish to remind you that the aim of the research is to develop a syllabus on antimicrobials for medicine programme; this could provide a foundation of knowledge that will enhance understanding in antimicrobial use and promote the judicious use of antimicrobials in clinical practice. For each item, please rate the extent to which you agree or disagree using the scale below.
Topic 1: Infection prevention and control (example)
Suggested addition	Rationale	Rating Please use the following rating scale: 1=Strongly disagree 2=Disagree 3=Neutral 4=Agree 5=Strongly Agree
To add ‘XXX’		Your rating: Comment:
To add ‘YYY’		Your rating: Comment:

As there is no published level of agreement, we have decided to adopt the classical criterion. Any item that achieve agreement will be accepted. Items with an agreement level between 80% and 100% will be accepted but modified according to the suggestions and advanced to the second round of survey. Items with fewer agreement will be rejected unless accompanied with comments to improve these specific items.

Round 2 of survey

Items that reach consensus will be included and labelled ‘consensus achieved’ and does not requires additional debates. Items that receive necessary feedback and require further consensus will be assessed on the second round of survey. The experts will only be given the options to accept or reject the final modifications. Comment boxes will be provided only for any corrections of typographical errors. Data will be analysed similar to the round 1 of Delphi.

Post-Delphi

This stage is created to verify the final curriculum contents by the group of researchers only. Any typographical errors not identified in the previous rounds will be corrected as well as the refinement and standardisation of the format will be done.

Recruitment and identification of experts

Experts refer to individuals who are knowledgeable and experienced on a certain subject of AMR and AMS as well as able to influence policy.²⁶ Since there is no fixed definition of an AMR expert, we have adopted a definition which includes ‘individuals reflecting expertise in prescribing and medicines management with regard to the education and practice of healthcare professionals; and antimicrobial prescribing and stewardship’.^{19 27} Individuals that does not meet any of these criteria will not be included in this study.

There is also no agreement on the number of experts required for this type of study as it is dependent on the purpose and complexity of the research aim as well as the population and resources available.²⁷ Sample size was calculated using PS Software.²⁸ This study will require a minimum sample of 82 participants in quantitative group and 41 participants in qualitative group, with ratio of 2:1 and estimated SD of 1.6 (22) with 80% certainty (power=0.8 and alpha=0.05). Considering different form of attritions, the minimum sample is inflated by 20%. Therefore, the expected sample size will be 100 for quantitative study group and 50 for qualitative group.

Pilot Study: A subset (5–10) of experts will be contacted and a pretest will be conducted to estimate expert inputs, recruitment retention, withdrawal rates, and acceptance of evaluation measures. Participants from the pretest will not be part of the main survey. Changes, if any, will be notified to ethics committee before proceeding with main survey.

A purposive and snowball sampling will be applied to recruit eligible experts in four countries including Australia, Brunei Darussalam, Indonesia and Malaysia. The panel of experts in this study involves qualified medical clinical academicians and medical education professionals. An initial email of invitation consisting of a participant information sheet and a consent form with the individual participant identification number (PIN) will be sent to the identified experts in these countries. Informed consent will also be obtained from participants as agreement to participate prior to commencement of data collection.

Specifically, the inclusion and exclusion criteria for participants’ selection are:

Inclusion criteria:

• Qualified academicians or healthcare professionals with at least 5 years clinical practice experience on AMR or related disciplines.

• Academic healthcare professionals involved in research and teaching of antimicrobial-related disciplines.

• Prescribers and non-prescribers including medical doctors and specialists must have at least 5 years industry working experience.

• Experts with academic qualifications level of a minimum basic bachelor degree.

Exclusion criteria:

• Non-experts on AMR-related field.

• Other healthcare professionals not stated in inclusion criteria.

Data collection and data analysis

Each round of quantitative survey will be conducted using a commonly used web survey tool in the university, Qualtrics, while the qualitative surveys will be performed using Zoom software.

The participants will be sent an email containing the links to the online surveys on the specified dates. The link will be accessible for 4 weeks to complete and submit the questionnaire. A reminder email will be sent to any non-responders on the second week of the data collection.

Data obtained will be analysed using RStudio software for Windows. Descriptive and inferential statistics will be done to analyse quantitative survey responses. A qualitative analysis of the justifications will be performed among the panel of researchers.

Ethics and dissemination

Ethical approval was obtained from the Institute of Health Sciences Research Ethics Committee of Universiti Brunei Darussalam (UBD/PAPRSBIHSREC/2020/124).

Experts who understand the terms and conditions and agrees to participate will have to complete and sign the given consent form designated with their PIN. Identified experts will initially be explained about this research study as well their expected roles and rights to avoid conflicts during the duration of the study. Their participation will be completely voluntary and they are free to withdraw without justification by informing the researcher their PIN number. However, data from previous set of questionnaires submitted will not be removed from the analysis and publication. No personal disclosing information is to be collected to ensure anonymity. Experts will only be asked to state their age, place and years of practice, principal role as well as their qualifications. Participants will only be asked to provide their name and on the informed consent form as to be acknowledge as a member of the expert panel in publications of the study. Furthermore, all information will be kept confidential and locked in a password-protected computer and accessible to the research team only.

Results will be made available to medical educators and also researchers on AMR and stewardship. The results also will be disseminated at feedback sessions to officers at Ministry of Health and Ministry of Education, Brunei Darussalam.

Patient and public involvement

No patient or public is involved in this study.

Discussion

The knowledge and competency of medical professionals will definitely ensure the highest quality of healthcare and this can be achieved by ensuring a robust foundation in their knowledge. Furthermore, a standardised curriculum syllabus will not only be helpful to the higher education institutions and educators, but also function as a guidance for the students learning. To the best of our knowledge, the current antimicrobial and AMR curriculum for undergraduate medical education in the region has not been validated nor there has been any study conducted to validate or develop a robust draft.

This research study will produce a locally tailored curriculum based on reviews of curriculums and literatures and further validations by consensus among experts on antimicrobials and AMR in the region. This study will address any gaps in the curriculum particularly on antimicrobials and AMR. The curriculum produced will then be assimilated into the existing medicine programmes which overall will hopefully result in a positive solution on the escalating issue of AMR.

This study will have some limitations. The survey includes panel of experts from limited countries and may not completely represents the international medical experts. Therefore, the findings of this study might not be generalised globally. Furthermore, the curriculum that will be produced is time-limited and will require revisions over time in accordance to the development in antimicrobials and AMR.

Authors would like to thank Mr Ang Chuan Yok for his support in ethical application documentation.

Ethics statements

Patient consent for publication

Not applicable.

Footnote

Contributors LCM schemed out the framework and design of the search strategy. MAMMY wrote and developed the study protocol. JSD, LCM, SHM, MEA, JSD, YYK, HR, SKSD and ZH revised and reviewed the final protocol. All the authors read and approved the final version.

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests None declared.

Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review Not commissioned; externally peer reviewed.

References

¹ Gupta P, Deka S. The menace of antimicrobial resistance. Indian J Community Health 2018; 30: 317–22.

² Munita JM, Arias CA. Mechanisms of antibiotic resistance. Microbiol Spectr 2016; 4. doi:10.1128/microbiolspec.VMBF-0016-2015 http://www.ncbi.nlm.nih.gov/pubmed/27227291

³ Zaman SB, Hussain MA, Nye R, et al. A review on antibiotic resistance: alarm Bells are Ringing. Cureus 2017; 9: e1403. doi:10.7759/cureus.1403 http://www.ncbi.nlm.nih.gov/pubmed/28852600

⁴ Shamsuddin S, Akkawi ME, Zaidi STR, et al. Antimicrobial drug use in primary healthcare clinics: a retrospective evaluation. Int J Infect Dis 2016; 52: 16–22. doi:10.1016/j.ijid.2016.09.013 http://www.ncbi.nlm.nih.gov/pubmed/27639454

⁵ Bhatia R. Antimicrobial resistance in developing Asian countries: burgeoning challenge to global health security demanding innovative approaches. Global Biosecurity 2019; 1: 50. doi:10.31646/gbio.4

⁶ Stalteri Mastrangelo R, Santesso N, Bognanni A, et al. Consideration of antimicrobial resistance and contextual factors in infectious disease guidelines: a systematic survey. BMJ Open 2021; 11: e046097. doi:10.1136/bmjopen-2020-046097 http://www.ncbi.nlm.nih.gov/pubmed/34330853

⁷ Elias C, Moja L, Mertz D, et al. Guideline recommendations and antimicrobial resistance: the need for a change. BMJ Open 2017; 7: e016264. doi:10.1136/bmjopen-2017-016264 http://www.ncbi.nlm.nih.gov/pubmed/28751488

⁸ Lomazzi M, Moore M, Johnson A, et al. Antimicrobial resistance - moving forward? BMC Public Health 2019; 19: 858. doi:10.1186/s12889-019-7173-7 http://www.ncbi.nlm.nih.gov/pubmed/31266477

⁹ Abdul Momin M. Antibiotic resistance in gram-negative bacteria in Brunei Darussalam: molecular characterisation, epidemiology, surveillance and virulence. Queen Mary University of London, 2019.

¹⁰ Organization WH. WHO report on surveillance of antibiotic consumption: 2016-2018 early implementation, 2018.

¹¹ Shahpawee NS, Chaw LL, Muharram SH, et al. University students' antibiotic use and knowledge of antimicrobial resistance: what are the common myths? Antibiotics 2020; 9. doi: doi:10.3390/antibiotics9060349. [Epub ahead of print: 20 Jun 2020 ]. http://www.ncbi.nlm.nih.gov/pubmed/32575716

¹² WHO Western Pacific and South East Asian Gonococcal Antimicrobial Surveillance Programmes. Surveillance of antibiotic resistance in Neisseria gonorrhoeae in the who Western Pacific and South East Asian regions, 2009. Commun Dis Intell Q Rep 2011; 35: 2–7. http://www.ncbi.nlm.nih.gov/pubmed/21698977

¹³ Surveillance of antibiotic resistance in Neisseria gonorrhoeae in the who Western Pacific region, 2001. World Health organization. Commun Dis Intell Q Rep 2002; 26: 541–5. http://www.ncbi.nlm.nih.gov/pubmed/12549520

¹⁴ Hsu J. How covid-19 is accelerating the threat of antimicrobial resistance. BMJ 2020; 369: m1983. doi:10.1136/bmj.m1983 http://www.ncbi.nlm.nih.gov/pubmed/32423901

¹⁵ Getahun H, Smith I, Trivedi K, et al. Tackling antimicrobial resistance in the COVID-19 pandemic. Bull World Health Organ 2020; 98: 442–442A. doi:10.2471/BLT.20.268573 http://www.ncbi.nlm.nih.gov/pubmed/32742026

¹⁶ Clancy CJ, Buehrle DJ, Nguyen MH. Pro: the COVID-19 pandemic will result in increased antimicrobial resistance rates. JAC Antimicrob Resist 2020; 2: dlaa049. doi:10.1093/jacamr/dlaa049 http://www.ncbi.nlm.nih.gov/pubmed/34192248

¹⁷ Ang CY, Dhaliwal JS, Muharram SH, et al. Educational resource for antimicrobial resistance and stewardship for dentistry programmes: a research protocol. BMJ Open 2021; 11: e048609. doi:10.1136/bmjopen-2021-048609 http://www.ncbi.nlm.nih.gov/pubmed/34233993

¹⁸ Tang KL, Teoh TF, Ooi TT, et al. Public hospital pharmacists' perceptions and knowledge of antibiotic use and resistance: a multicenter survey. Antibiotics 2020; 9. doi: doi:10.3390/antibiotics9060311. [Epub ahead of print: 09 Jun 2020 ]. http://www.ncbi.nlm.nih.gov/pubmed/32526821

¹⁹ Courtenay M, Lim R, Castro-Sanchez E, et al. Development of consensus-based national antimicrobial stewardship competencies for UK undergraduate healthcare professional education. J Hosp Infect 2018; 100: 245–56. doi:10.1016/j.jhin.2018.06.022 http://www.ncbi.nlm.nih.gov/pubmed/29966757

²⁰ Cumyn A, Harris IB. A comprehensive process of content validation of curriculum consensus guidelines for a medical specialty. Med Teach 2012; 34: e566–72. doi:10.3109/0142159X.2012.668623 http://www.ncbi.nlm.nih.gov/pubmed/22489987

²¹ Rawson TM, Butters TP, Moore LSP, et al. Exploring the coverage of antimicrobial stewardship across UK clinical postgraduate training curricula. J Antimicrob Chemother 2016; 71: 3284–92. doi:10.1093/jac/dkw280 http://www.ncbi.nlm.nih.gov/pubmed/27432603

²² Humphrey-Murto S, Wood TJ, Gonsalves C, et al. The Delphi method. Acad Med 2020; 95: 168. doi:10.1097/ACM.0000000000002887 http://www.ncbi.nlm.nih.gov/pubmed/31335812

²³ Diamond IR, Grant RC, Feldman BM, et al. Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol 2014; 67: 401–9. doi:10.1016/j.jclinepi.2013.12.002 http://www.ncbi.nlm.nih.gov/pubmed/24581294

²⁴ Humphrey-Murto S, Varpio L, Wood TJ, et al. The use of the Delphi and other consensus group methods in medical education research: a review. Acad Med 2017; 92: 1491–8. doi:10.1097/ACM.0000000000001812 http://www.ncbi.nlm.nih.gov/pubmed/28678098

²⁵ Romero-Collado A. Essential elements to elaborate a study with the (e)Delphi method. Enfermería Intensiva 2021; 32: 100–4. doi:10.1016/j.enfie.2020.09.003

²⁶ Finn GM, Hitch G, Apampa B, et al. The anatomical Society core anatomy syllabus for pharmacists: outcomes to create a foundation for practice. J Anat 2018; 232: 729–38. doi:10.1111/joa.12787 http://www.ncbi.nlm.nih.gov/pubmed/29417585

²⁷ Courtenay M, Castro-Sánchez E, Deslandes R, et al. Defining antimicrobial stewardship competencies for undergraduate health professional education in the United Kingdom: a study protocol. J Interprof Care 2018; 32: 638–40. doi:10.1080/13561820.2018.1463200 http://www.ncbi.nlm.nih.gov/pubmed/29658808

²⁸ Dupont WD, Plummer WD. Ps: power and sample size calculation. Control Clin Trials 1997: 18–274.