Evidence based medicine: an approach to clinical problem-solving

Doctors within the NHS are confronting major changes at work. While we endeavour to improve the quality of health care, junior doctors' hours have been reduced and the emphasis on continuing medical education has increased. We are confronted by a growing body of information, much of it invalid or irrelevant to clinical practice. This article discusses evidence based medicine, a process of turning clinical problems into questions and then systematically locating, appraising, and using contemporaneous research findings as the basis for clinical decisions. The computerisation of bibliographies and the development of software that permits the rapid location of relevant evidence have made it easier for busy clinicians to make best use of the published literature. Critical appraisal can be used to determine the validity and applicability of the evidence, which is then used to inform clinical decisions. Evidence based medicine can be taught to, and practised by, clinicians at all levels of seniority and can be used to close the gulf between good clinical research and clinical practice. In addition it can help to promote self directed learning and teamwork and produce faster and better doctors.

Doctors must cope with a rapidly changing body of relevant evidence and maximise the quality of medical care despite the reduction in junior doctors' working hours and scarce resources. We are deluged with information, and although much of it is either invalid or irrelevant to clinical practice, an increasing amount comes from powerful investigations such as randomised controlled trials. Yet we continue to base our clinical decisions on increasingly out of date primary training or the overinterpretation of experiences with individual patients,¹ and even dramatically positive results from rigorous clinical studies remain largely unapplied.² Doctors need new skills to track down the new types of strong and useful evidence, distinguish it from weak and irrelevant evidence, and put it into practice. In this paper we discuss evidence based medicine, a new framework for clinical problem solving which may help clinicians to meet these challenges.

What is evidence based medicine?

Evidence based medicine is the process of systematically finding, appraising, and using contemporaneous research findings as the basis for clinical decisions. For decades people have been aware of the gaps between research evidence and clinical practice, and the consequences in terms of expensive, ineffective, or even harmful decision making.³⁴ Inexpensive electronic databases and widespread computer literacy now give doctors access to enormous amounts of data. Evidence based medicine is about asking questions, finding and appraising the relevant data, and harnessing that information for everyday clinical practice.

Most readers will recognise that the ideas underlying evidence based medicine are not new. Clinicians identify the questions raised in caring for their patients and consult the literature at least occasionally, if not routinely. The difference with using an explicit, evidence based medicine framework is twofold: it can make consulting and evaluating the literature a relatively simple, routine procedure, and it can make this process workable for clinical teams, as well as for individual clinicians. The term "evidence based medicine" was coined at McMaster Medical School in Canada in the 1980s to label this clinical learning strategy, which people at the school had been developing for over a decade.⁵

Evidence based medicine in practice

Evidence based medicine can be practised in any situation where there is doubt about an aspect of clinical diagnosis, prognosis, or management.

Four steps in evidence based medicine

Formulate a clear clinical question from a patient's problem

Search the literature for relevant clinical articles

Evaluate (critically appraise) the evidence for its validity and usefulness

Implement useful findings in clinical practice

SETTING THE QUESTION

A 77 year old woman living alone is admitted with non-rheumatic atrial fibrillation and her first bout of mild left ventricular failure, and she responds to digoxin and diuretics. She has a history of well controlled hypertension. An echocardiogram shows moderately impaired left ventricular function. She is an active person and anxious to maintain her independence. During the ward round on the following day a debate ensues about the risks and benefits of offering her long term anticoagulation with warfarin, and rather than defer to seniority or abdicate responsibility to consensus by committee, team members convert the debate into a question: "How does her risk of embolic stroke, if we don't give her anticoagulant drugs, compare with her risk of serious haemorrhage and stroke if we do?"

The questions that initiate evidence based medicine can relate to diagnosis, prognosis, treatment, iatrogenic harm, quality of care, or health economics. In any event, they should be as specific as possible, including the type of patient, the clinical intervention, and the clinical outcome of interest. In this example two questions are prepared for a literature search. One question relates to prognosis and her susceptibility: "How great is the annual risk of embolic stroke in a 77 year old woman with non-rheumatic atrial fibrillation, hypertension, and moderate left ventricular enlargement if she is not given anticoagulants?" The other question concerns treatment and asks, "What is the risk reduction for stroke from warfarin therapy in such a patient, and what is the risk of harming her with this therapy?"

FINDING THE EVIDENCE

The second step is a search for the best available evidence. To conduct searches on a regular basis, clinicians need effective searching skills and easy access to bibliographic databases. Increasingly the access can be proved by ward or surgery based computers, complemented by assistance in obtaining hard copies of articles, and enabled by librarians who teach searching skills and guide the unwary through the 25000 biomedical journals now in print.⁶⁷

Two sorts of electronic databases are available. The first sort is bibliographic and permits users to identify relevant citations in the clinical literature, using variations of Medline. The second sort of database takes the user directly to primary or secondary publications of the relevant clinical evidence-the rapidly growing numbers include the Cochrane Database of Systematic Reviews, Scientific American Medicine on CD-ROM, and the ACP Journal Club (a bimonthly supplement to the Annals of Internal Medicine which abstracts the relevant and rigorous articles on diagnosis, prognosis, treatment, quality of care, and medical economics from over 30 general medical journals). All these databases are, or soon will be, available on line from local, national, and international networks such as the internet.

For our patient, the searches were conducted with Medline and the Knowledge Finder searching software. "Atrial fibrillation" and "cerebrovascular disorders" were entered as major medical subject headings and "randomised controlled trial" as a publication type selected from the "dictionaries" menu. The search was performed twice, once with "prognosis" entered as a freetext search parameter and a second time with "therapy" included. The years 1990-4 were searched and 10 articles were identified, of which eight seemed to contain the relevant information (two on prognosis⁸⁹ and six reporting randomised trials of therapy^101112131415 ). Five^1011121314 were available in the library.

The search was repeated for 1992-4 with "review" as the publication type, and one recent article was identified.¹⁶ The term "review" includes subjective reviews, systematic reviews, and meta-analyses. The newer term "meta-analysis" could have been used as a publication type to narrow the search but would have missed potentially useful reviews and systematic reviews, as well as meta-analyses that have not yet been classified as such in Medline.

The two articles on prognosis, four on therapy, and the review (in fact a meta-analysis) were then pulled from the library. The keyboard time taken for this search was 15 minutes. The ACP Journal Club, whose electronic version is currently being tested, has summarised these trials, and Cochrane reviews on the prevention and treatment of stroke will be available in 1995, but on this occasion we examined the evidence presented in conventional forms of clinical research publication.

While clinicians may make greater use of meta-analyses in the future, the ability to appraise critically publications of all types will remain an invaluable skill. Searches may fail to uncover well conducted and relevant meta-analyses and often it will be impractical for a busy clinician to conduct an independent systematic review of the literature each time a clinical question is generated. On these occasions the most effective strategy will be to seek out the best of the available literature and to appraise critically the evidence by using skills that can readily be learnt.

APPRAISING THE EVIDENCE

The third step is to evaluate, or appraise, the evidence for its validity and clinical usefulness. This step is crucial because it lets the clinician decide whether an article can be relied on to give useful guidance. Unfortunately, a large proportion of published medical research lacks either relevance or sufficient methodological rigour to be reliable enough for answering clinical questions.¹⁷ To overcome this, a structured but simple method, named "critical appraisal," developed by several teams working in North America and the United Kingdom, enables individuals without research expertise to evaluate clinical articles. Mastering critical appraisal entails learning how to ask a few key questions about the validity of the evidence and its relevance to a particular patient or group of patients. Its fundamentals can be learnt within a few hours in small tutorials, workshops, interactive lectures, and at the bedside by a wide range of users, including those without a biomedical background. This strategy has been developed for many different types of articles, and can be used to evaluate original articles about diagnosis, treatment, prognosis, quality of care, and economics as well as to evaluate reviews, overviews, and meta-analyses for their validity and applicability.

The table shows a typical set of critical appraisal questions for evaluating articles about treatment. Although they reflect common sense, the questions are not entirely self explanatory; some instruction is needed to help clinicians apply them to specific articles and individual patients. Self directed learning materials have been developed to help users apply different critical appraisal questions to the different sorts of clinical research articles on diagnosis, prognosis, therapy, quality of care, economic analysis, and screening. These materials include the JAMA series of user's guides and the text Clinical Epidemiology: A Basic Science for Clinical Medicine.¹⁸ Week long training workshops in evidence based medicine are held in various venues, but we have found that even people with limited experience can readily learn how to practise evidence based medicine in the context of their own clinical practice. As with any other skill, expertise and speed come with practice, and experienced practitioners can learn to appraise critically most articles in under 10 minutes, transforming themselves from passive, opinion based spectators to active, evidence based clinicians.

Yes Can't tell No
Are the results valid?
Was the assignment of patients to treatments randomised?	-
Were all patients who entered the trial properly accountedfor and attributed at its conclusion?	-
Was follow up complete?	-
Were patients analysed in the groups to which they were randomised?	-
Were patients, health workers, and study personnel blindedto treatment?	-
Were the groups similar at the start of the trial?	-
Aside from the experimental intervention, were the groups treated equally?	-
What are the results?	-
How large was the treatment effect?	-
How precise was the treatment effect?	-
Will the results help me care for my patients?	-
Can the results be applied to my patient care?	-
Were all clinically important outcomes considered?	-
Are the likely benefits worth the potential harms and costs?

This transformation is borne out in the critical appraisal of the evidence surrounding the management of the 77 year old woman with atrial fibrillation. The two articles on prognosis fulfil criteria for validity and applicability and reveal that our particular patient faces an 18% annual risk of stroke if left untreated.⁸⁹ Applying criteria given in the Users' guides to the medical literature: how to use an article about therapy or prevention,¹⁹²⁰ we decided that the articles we have pulled provide valid and applicable evidence. We used them to obtain the relative risk reduction of stroke due to treatment with warfarin, which is 70%. The annual risk of stroke for our patient without treatment was used, in conjunction with relative risk reduction obtained from the prognosis articles, to calculate the absolute risk reduction (ARR) of stroke attributable to anticoagulation with warfarin. This figure, which is 0.13, was then used to calculate the "number needed to treat" (NNT=1/ARR) with warfarin to save one stroke. Thus treating eight patients (1/0.13) for one year will prevent one stroke. The annual rate of major haemorrhage in patients receiving warfarin is 1%, so one patient in every hundred taking warfarin will experience a major bleed each year, and we therefore can expect to prevent about 13 strokes in patients such as ours with warfarin for every major bleed we will cause through such treatment. Although the benefit:risk ratio seems acceptable in this instance, we know that bleeding rates vary between centres and a higher local risk of intracranial haemorrhage might lead other clinicians and patients to a very different decision. The evidence will not automatically dictate patient care but will provide the factual basis on which decisions can be made, taking all aspects of patient care into consideration.

ACTING ON THE EVIDENCE

Having identified evidence that is both valid and relevant, clinicians can either implement it directly in a patient's care or use it to develop team protocols or even hospital guidelines. They can also use evidence to revolutionise continuing medical education programmes or audit. In our experience, implementing the evidence is best learned through group discussions, either on ward rounds or in other meetings of the clinical team in which members explore ways of incorporating the evidence into a patient's clinical management.

At the weekly firm meeting the evidence extracted from the critically appraised literature on warfarin was presented in a summarised form as a critically appraised topic by a junior member of the team (table). During the subsequent ward round the team discussed the evidence with the patient and she decided to start taking warfarin. It was decided to set a target international normalised ratio of 1.5-2.0, and her general practitioner, who asked for a copy of the critically appraised topic to accompany the discharge letter, agreed to monitor her treatment.

Other requirements for practising evidence based medicine

CLEAR DATA PRESENTATION

The ability to present published evidence quickly and clearly is crucial for clinical teams with little time and much information to absorb.²¹ Medical journals have led the way here with structured abstracts to help readers quickly retrieve key information. Such clarity and quickness are equally important for clinicians when they present evidence to their team. A preset, one page, user friendly summary such as the one developed by doctors in training at McMaster University in Ontario (unpublished data) can help this process and was the model for the critically appraised topic that appears in the table.

Added advantages in practising evidence based medicine

For individuals

Enables clinicians to upgrade their knowledge base routinely

Improves clinicians' understanding of research methods and makes them more critical in using data

Improves confidence in management decisions

Improves computer literacy and data searching techniques

Improves reading habits

For clinical teams

Gives team a framework for group problem solving and for teaching

Enables juniors to contribute usefully to team

For patients

More effective use of resources

Better communication with patients about the rationale behind management decisions

SENIOR SUPPORT

Support from senior clinicians is critical to the success of introducing evidence based medicine.²² Seniors who practice evidence based medicine are excellent role models for training newcomers and allocating questions according to the skills and time commitments of individual team members. Even when senior staff are themselves unfamiliar with evidence based medicine, their willingness to admit uncertainty, to encourage scepticism, and to be flexible can help the team to accommodate new evidence which may contradict previous assumptions and practice.

Does it work?

An evidence based approach to clinical care has been practised in many countries under various guises. In the structured form described above it attracts both support and criticism, often within the same hospital. The problem, ironically, is that the approach is difficult to evaluate.²³ It is a process for solving problems, and it will have different outcomes depending on the problem being solved. Trying to monitor all the possible outcomes would be impossible, especially since many are difficult to quantify. For example, a medical student who learns the importance of good research methodology through practising critical appraisal may later on carry out better research, but it would be hard either to quantify this or to link it directly to evidence based medicine.

None the less, evidence of the effectiveness of evidence based medicine is growing as it spreads to new settings. Short term trials have shown better and more informed clinical decisions following even brief training in critical appraisal,²⁴ and although graduates from traditional medical curriculums progressively decline in their knowledge of appropriate clinical practice, graduates of a medical school that teaches lifelong, self directed, evidence based medicine are still up to date as long as 15 years after graduation.²⁵ The review of the benefits and drawbacks of evidence based medicine that follows draws on our experience of teaching and practising evidence based medicine with clinicians and purchasers in Oxford.

Advantages

An immediate attraction of evidence based medicine is that it integrates medical education with clinical practice. We have observed that students and doctors who begin to learn evidence based medicine become adept at generating their own questions and following them through with efficient literature searches. For example, learners quickly learn to pick out good review articles and to use resources such as the ACP Journal Club when they are appropriate to the question being asked.²⁶

Another advantage of evidence based medicine is that it can be learnt by people from different backgrounds and at any stage in their careers. Medical students carrying out critical appraisals not only learn evidence based medicine for themselves but contribute their appraisals to their teams and update their colleagues. At the other extreme, seasoned clinicians can master evidence based medicine and transform a journal club from a passive summary of assigned journals into an active inquiry in which problems arising from patient care are used to direct searches and appraisals of relevant evidence to keep their practice up to date.

The evidence based approach is being taken up by non-clinicians as well. Consumer groups concerned with obtaining optimal care during pregnancy and childbirth are evolving evidence based patient choice. The critical appraisal skills for purchasers project in the former Oxford region involves teaching evidence based medicine to purchasers who have no medical training so that it can inform their decisions on purchasing.²⁷

A third attraction of evidence based medicine is its potential for improving continuity and uniformity of care through the common approaches and guidelines developed by its practitioners. Shift work and cross cover make communication between health workers both more important and more difficult. Although evidence based medicine cannot alter work relationships, in our experience it does provide a structure for effective team work and the open communication of team generated (rather than externally imposed) guidelines for optimal patient care. It also provides a common framework for problem solving and improving communication and understanding between people from different backgrounds, such as clinicians and patients or non-medical purchasers and clinicians.

Evidence based medicine can help providers make better use of limited resources by enabling them to evaluate clinical effectiveness of treatments and services. Remaining ignorant of valid research findings has serious consequences. For example, it is now clear that giving steroids to women at risk of premature labour greatly reduces infant respiratory distress and consequent morbidity, mortality, and costs of care,²⁸ and it is equally clear that aspirin and streptokinase deserve to be among the mainstays of care for victims of heart attack.

Disadvantages

Evidence based medicine has several drawbacks. Firstly, it takes time both to learn and to practise. For example, it takes about two hours to properly set the question, find the evidence, appraise the evidence, and act on the evidence, and for teams to benefit all members should be present for the first and last steps. Senior staff must therefore be good at time management. They can help to make searches less onerous by setting achievable contracts with the team members doing the searches and by ensuring that the question has direct clinical usefulness. These responsibilities of the team leader are time consuming.

Establishing the infrastructure for practising evidence based medicine costs money. Hospitals and general practices may need to buy and maintain the necessary computer hardware and software. CD-ROM subscriptions can vary from £250 to £2000 a year, depending on the database and specifications. But a shortage of resources need not stifle the adoption of evidence based medicine. The BMA provides Medline free of charge to members with modems, and Medline is also available for a small fee on the internet. Compared with the costs of many medical interventions (to say nothing of journal subscriptions and out of date texts), these costs are small and may recover costs many times their amount by reducing ineffective practice.

Inevitably, evidence based medicine exposes gaps in the evidence.⁴ This can be frustrating, particularly for inexperienced doctors. Senior staff can help to overcome this problem by setting questions for which there is likely to be good evidence. The identification of such gaps can be helpful in generating local and national research projects, such as those being commissioned by the York Centre for Reviews and Dissemination.²⁹

Another problem is that Medline and the other electronic databases used for finding relevant evidence are not comprehensive and are not always well indexed. At times even a lengthy literature search is fruitless. For some older doctors the computer skills needed for using databases regularly may also seem daunting. Although the evidence based approach requires a minimum of computer literacy and keyboard skills, and while these are now almost universal among medical students and junior doctors, many older doctors are still unfamiliar with computers and