How to improve healthcare improvement—an essay by Mary Dixon-Woods

Abstract

As improvement practice and research begin to come of age, Mary Dixon-Woods considers the key areas that need attention if we are to reap their benefits

[email protected]

In the NHS, as in health systems worldwide, patients are exposed to risks of avoidable harm ¹ and unwarranted variations in quality.²³⁴ But too often, problems in the quality and safety of healthcare are merely described, even “admired,”⁵ rather than fixed; the effort invested in collecting information (which is essential) is not matched by effort in making improvement. The National Confidential Enquiry into Patient Outcome and Death, for example, has raised many of the same concerns in report after report.⁶ Catastrophic degradations of organisations and units have recurred throughout the history of the NHS, with depressingly similar features each time.⁷⁸⁹

More resources are clearly necessary to tackle many of these problems. There is no dispute about the preconditions for high quality, safe care: funding, staff, training, buildings, equipment, and other infrastructure. But quality health services depend not just on structures but on processes.¹⁰ Optimising the use of available resources requires continuous improvement of healthcare processes and systems.⁵

The NHS has seen many attempts to stimulate organisations to improve using incentive schemes, ranging from pay for performance (the Quality and Outcomes Framework in primary care, for example) to public reporting (such as annual quality accounts). They have had mixed results, and many have had unintended consequences.¹¹¹² Wanting to improve is not the same as knowing how to do it.

In response, attention has increasingly turned to a set of approaches known as quality improvement (QI). Though a definition of exactly what counts as a QI approach has escaped consensus, QI is often identified with a set of techniques adapted from industrial settings. They include the US Institute for Healthcare Improvement’s Model for Improvement, which, among other things, combines measurement with tests of small change (plan-do-study-act cycles).⁸ Other popular approaches include Lean and Six Sigma. QI can also involve specific interventions intended to improve processes and systems, ranging from checklists and “care bundles” of interventions (a set of evidence based practices intended to be done consistently) through to medicines reconciliation and clinical pathways.

QI has been advocated in healthcare for over 30 years¹³; policies emphasise the need for QI and QI practice is mandated for many healthcare professionals (including junior doctors). Yet the question, “Does quality improvement actually improve quality?” remains surprisingly difficult to answer.¹⁴ The evidence for the benefits of QI is mixed¹⁴ and generally of poor quality. It is important to resolve this unsatisfactory situation. That will require doing more to bring together the practice and the study of improvement, using research to improve improvement, and thinking beyond effectiveness when considering the study and practice of improvement.

Uniting practice and study

The practice and study of improvement need closer integration. Though QI programmes and interventions may be just as consequential for patient wellbeing as drugs, devices, and other biomedical interventions, research about improvement has often been seen as unnecessary or discretionary,¹⁵¹⁶ particularly by some of its more ardent advocates. This is partly because the challenges faced are urgent, and the solutions seem obvious, so just getting on with it seems the right thing to do.

But, as in many other areas of human activity, QI is pervaded by optimism bias. It is particularly affected by the “lovely baby” syndrome, which happens when formal evaluation is eschewed because something looks so good that it is assumed it must work. Five systematic reviews (published 2010-16) reporting on evaluations of Lean and Six Sigma did not identify a single randomised controlled trial.^1718192021 A systematic review of redesigning care processes identified no randomised trials.²² A systematic review of the application of plan-do-study-act in healthcare identified no randomised trials.²³ A systematic review of several QI methods in surgery identified just one randomised trial.⁵⁶

The sobering reality is that some well intentioned, initially plausible improvement efforts fail when subjected to more rigorous evaluation.²⁴ For instance, a controlled study of a large, well resourced programme that supported a group of NHS hospitals to implement the IHI’s Model for Improvement found no differences in the rate of improvement between participating and control organisations.²⁵²⁶ Specific interventions may, similarly, not survive the rigours of systematic testing. An example is a programme to reduce hospital admissions from nursing homes that showed promise in a small study in the US,²⁷ but a later randomised implementation trial found no effect on admissions or emergency department attendances.²⁸

Some interventions are probably just not worth the effort and opportunity cost: having nurses wear “do not disturb” tabards during drug rounds, is one example.²⁹ And some QI efforts, perversely, may cause harm—as happened when a multicomponent intervention was found to be associated with an increase rather than a decrease in surgical site infections.³⁰

Producing sound evidence for the effectiveness of improvement interventions and programmes is likely to require a multipronged approach. More large scale trials and other rigorous studies, with embedded qualitative inquiry, should be a priority for research funders.

Not every study of improvement needs to be a randomised trial. One valuable but underused strategy involves wrapping evaluation around initiatives that are happening anyway, especially when it is possible to take advantage of natural experiments or design roll-outs.³¹ Evaluation of the reorganisation of stroke care in London and Manchester³² and the study of the Matching Michigan programme to reduce central line infections are good examples.³³³⁴

It would be impossible to externally evaluate every QI project. Critically important therefore will be increasing the rigour with which QI efforts evaluate themselves, as shown by a recent study of an attempt to improve care of frail older people using a “hospital at home” approach in southwest England.³⁵ This ingeniously designed study found no effect on outcomes and also showed that context matters.

Despite the potential value of high quality evaluation, QI reports are often weak,¹⁸ with, for example, interventions so poorly reported that reproducibility is frustrated.³⁶ Recent reporting guidelines may help,³⁷ but some problems are not straightforward to resolve. In particular, current structures for governance and publishing research are not always well suited to QI, including situations where researchers study programmes they have not themselves initiated. Systematic learning from QI needs to improve, which may require fresh thinking about how best to align the goals of practice and study, and to reconcile the needs of different stakeholders.³⁸

Using research to improve improvement

Research can help to support the practice of improvement in many ways other than evaluation of its effectiveness. One important role lies in creating assets that can be used to improve practice, such as ways to visualise data, analytical methods, and validated measures that assess the aspects of care that most matter to patients and staff. This kind of work could, for example, help to reduce the current vast number of quality measures—there are more than 1200 indicators of structure and process in perioperative care alone.³⁹

The study of improvement can also identify how improvement practice can get better. For instance, it has become clear that fidelity to the basic principles of improvement methods is a major problem: plan-do-study-act cycles are crucial to many improvement approaches, yet only 20% of the projects that report using the technique have done so properly.²³ Research has also identified problems in measurement—teams trying to do improvement may struggle with definitions, data collection, and interpretation⁴⁰—indicating that this too requires more investment.

Improvement research is particularly important to help cumulate, synthesise, and scale learning so that practice can move forward without reinventing solutions that already exist or reintroducing things that do not work. Such theorising can be highly practical,⁴¹ helping to clarify the mechanisms through which interventions are likely to work, supporting the optimisation of those interventions, and identifying their most appropriate targets.⁴²

Research can systematise learning from “positive deviance,” approaches that examine individuals, teams, or organisations that show exceptionally good performance.⁴³ Positive deviance can be used to identify successful designs for clinical processes that other organisations can apply.⁴⁴

Crucially, positive deviance can also help to characterise the features of high performing contexts and ensure that the right lessons are learnt. For example, a distinguishing feature of many high performing organisations, including many currently rated as outstanding by the Care Quality Commission, is that they use structured methods of continuous quality improvement. But studies of high performing settings, such as the Southmead maternity unit in Bristol, indicate that although continuous improvement is key to their success, a specific branded improvement method is not necessary.⁴⁵ This and other work shows that not all improvement needs to involve a well defined QI intervention, and not everything requires a discrete project with formal plan-do-study-act cycles.

More broadly, research has shown that QI is just one contributor to improving quality and safety. Organisations in many industries display similar variations to healthcare organisations, including large and persistent differences in performance and productivity between seemingly similar enterprises.⁴⁶ Important work, some of it experimental, is beginning to show that it is the quality of their management practices that distinguishes them.⁴⁷ These practices include continuous quality improvement as well as skills training, human resources, and operational management, for example. QI without the right contextual support is likely to have limited impact.

Beyond effectiveness

Important as they are, evaluations of the approaches and interventions in individual improvement programmes cannot answer every pertinent question about improvement.⁴⁸ Other key questions concern the values and assumptions intrinsic to QI.

Consider the “product dominant” logic in many healthcare improvement efforts, which assumes that one party makes a product and conveys it to a consumer.⁴⁹ Paul Batalden, one of the early pioneers of QI in healthcare, proposes that we need instead a “service dominant” logic, which assumes that health is co-produced with patients.⁴⁹

More broadly, we must interrogate how problems of quality and safety are identified, defined, and selected for attention by whom, through which power structures, and with what consequences. Why, for instance, is so much attention given to individual professional behaviour when systems are likely to be a more productive focus?⁵⁰ Why have quality and safety in mental illness and learning disability received less attention in practice, policy, and research⁵¹ despite high morbidity and mortality and evidence of both serious harm and failures of organisational learning? The concern extends to why the topic of social inequities in healthcare improvement has remained so muted⁵² and to the choice of subjects for study. Why is it, for example, that interventions like education and training, which have important roles in quality and safety and are undertaken at vast scale, are often treated as undeserving of evaluation or research?

How QI is organised institutionally also demands attention. It is often conducted as a highly local, almost artisan activity, with each organisation painstakingly working out its own solution for each problem. Much improvement work is conducted by professionals in training, often in the form of small, time limited projects conducted for accreditation. But working in this isolated way means a lack of critical mass to support the right kinds of expertise, such as the technical skill in human factors or ergonomics necessary to engineer a process or devise a safety solution. Having hundreds of organisations all trying to do their own thing also means much waste, and the absence of harmonisation across basic processes introduces inefficiencies and risks.¹⁴

A better approach to the interorganisational nature of health service provision requires solving the “problem of many hands.”⁵³ We need ways to agree which kinds of sector-wide challenges need standardisation and interoperability; which solutions can be left to local customisation at implementation; and which should be developed entirely locally.¹⁴ Better development of solutions and interventions is likely to require more use of prototyping, modelling and simulation, and testing in different scenarios and under different conditions,¹⁴ ideally through coordinated, large scale efforts that incorporate high quality evaluation.

Finally, an approach that goes beyond effectiveness can also help in recognising the essential role of the professions in healthcare improvement. The past half century has seen a dramatic redefining of the role and status of the healthcare professions in health systems⁵⁴: unprecedented external accountability, oversight, and surveillance are now the norm. But policy makers would do well to recognise how much more can be achieved through professional coalitions of the willing than through too many imposed, compliance focused diktats. Research is now showing how the professions can be hugely important institutional forces for good.⁵⁴⁵⁵ In particular, the professions have a unique and invaluable role in working as advocates for improvement, creating alliances with patients, providing training and education, contributing expertise and wisdom, coordinating improvement efforts, and giving political voice for problems that need to be solved at system level (such as, for example, equipment design).

Conclusion

Improvement efforts are critical to securing the future of the NHS. But they need an evidence base. Without sound evaluation, patients may be deprived of benefit, resources and energy may be wasted on ineffective QI interventions or on interventions that distribute risks unfairly, and organisations are left unable to make good decisions about trade-offs given their many competing priorities. The study of improvement has an important role in developing an evidence-base and in exploring questions beyond effectiveness alone, and in particular showing the need to establish improvement as a collective endeavour that can benefit from professional leadership.

Biography

Mary Dixon-Woods is the Health Foundation professor of healthcare improvement studies and director of The Healthcare Improvement Studies (THIS) Institute at the University of Cambridge, funded by the Health Foundation. Co-editor-in-chief of BMJ Quality and Safety, she is an honorary fellow of the Royal College of General Practitioners and the Royal College of Physicians. This article is based largely on the Harveian oration she gave at the RCP on 18 October 2018, in the year of the college’s 500th anniversary. The oration is available here: http://www.clinmed.rcpjournal.org/content/19/1/47 and the video version here: https://www.rcplondon.ac.uk/events/harveian-oration-and-dinner-2018

Footnote

This article is one of a series commissioned by The BMJ based on ideas generated by a joint editorial group with members from the Health Foundation and The BMJ, including a patient/carer. The BMJ retained full editorial control over external peer review, editing, and publication. Open access fees and The BMJ’s quality improvement editor post are funded by the Health Foundation.

Competing interests: I have read and understood BMJ policy on declaration of interests and a statement is available here: https://www.bmj.com/about-bmj/advisory-panels/editorial-advisory-board/mary-dixonwoods

Provenance and peer review: Commissioned; not externally peer reviewed.

¹ Hogan H Zipfel R Neuburger J Hutchings A Darzi A Black N. Avoidability of hospital deaths and association with hospital-wide mortality ratios: retrospective case record review and regression analysis. BMJ 2015; 351: h3239. 10.1136/bmj.h3239   26174149

² Stewart K Bray B Buckingham R. Improving quality of care through national clinical audit. Future Hosp J 2016; 3: 203 - 6. 10.7861/futurehosp.3-3-203   31098227

³ Castelli A Street A Verzulli R Ward P. Examining variations in hospital productivity in the English NHS. Eur J Health Econ 2015; 16: 243 - 54. 10.1007/s10198-014-0569-5   24566702

⁴ Majeed A Allwood D Foley K Bindman A. Healthcare outcomes and quality in the NHS: how do we compare and how might the NHS improve? BMJ 2018; 362: k3036. 10.1136/bmj.k3036   30006368

⁵ Sunstein CR. The real world of cost-benefit analysis: thirty-six questions (and almost as many answers). Columbia Law Rev 2014; 114: 167 - 211.

⁶ Healthcare Quality Improvement Partnership. NCEPOD common themes and recommendations. HQIP, 2018.

⁷ Walshe K Shortell SM. When things go wrong: how health care organizations deal with major failures. Health Aff (Millwood) 2004; 23: 103 - 11. 10.1377/hlthaff.23.3.103   15160808

⁸ Martin GP Dixon-Woods M. After Mid Staffordshire: from acknowledgement, through learning, to improvement. BMJ Qual Saf 2014; 23: 706 - 8. 10.1136/bmjqs-2014-003359   25038039

⁹ Walshe K. Gosport deaths: lethal failures in care will happen again. BMJ 2018; 362: k2931. 10.1136/bmj.k2931   29973369

¹⁰ Donabedian A. The quality of care. How can it be assessed? JAMA 1988; 260: 1743 - 8. 10.1001/jama.1988.03410120089033   3045356

¹¹ Himmelstein DU Ariely D Woolhandler S. Pay-for-performance: toxic to quality? Insights from behavioral economics. Int J Health Serv 2014; 44: 203 - 14. 10.2190/HS.44.2.a   24919299

¹² Woolhandler S Ariely D Himmelstein DU. Why pay for performance may be incompatible with quality improvement. BMJ 2012; 345: e5015. 10.1136/bmj.e5015   22893567

¹³ Berwick DM. Continuous improvement as an ideal in health care. N Engl J Med 1989; 320: 53 - 6. 10.1056/NEJM198901053200110   2909878

¹⁴ Dixon-Woods M Martin GP. Does quality improvement improve quality? Future Hosp J 2016; 3: 191 - 4. 10.7861/futurehosp.3-3-191   31098223

¹⁵ Ioannidis JPA Prasad V. Evaluating health system processes with randomized controlled trials. JAMA Intern Med 2013; 173: 1279 - 80. 10.1001/jamainternmed.2013.1044   23689271

¹⁶ Marshall M Pronovost P Dixon-Woods M. Promotion of improvement as a science. Lancet 2013; 381: 419 - 21. 10.1016/S0140-6736(12)61850-9   23374480

¹⁷ Glasgow JM Scott-Caziewell JR Kaboli PJ. Guiding inpatient quality improvement: a systematic review of Lean and Six Sigma. Jt Comm J Qual Patient Saf 2010; 36: 533 - 40. 10.1016/S1553-7250(10)36081-8   21222355

¹⁸ Mason SE Nicolay CR Darzi A. The use of Lean and Six Sigma methodologies in surgery: a systematic review. Surgeon 2015; 13: 91 - 100. 10.1016/j.surge.2014.08.002   25189692

¹⁹ Deblois S Lepanto L. Lean and Six Sigma in acute care: a systematic review of reviews. Int J Health Care Qual Assur 2016; 29: 192 - 208. 10.1108/IJHCQA-05-2014-0058   26959898

²⁰ Moraros J Lemstra M Nwankwo C. Lean interventions in healthcare: do they actually work? A systematic literature review. Int J Qual Health Care 2016; 28: 150 - 65. 10.1093/intqhc/mzv123   26811118

²¹ Amaratunga T Dobranowski J. Systematic review of the application of lean and six sigma quality improvement methodologies in radiology. J Am Coll Radiol 2016; 13: 1088 - 1095.e7. 10.1016/j.jacr.2016.02.033   27209599

²² van Leijen-Zeelenberg JE Elissen AMJ Grube K. The impact of redesigning care processes on quality of care: a systematic review. BMC Health Serv Res 2015; 16: 19. 10.1186/s12913-016-1266-0   26782132

²³ Taylor MJ McNicholas C Nicolay C Darzi A Bell D Reed JE. Systematic review of the application of the plan-do-study-act method to improve quality in healthcare. BMJ Qual Saf 2014; 23: 290 - 8. 10.1136/bmjqs-2013-001862   24025320

²⁴ Morgan DJ Diekema DJ Sepkowitz K Perencevich EN. Adverse outcomes associated with Contact Precautions: a review of the literature. Am J Infect Control 2009; 37: 85 - 93. 10.1016/j.ajic.2008.04.257   19249637

²⁵ Benning A Dixon-Woods M Nwulu U. Multiple component patient safety intervention in English hospitals: controlled evaluation of second phase. BMJ 2011; 342: d199. 10.1136/bmj.d199   21292720

²⁶ Benning A Ghaleb M Suokas A. Large scale organisational intervention to improve patient safety in four UK hospitals: mixed method evaluation. BMJ 2011; 342: d195. 10.1136/bmj.d195   21292719

²⁷ Ouslander JG Lamb G Tappen R. Interventions to reduce hospitalizations from nursing homes: evaluation of the INTERACT II collaborative quality improvement project. J Am Geriatr Soc 2011; 59: 745 - 53. 10.1111/j.1532-5415.2011.03333.x   21410447

²⁸ Kane RL Huckfeldt P Tappen R. Effects of an intervention to reduce hospitalizations from nursing homes: a randomized implementation trial of the INTERACT program. JAMA Intern Med 2017; 177: 1257 - 64. 10.1001/jamainternmed.2017.2657   28672291

²⁹ Westbrook JI Li L Hooper TD Raban MZ Middleton S Lehnbom EC. Effectiveness of a ‘Do not interrupt’ bundled intervention to reduce interruptions during medication administration: a cluster randomised controlled feasibility study. BMJ Qual Saf 2017; 26: 734 - 42. 10.1136/bmjqs-2016-006123   28232390

³⁰ Anthony T Murray BW Sum-Ping JT. Evaluating an evidence-based bundle for preventing surgical site infection: a randomized trial. Arch Surg 2011; 146: 263 - 9. 10.1001/archsurg.2010.249   21079110

³¹ Portela MC Pronovost PJ Woodcock T Carter P Dixon-Woods M. How to study improvement interventions: a brief overview of possible study types. BMJ Qual Saf 2015; 24: 325 - 36. 10.1136/bmjqs-2014-003620   25810415

³² Ramsay AIG Morris S Hoffman A. Effects of centralizing acute stroke services on stroke care provision in two large metropolitan areas in England. Stroke 2015; 46: 2244 - 51. 10.1161/STROKEAHA.115.009723   26130092

³³ Bion J Richardson A Hibbert P Matching Michigan Collaboration & Writing Committee. ‘Matching Michigan’: a 2-year stepped interventional programme to minimise central venous catheter-blood stream infections in intensive care units in England. BMJ Qual Saf 2013; 22: 110 - 23. 10.1136/bmjqs-2012-001325   22996571

³⁴ Dixon-Woods M Leslie M Tarrant C Bion J. Explaining Matching Michigan: an ethnographic study of a patient safety program. Implement Sci 2013; 8: 70. 10.1186/1748-5908-8-70   23786847

³⁵ Pearson M Hemsley A Blackwell R Pegg L Custerson L. Improving Hospital at Home for frail older people: insights from a quality improvement project to achieve change across regional health and social care sectors. BMC Health Serv Res 2017; 17: 387. 10.1186/s12913-017-2334-9   28583180

³⁶ Jones EL Lees N Martin G Dixon-Woods M. How well is quality improvement described in the perioperative care literature? A systematic review. Jt Comm J Qual Patient Saf 2016; 42: 196 - 206. 10.1016/S1553-7250(16)42025-8   27066922

³⁷ Ogrinc G Davies L Goodman D. SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process. BMJ Qual Saf 2016; 25: 986 - 92. 10.1136/bmjqs-2015-004411.  26369893

³⁸ Watson SI Dixon-Woods M Taylor CA. Revising ethical guidance for the evaluation of programmes and interventions not initiated by researchers. J Med Ethics 2019: medethics-2018-105263. 10.1136/medethics-2018-105263.  31481472

³⁹ Chazapis M Gilhooly D Smith AF. Perioperative structure and process quality and safety indicators: a systematic review. Br J Anaesth 2018; 120: 51 - 66. 10.1016/j.bja.2017.10.001   29397138

⁴⁰ Woodcock T Liberati EG Dixon-Woods M. A mixed-methods study of challenges experienced by clinical teams in measuring improvement. BMJ Qual Saf 2019 [ Epub ahead of print].. 10.1136/bmjqs-2018-009048   31446424

⁴¹ Davidoff F Dixon-Woods M Leviton L Michie S. Demystifying theory and its use in improvement. BMJ Qual Saf 2015; 24: 228 - 38. 10.1136/bmjqs-2014-003627   25616279

⁴² Michie S Johnston M Abraham C Lawton R Parker D Walker A “Psychological Theory” Group. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care 2005; 14: 26 - 33. 10.1136/qshc.2004.011155   15692000

⁴³ Lawton R Taylor N Clay-Williams R Braithwaite J. Positive deviance: a different approach to achieving patient safety. BMJ Qual Saf 2014; 23: 880 - 3. 10.1136/bmjqs-2014-003115   25049424

⁴⁴ Bradley EH Curry LA Spatz ES. Hospital strategies for reducing risk-standardized mortality rates in acute myocardial infarction. Ann Intern Med 2012; 156: 618 - 26. 10.7326/0003-4819-156-9-201205010-00003   22547471

⁴⁵ Liberati EG Tarrant C Willars J. How to be a very safe maternity unIT: An ethnographic study. Soc Sci Med 2019; 223: 64 - 72. 10.1016/j.socscimed.2019.01.035   30710763

⁴⁶ Syverson C. What determines productivity? J Econ Lit 2011; 49: 326 - 65 10.1257/jel.49.2.326.

⁴⁷ Bloom N Eifert B Mahajan A. Does management matter? Evidence from India. Q J Econ 2013; 128: 1 - 51 10.1093/qje/qjs044

⁴⁸ Cribb A. Improvement science meets improvement scholarship: reframing research for better healthcare. Health Care Anal 2018; 26: 109 - 23. 10.1007/s10728-017-0354-6   29270810

⁴⁹ Batalden P. Getting more health from healthcare: quality improvement must acknowledge patient coproduction. BMJ 2018; 362: k3617. 10.1136/bmj.k3617

⁵⁰ Aveling EL Parker M Dixon-Woods M. What is the role of individual accountability in patient safety? A multi-site ethnographic study. Sociol Health Illn 2016; 38: 216 - 32. 10.1111/1467-9566.12370   26537016

⁵¹ D’Lima D Archer S Thibaut BI Ramtale SC Dewa LH Darzi A. A systematic review of patient safety in mental health: a protocol based on the inpatient setting. Syst Rev 2016; 5: 203. 10.1186/s13643-016-0365-7   27894331

⁵² Boozary AS Shojania KG. Pathology of poverty: the need for quality improvement efforts to address social determinants of health. BMJ Qual Saf 2018; 27: 421 - 4. 10.1136/bmjqs-2017-007552   29511090

⁵³ Dixon-Woods M Pronovost PJ. Patient safety and the problem of many hands. BMJ Qual Saf 2016; 25: 485 - 8. 10.1136/bmjqs-2016-005232   26912578

⁵⁴ Martin GP Armstrong N Aveling EL Herbert G Dixon-Woods M. Professionalism redundant, reshaped, or reinvigorated? realizing the “third logic” in contemporary health care. J Health Soc Behav 2015; 56: 378 - 97. 10.1177/0022146515596353   26276676

⁵⁵ Freidson E. Professionalism, the third logic: on the practice of knowledge. University of Chicago Press, 2001.

⁵⁶ Nicolay CR Purkayastha S Greenhalgh A. Systematic review of the application of quality improvement methodologies from the manufacturing industry to surgical healthcare. Br J Surg 2012; 99: 324 - 35. 10.1002/bjs.7803   22101509