A Prescription For Enhancing Electronic Prescribing Safety

Abstract

While electronic prescribing has been shown to reduce medication errors and improve prescribing safety, it is vulnerable to error-prone processes. We review six intersecting areas in which changes to electronic prescribing systems, particularly in the outpatient setting, could transform medication ordering quality and safety. We recommend incorporating medication indications into electronic prescribing, establishing a single shared online medication list, implementing the transmission of electronic cancellation orders to pharmacies (cancelRx) to ensure that drugs are safely and reliably discontinued, implementing standardized structured and codified prescription instructions, reengineering clinical decision support, and redesigning electronic prescribing to facilitate the ordering of nondrug alternatives.

Headnote

ABSTRACT While electronic prescribing has been shown to reduce medication errors and improve prescribing safety, it is vulnerable to error-prone processes. We review six intersecting areas in which changes to electronic prescribing systems, particularly in the outpatient setting, could transform medication ordering quality and safety. We recommend incorporating medication indications into electronic prescribing, establishing a single shared online medication list, implementing the transmission of electronic cancellation orders to pharmacies (cancelRx) to ensure that drugs are safely and reliably discontinued, implementing standardized structured and codified prescription instructions, reengineering clinical decision support, and redesigning electronic prescribing to facilitate the ordering of nondrug alternatives.

Ensuring that medication use is free from preventable harm is a World Health Organization (WHO) top priority for global patient safety,1 and ordering medications electronically is a key strategy for advancing medication safety. If the WHO goal is to be achieved, however, medication ordering must be conceptualized as a system, and key failure modes and opportunities for improvement must be identified. Current systems for electronic prescribing-particularly in the outpatient setting, in which most drugs are ordered-are prone to failures that often cause patient harm.

We identified six areas in which improvements in the quality and safety of outpatient electronic prescribing particularly could be made. Such improvements will require understanding current problems, appreciating how potential solutions can enhance prescribing safety, and identifying policy barriers and ways to overcome them. Solutions to current error-prone processes exist in each area, but policy efforts will be required to advance their implementation, safety, and effectiveness. Our recommendations, discussed in detail below, are to incorporate medication indications into electronic prescribing, establish a single shared online medication list, implement the use of electronic cancellation orders to pharmacies for discontinued drugs, implement standardized structured and codified patient instructions, reengineer clinical decision support, and redesign electronic prescribing to make it easier to order nondrug alternatives.

Indications Incorporated Into Electronic Prescribing

Drug indications represent the intersection between the drug, the patient, and health policy. Thirty years ago the National Coordinating Council for Medication Error Reporting and Prevention recommended that "prescription orders should include a brief notation of purpose (e.g. for cough), unless considered inappropriate by the prescriber."2 Despite continuing widespread recommendations, especially from the pharmacy and medication-safety communities, that reiterate the value of including medication indications in the prescription, only a small percentage of medications ordered include the indication either in the prescription, on the label, or both. For example, at Partners Healthcare, only 0.7 percent of the 5.4 million outpatient medications ordered annually in the Epic electronic medical record (EMR) prescribing system include an indication entered from the embedded list of drug-specific indications. Another 16.7 percent had a billing diagnosis code entered, but this does not appear on the patient's pharmacy medication label and is often unrelated to the drug's indication.

As part of a health information technology (IT) patient safety project funded by the Agency for Healthcare Research and Quality, our team convened a diverse group of key stakeholders to synthesize the rationale and challenges for including indications in electronic prescriptions and to design a new way to electronically prescribe medications that starts with the indication rather than the drug.2,3 Both conceptually and practically, this novel approach to electronic prescribing has the potential to transform prescribing and enhance team communication and patient education by eliminating the need for prescribers to add the indication manually. Beyond work-flow efficiency, there are broad safety and policy benefits.

First, prescribers would be guided in selecting drugs of choice. There are simply too many new drugs, changing regimens, and recommendations for clinicians to reliably know and recall. Also, prescribers cannot keep track of all the patient-specific factors that influence drug and regimen choice (such as current and prior drugs taken, relevant lab test results, and drug formulary status). The computer "knows" this information and can suggest the best choice accordingly.

Second, error-free dosing regimens would be composed as default orders specific for that indication. As an example, methotrexate ordered for rheumatoid arthritis would default to 7.5 mg once weekly; the same drug ordered for choriocarcinoma would default to 15-30 mg daily for five days. The frequently reported harmful error of giving this potentially toxic medication daily rather than weekly could be eliminated with indications-based prescribing.4

Third, communication between prescribers and pharmacists would be enhanced, supporting pharmacists' counseling role. To counsel patients, pharmacists must be able to explain the reason for taking a given drug. Pharmacy systems are already equipped to receive coded indications and add them to medication labels.

Fourth, instruction of patients, and their understanding and error-free medication adherence, would be improved. Patients are more likely to take medications if they understand the reasons. Evidence shows that more than 90 percent of patients prefer medication labels that include the indication.5

Fifth, entering the indication as the first step of prescribing provides a forcing function that prevents the display of a wrong drug for that indication. For instance, hydroxyzine (used for itching or anxiety) and hydralazine (used for hypertension) are occasionally confused, but the former would never appear on a list of possible medications when hypertension is selected as the indication. If an error occurs in which a patient is identified incorrectly, and the patient is given a prescription for an indication they don't recognize-such as "take once daily for gout"- the patient, family member, or pharmacist could easily ask why the medication was ordered, if the patient does not have gout. Furthermore, alerts could be triggered when a drug was ordered for a problem not on the patient's problem list, and there is evidence that sending an alert for such mismatches can detect and prevent errors.6,7

Finally, including indications in the prescription would enable medication reconciliation and deprescribing. Sorting medications by indication facilitates easier recognition of duplicates in therapy and makes it easier to discontinue medications that are no longer needed-because decisions to stop a drug require knowing why it was started.8

Various barriers will need to be overcome, including curating the "drugs of choice" content and addressing privacy concerns (for example, a patient who does not wish to have "for gonorrhea" on their medication label). However, these are all achievable with modest implementation efforts, as spelled out elsewhere.2,3

A Single Shared Medication List

The absence of a single unified "source of truth" about patients' current medications causes errors and inefficiencies, with reported discrepancy rates of25-70 percent.9,10 The enormous effort involved in reconciling medications in hospitals, clinics, nursing homes, and pharmacies; during home health visits; andbypatients themselves is largely a work-around for the lack of a single shared medication list.11,12 Reconciled outpatient medication lists are out of date as soon as the patient has a new prescription written by a specialist, is admitted to the hospital, or stops taking a medication due to a side effect.

The benefits of prescribing from a unified medication source extend beyond the value of an inherently reconciled list. If all prescribing software were interfaced with a single online database, access to patients' current medication lists would not be constrained by geography, institution, practice type, pharmacy, or insurance plan. Prescribers would not have to worry about which pharmacy a patient goes to, and pharmacies would be able to fill any active prescription (with checks to avoid duplicate filling). Primary care clinicians would not have to review medications to reconcile disparate lists but could instead concentrate on shared decision making with the patient regarding which medications on the list should be continued or changed.

An active medication list is not just a listing of all prescribed medications; it should reflect the medications the patient is actually taking.13 We envision patients routinely reviewing their online lists, noting any medications they are not taking or are taking differently than prescribed. Such departures could then be visually highlighted, so clinicians could move these to a list of inactive medications, correct the dosage, or counsel the patient.

Problems related to communication of health information and medications have been widely acknowledged, but they have been mainly conceptualized as problems of interoperability between health information systems. In the US there is so much fragmentation of electronic information stored in disconnected systems that promoting interoperabilityhas become a top priority for the federal health IT agency, the Office of the National Coordinator for Health Information Technology (ONC).14 However, these efforts to expand data-sharing traffic pale in comparison to those that would be needed to achieve the more meaningful goal of reaching a more reliable destination-where medication information is automatically comprehensive and current. Creating a single shared list would eliminate the need for much of the current faxing and reconciling between different providers and at transitions of care.

However, posting and sharing patients' medication lists in a centralized site inevitably raises privacy and security concerns. Such concerns have inhibited mainstream acceptance of this idea, and highly publicized data breaches (for example, Facebook and Equifax) do little to reassure the public. Nonetheless, most health care providers are already moving in the direction of secure cloud-based storage for health data, and other countries, such as Denmark, are leading the way in creating and using centralized databases.15 Housing data on a single secure site could provide a level of security that far exceeds the current protections under which individual providers, some with limited IT resources, store medication data.16 A system built on a foundation of safeguards in the Health Insurance Portability and Accountability Act (HIPAA) of 1996- enhanced by more recent and stringent regulatory approaches, such as those advanced by the European Union's General Data Protection Regulation and operationalized by a national investment in physical security and encryption (beyond what individual organizations, especially smaller ones, are now capable of)-should overcome many of the public's fears. In addition to technical protections, we envision a series of organizational and administrative protections similar to those outlined in the ONC's Shared Nationwide Interoperability Roadmap,14 including well-defined access management, identity proofing, credentials, authentication, authorization, audit trails, and data transmission protection. Finally, patients should be able to control who has access to their data or opt out of the system entirely. Such patients could elect to have their data stored locally in doctors' offices or pharmacies, as is currently done.

CancelRx: Ensuring That Drugs Are Safely And Reliably Discontinued

Until such a centralized medication list exists, there needs to be a reliable mechanism to ensure that when a prescriber discontinues a medication in their local system (for instance, because of an erroneous prescription, adverse drug reaction, or resolved problem), it is not dispensed by the pharmacy. Placing an order to discontinue a medication in electronic prescribing systems does not typically result in the transmission of a discontinuation order to the pharmacy.17-19 Instead, the order remains active in the pharmacy computer, posing a risk for erroneous dispensing the next time a refill is due. Many pharmacies have automated systems that call patients to issue refill reminders. Hopefully, a patient would know not to take a discontinued medication, but the potential for error (and waste) is significant: Reported erroneous dispensing rates are as high as 2-10 percent.18-20

Recognizing this danger, the National Council for Prescription Drug Programs (NCPDP) standards group developed CancelRx, an electronic message system that communicates when a medication has been discontinued. The functionality has been implemented by Surescripts, the leading health information network that electronically transmits medication orders between prescribers and pharmacies. However, to communicate discontinuations, the CancelRx functionality needs to be activated at both the electronic prescriber and pharmacy ends. Although the NCPDP SCRIPT Standard has supported CancelRx since 2006, implementation has suffered from a "chicken-and-egg" phenomenon: Pharmacies were reluctant to implement it because so few EMRs were configured to send CancelRx messages, and EMR vendors and health care organizations did not see value in implementing it because so few pharmacies had done so. This is now beginning to change, with several large EMR vendors and pharmacy chains developing the capability to transmit and receive discontinuation messages. In 2017, 3.8 million CancelRx messages were generated. However, currently only 11.5 percent of EMR vendors (104 of 908 vendors) are listed by Surescripts as certified to transmit CancelRx messaging-and for those that are, few of their users have turned on this functionality.21

Moreover, successful and safe implementation of the CancelRx functionality requires more than simply flipping a switch to enable the flow of messages. Critical work-flow challenges at both the prescriber and pharmacy ends will need to be overcome.22-24 These issues include whether and how discontinuation messages are generated (manually or automatically), how far back prescribing systems should reach in notifying pharmacies that an old prescription is being discontinued to trigger CancelRx messages, how to know which pharmacy accepts CancelRx messages, which pharmacies should receive the message (the original or the current pharmacy), and howto avoid extra work and "alert fatigue" from the need to respond to "denial messages" sent back from pharmacies that reject a CancelRx message because they cannot identify the patient or prescription in their system. This latter issue is not trivial: Each CancelRx message must "find" the original prescription, which is not always successful. Either a confirmation message of a successful match is sent, or a failure-to-match message is generated. These messages in turn must be processed back at the prescriber end, requiring EMR and work-flow configurations, complexities, and manual interventions.

How to do all of this without adding clutter to clinicians' in-boxes and requiring manual reconciliation, while ensuring that the status of these medications is visible at a glance to all parties, are details onlybeginning to be addressed. There is potential not only for additional burden and complexity but also for legal risks, should a critical prescription fail to be discontinued. Thus, the safe cancellation of electronically prescribed medications illustrates the need for a thoughtful multidisciplinary process that engages prescribers, pharmacists, and IT specialists in product design, user training, and usability testing to understand ways to optimize this process.

Structured And Codified Prescription Instructions

The transmission of complete and unambiguous prescription instructions, known as sigs, to communicate how a prescriber intends a patient to take a medication is fundamental for safe and effective care and essential for accurate prescription labeling, appropriate pharmacist counseling, and optimal medication use.

Prescription instructions are typically communicated via a 140-character, free-text field in the ambulatory e-prescription message using the NCPDP SCRIPT Standard, version 10.6. Although this standard also provides the ability to transmit a structured and codified sig data segment, the e-prescribing industry has been slow to adopt this critical new functionality. In 2017 only 2 percent of e-prescriptions included this data segment; for the remaining 98 percent, only free-text sigs were sent.21

Substantial variation exists in prescribergenerated free-text strings. More than half of all ambulatory sigs can be categorized into twenty-five distinct sig concepts and thus should be easy targets for structured sigs. With free-text sigs, a seemingly simple concept-such as "Take one tablet by mouth once daily"-can be represented in 832 different ways. Furthermore, freetext data strings can be ambiguous, conflicting, or confusing.25 A recent study suggests that more than 10 percent of ambulatory sigs contain quality problems that could necessitate a pharmacy clarification call or threaten patient safety.25

Most EMR and pharmacy applications have deployed their own proprietary tools for populating sigs. While these tools facilitate clinical decision support and safety checks for internal use, many downstream problems result from the failure to use and transmit structured and codified sigs using NCPDP standard format. The receipt of a free-text sig string at the pharmacy necessitates manual transcription and mapping into its system. In addition to huge inefficiencies, the current process is fraught with potential for transcription errors. Such problems occur at both the pharmacy and the prescribing ends. When renewal requests return from the pharmacy to prescribers, they are unstructured and then need to be manually reentered in the prescribers' computerized provider order entry section of the EMR. Industrywide implementation of this structured and coded sig functionality would improve interoperability and usability, reduce errors, and support additional clinical decision support safety checks.

Redesigned Clinical Decision Support

In the context of medications, clinical decision support refers to health IT systems designed to support clinicians with recommendations, alerts, and reminders to ease the cognitive burdens of keeping track of information and recom- mendations related to patients and their drugs and lab data. One key advantage of electronic ordering is the ability to provide clinical decision support when a prescription is entered. Considerable evidence suggests that medication-related clinical decision support can reduce errors and improve safety. For example, systematic reviews show reductions in prescribing error rates, adverse event rates, and harms from medications when decision support systems are used.26-29

Building on a framework introduced by Gilad Kuperman and coauthors,30 we conceptualized medication-related clinical decision support in electronic prescribing systems as basic, more advanced, and emerging (exhibit 1). For example, at a basic level, these clinical decision support systems could check for drug allergies and duplicate therapies, while at more advanced and emerging levels, they could provide advanced dosing guidance and continuously weigh the benefits and risks of various treatments. At each level, there are numerous challenges to overcome to more accurately, meaningfully, and effectively design and implement clinical decision support. In the past decade, while medicationrelated clinical decision support has become more widely deployed, a variety of issues have surfaced, including serious problems with alert fatigue, loss of provider autonomy, and difficulties in accurately designing and maintaining clinical decision support systems.31

Several members of our research team recently evaluated the effect of transitioning from a selfdeveloped EMR with a tightly tailored clinical decision support knowledge base to a commercial system. After the transition, alert burden increased sixfold, and the acceptance of the most serious medication-related clinical decision support alerts, previously above 90 percent, fell to below 10 percent. Another recently reported challenge with clinical decision support systems is their propensity to malfunction.32'33 Because these systems rely on complex knowledge bases and clinical vocabularies, changes made to EMR systems (such as changes to lab result configuration or medication codes) can cause an alert to be triggered inappropriately or fail to fire- malfunctions that can be difficult to detect.

Clinical decision support is much more than alerts and reminders.34 Where possible, we recommend using both upstream clinical decision support, such as indications-based prescribing guided by patient characteristics (for example, renal function and age) to steer users to pick the optimal medication and dosing from the outset, and anticipatory guidance to ensure follow-up monitoring for potential adverse reactions. Knowledge bases for these systems must be accurate and up-to-date, as users quickly lose faith in clinical decision support when they receive inaccurate suggestions. Finally, clinical decision support should be available not just to physicians but also to other team members such as nurses, pharmacists, and patients.

Facilitating The Ordering Of Nondrug Alternatives

Electronic prescribing is conceptually and, in practice, often narrowly viewed as the electronic ordering of drugs. This bias toward pharmacologic treatment, deeply entrenched in medical care systems worldwide, is reinforced by the design of current electronic prescribing systems. The difficulty in ordering nondrug alternatives is compounded by the fact that carrying out many of these treatments is not as simple as swallowing a pill once or twice daily. However, there is widespread evidence from well-designed trials that interventions suchas diet, exercise, physical therapy, counseling, cognitive behavioral therapyand other alternative or social support system changes can be both more effective and safer than drug therapy.35,36

Of course, the bias in favor of drug therapy did not originate with, nor will itbe overcome simply by changes in the design of, electronic prescribing systems. But consider the possibilities for facilitating the choice of nondrug treatments if they could be given equal prominence in the display, work flow, and ease of prescribing. If we redesign electronic prescribing to start with the clinical indication, then biases favoring drug treatment could be counterbalanced-because in many cases, the best treatments for an indication may be nonpharmacologic. For instance, when a treatment for insomnia is prescribed, sleep hygiene measures could be as easily prescribed as a sleep medication. Displaying these choices to permit their ordering with a single click elevates prescribing nondrug alternatives to its appropriate place in electronic prescribing.

We recognize that ordering nondrug treatments is often not as simple as creating a structured drug order. Although drug prescriptions also must be accompanied by patient instructions, instructions for nondrug treatments such as diet change, back exercises, and smoking cessation are often more extensive. They require greater patient customization, including to patients' demographics, health literacy, or clinical problems, and they face economic obstacles such as a lack of insurance coverage. Nondrug treatments may even need to be accompanied by instructional video links, and referrals to a dietitian or physical therapy may need to be included in the prescription order. While most electronic prescribing systems do include ways to provide video instructions and referrals, they are rarely integrated into electronic prescribing work flows.

There are several reasons for giving priority to nondrug treatments as an alternative to drugs. First, in many situations, an initial trial of a conservative therapy is recommended, and such "stepped therapy" can be facilitated by listing a nondrug therapy as the first electronic prescribing choice. Second, the availability of nondrug alternatives together with drug therapies eliminates the need to search for alternatives and thus increases prescribers' efficiency. Third, having the ability to more easily prescribe conservative, nondrug alternatives can support clinicians in adhering to various recommendations of Choosing Wisely, an initiative of the American Board of Internal Medicine (ABIM) Foundation to promote conversation between clinicians and patients; exercising stewardship over limited societal resources; and protecting patients against inappropriate antimicrobials or inappropriate drug treatments for elderly patients. Finally, patients will see that nondrug alternatives are valued on par with drugs. The message that there is only one way to treat a problem-with drugs-is not lost on patients who see ads exclusively targeting drugs on TV and then in the exam room see these as the only options that appear on their doctors' computer screens.While developing the content for such nondrug therapy recommendations faces barriers similar to those of providing indications-based drugs of choice (including counterbalancing the financial interest of pharmaceutical manufacturers in increasing prescriptions for their drugs), from a policy perspective it is advisable to prospectively develop recommendations that carefully weigh best options and choices and deliver this guidance at the moment the prescription is being written. Investing in this up-front deliberative process is likely to provide more cautious advice than the current system in which clinicians' ordering is guided by less evidence-based ad hoc selection of heavily promoted newer, and often expensive, drugs.

Conclusion

These six areas for improvement have the potential to radically transform the safety of electronic prescribing. If these actions are coupled with enhanced systems for monitoring patients on medications, getting feedback from their experiences and outcomes, as well as mechanisms to monitor the functioning of these redesigned systems overall, we can anticipate moving to a world where medications are radically safer to order and use. ?

Gordon Schiff and Adam Wright evaluated Medware CDS software (not mentioned in or relevant to this article). Ajit Dhavle is former vice president of Clinical Quality at Surescripts, LLC. This work was funded in part by an Agency for Healthcare Research and Quality health information technology safety grant (Grant No. R01HS23694) for the Indications Based Prescribing Project and by the Gordon and Betty Moore Foundation's Conservative Prescribing: Prescriber Profiling and Education Project. The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Footnote

NOTES

1 Donaldson LJ, Kelley ET, DhingraKumar N, Kieny MP, Sheikh A. 3 Medication without harm: WHO's third global patient safety challenge. Lancet. 2017;389(10080):1680-1.

2 Kron K, Myers S, Volk L, Nathan A, Neri P, Salazar A, et al. Incorporating medication indications into the prescribing process. Am J Health Syst Pharm. 2018;75(11):774-83.

3 Schiff GD, Seoane-Vazquez E, Wright A. Incorporating indications into medication ordering-time to enter the age of reason. N Engl J Med. 2016;375(4):306-9.

4Kelly K, Vaida AJ. Beware of erroneous daily oral methotrexate dosing! Pharmacy Times [serial on the Internet]. 2005 Feb 1 [cited 2018 Oct 10]. Available from: https://www .pharmacytimes.com/publications/ issue/2005/2005-02/2005-02-9297

5 Zargarzadeh AH, Law AV. Design and test of preference for a new prescription medication label. Int J Clin Pharm. 2011;33(2):252-9.

6 Galanter WL, Bryson ML, Falck S, Rosenfield R, Laragh M, Shrestha N, etal. Indication alerts intercept drug name confusion errors during computerized entry of medication orders. PLoS One. 2014;9(7):e101977.

7 Walton SM, Galanter WL, Rosencranz H, Meltzer D, Stafford RS, Tiryaki F, et al. A trial of inpatient indication based prescribing during computerized order entry with medications commonly used off-label. Appl Clin Inform. 2011; 2(1):94-103.

8 Linsky A, Meterko M, Stolzmann K, Simon SR. Supporting medication discontinuation: provider preferences for interventions to facilitate deprescribing. BMC Health Serv Res. 2017;17(1):447.

9 Cornish PL, Knowles SR, Marchesano R, Tam V, Shadowitz S, Juurlink DN, et al. Unintended medication discrepancies at the time of hospital admission. Arch Intern Med. 2005;165(4):424-9.

10 Mekonnen AB, Abebe TB, McLachlan AJ, Brien JA. Impact of electronic medication reconciliation interventions on medication discrepancies at hospital transitions: a systematic review and meta-analysis. BMC Med Inform Decis Mak. 2016;16(1):112.

11 Greenwald JL, Halasyamani L, Greene J, LaCivita C, Stucky E, Benjamin B, et al. Making inpatient medication reconciliation patient centered, clinically relevant, and implementable: a consensus statement on key principles and necessary first steps. J Hosp Med. 2010; 5(8):477-85.

12 Pevnick JM, Shane R, Schnipper JL. The problem with medication reconciliation. BMJ Qual Saf. 2016; 25(9):726-30.

13 Pandolfe F, Wright A, Slack WV, Safran C. Rethinking the outpatient medication list: increasing patient activation and education while architecting for centralization and improved medication reconciliation. J Am Med Inform Assoc. 2018; 25(8):1047-53.

14 Office of the National Coordinator for Health Information Technology. Connecting health and care for the nation: a Shared Nationwide Interoperability Roadmap: final version 1.0 [Internet]. Washington (DC): ONC; [cited 2018 Oct 10]. Available from: http://www.healthit.gov/ sites/default/files/hie-inter operability/nationwide-inter operability-roadmap-finalversion-1.0.pdf

15 Samadbeik M, Ahmadi M, Sadoughi F, Garavand A. A copmarative review of electronic prescription systems: lessons learned from developed countries. J Res Pharm Pract. 2017; 6(1):3-11.

16 Orcutt M. Your doctor's office is vulnerable to hackers, but Congress could change that. MIT Technology Review [serial on the Internet]. 2017 Jun 12 [cited 2018 Oct 10]. Available from: https://www.technology review.com/s/608052/yourdoctors-office-is-vulnerable-tohackers-but-congress-couldchange-that/

17 Fischer S, Rose A. Responsible e-prescribing needs e-discontinuation. JAMA. 2017;317(5):469-70.

18 Allen AS, Sequist TD. Pharmacy dispensing of electronically discontinued medications. Ann Intern Med. 2012;157(10):700-5.

19 Baranowski PJ, Peterson KL, StatzPaynter JL, Zorek JA. Incidence and cost of medications dispensed despite electronic medical record discontinuation. J Am Pharm Assoc (2003). 2015;55(3):313-9.

20 Eguale T, Verma A, Seoane-Vazquez E, Bates DW, Tamblyn R, Schiff GD. Pharmacy drug dispensing after physician discontinuation orders. Abstracts of the 31st International Conference on Pharmacoepidemiology and Therapeutic Risk Management, August 22-26, 2015, Boston, Massachusetts, USA. Pharmacoepidemiol Drug Saf. 2015;24(S1): 203-4.

21 Surescripts. 2017 national progress report [Internet]. Arlington (VA): Surescripts; c 2018 [cited 2018 Oct 10]. Available from: https://sure scripts.com/docs/default-source/ national-progress-reports/ 2151_npr_2017_finalB.pdf

22 Dhavle AA, Yang Y, Rupp MT, Singh H, Ward-Charlerie S, Ruiz J. Analysis of prescribers' notes in electronic prescriptions in ambulatory practice. JAMA Intern Med. 2016; 176(4):463-70.

23 Wang CJ, Patel MH, Schueth AJ, Bradley M, Wu S, Crosson JC, et al. Perceptions of standards-based electronic prescribing systems as implemented in outpatient primary care: a physician survey. J Am Med Inform Assoc. 2009;16(4):493-502.

24 Hammond WE. The making and adoption of health data standards. Health Aff (Millwood). 2005; 24(5):1205-13.

25 Yang Y, Ward-Charlerie S, Dhavle AA, Rupp MT, Green J. Quality and variability of patient directions in electronic prescriptions in the ambulatory care setting. J Manag Care Spec Pharm. 2018;24(7):691-9.

26 Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med. 2003;163(12):1409-16.

27 Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, et al. Effect of clinical decisionsupport systems: a systematic review. Ann Intern Med. 2012; 157(1):29-43.

28 Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293(10): 1223-38.

29 Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005;330(7494):765.

30 Kuperman GJ, Bobb A, Payne TH, Avery AJ, Gandhi TK, Burns G, et al. Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc. 2007;14(1): 29-40.

31 Ash JS, Sittig DF, Campbell EM, Guappone KP, Dykstra RH. Some unintended consequences of clinical decision support systems. AMIA Annu Symp Proc. 2007:26-30.

32 Wright A, Hickman TT, McEvoy D, Aaron S, Ai A, Andersen JM, et al. Analysis of clinical decision support system malfunctions: a case series and survey. J Am Med Inform Assoc. 2016;23(6):1068-76.

33 Wright A, Ai A, Ash J, Wiesen JF, Hickman TT, Aaron S, et al. Clinical decision support alert malfunctions: analysis and empirically derived taxonomy. J Am Med Inform Assoc. 2018;25(5):496-506.

34 Bates DW, Kuperman GJ, Wang S, Gandhi T, Kittler A, Volk L, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc. 2003;10(6):523-30.

35 Schiff GD, Galanter WL, Duhig J, Lodolce AE, Koronkowski MJ, Lambert BL. Principles of conservative prescribing. Arch Intern Med. 2011;171(16):1433-40.

36 Lean ME, Leslie WS, Barnes AC, Brosnahan N, Thom G, McCombie L, et al. Primary care-led weight management for remission of type 2 diabetes (DiRECT): an open-label, cluster-randomised trial. Lancet. 2018;391(10120):541-51.