Chapter 8. Computer Adverse Drug Event (ADE) Detection and Alerts

Tejal K. Gandhi, M.D., M.P.H.
David W. Bates, M.D., M.Sc.
Harvard Medical School

Background

Adverse drug events (ADEs) occur in both inpatient and outpatient settings.^1,2 Most institutions use spontaneous incident reporting to detect adverse events in general, and ADEs in particular. Spontaneous incident reporting relies exclusively on voluntary reports from nurses, pharmacists and physicians focused on direct patient care. However, spontaneous reporting is ineffective, identifying only one in 20 ADEs.³ Efforts to increase the frequency of spontaneous reporting have had only a minor impact.

Several studies demonstrate the effectiveness of using computerized detection and alert systems (referred to as computer "monitors") to detect ADEs. In 1991, Classen et al⁴ published information about a computerized ADE monitor that was programmed to identify signals—in effect mismatches of clinical information—that suggested the presence of an ADE. The signals included sudden medication stop orders, antidote ordering, and certain abnormal laboratory values.⁴ The computerized signals were then evaluated by a pharmacist who determined whether an ADE had occurred. Based on the rules of this study, Jha et al developed a similar monitor that identified approximately half the ADEs identified by chart review, at much lower cost.⁵ Similarly, Bowman and Carlstedt used the Regenstrief Medical Record System to create a computerized inpatient ADE detection system.⁶ Compared to the "gold standard" of chart review, the monitor had 66% sensitivity and 61% specificity, with a positive predictive value of 0.34. Finally, one community hospital implemented an ADE monitor with triggers that were reviewed by pharmacists who then contacted physicians to make appropriate regimen changes. This study identified opportunities to prevent patient injury at a rate of 64/1000 admissions.⁷ These studies and others demonstrate the potential value of computer monitors, especially when linked to effective integrated information systems. While monitors are not yet widely used, they offer an efficient approach for monitoring the frequency of ADEs on an ongoing basis, and the Health Care Financing Administration is considering mandating them.⁸

Practice Description

Computerized ADE alert monitors use rule sets to search signals that suggest the presence of adverse drug events. The most frequently studied rule sets (or "triggers") are those that search for drug names (e.g., naloxone, kayexalate), drug-lab interactions (e.g., heparin and elevated PTT) or lab levels alone (e.g., elevated digoxin levels) that frequently reflect an ADE. Simple versions can be implemented with pharmacy and laboratory data alone, although the yield and positive predictive value of signals is higher when the 2 databases are linked.

Further refinements include searches for International Classification of Diseases (ICD-9) codes, and text-searching of electronic nursing bedside charting notes or outpatient notes for drug-symptom combinations (e.g., medication list includes an angiotensin converting enzyme inhibitor and the patient notes mention "cough"). Although these refinements do increase the yield of monitors, they require linkage to administrative databases or electronic medical records.

The information captured with computer monitors is used to alert a responsible clinician or pharmacist, who can then change therapy based on the issue in question. Systems are designed to alert the monitoring clinician in various ways. Alerts can go to one central location (e.g., hospital pharmacist) or to individual physicians. Monitoring pharmacists typically review the alert and contact the appropriate physician if they determine that the alert has identified a true event. The alert modality also varies based on the available technology, from printed out reports, to automatic paging of covering physicians, to display of alerts on computer systems (either in results or ordering applications). It should be emphasized that computerized physician order entry (Chapter 6) is not a requirement for these monitors. Thus, a simple version of this approach could be implemented in most US hospitals.

Prevalence and Severity of the Target Safety Problem

It is estimated that over 770,000 people are injured or die in hospitals from ADEs annually,^3,9,10 but variations in study criteria and definitions prevent precise national estimates.¹¹ Fewer data address the epidemiology of ADEs in the outpatient setting. One recent study found an ADE rate of 3% in adult primary care outpatients,¹² while an older study reported a similar rate of 5% among ambulatory patients attending an internal medicine clinic over a 1-year period.²

Detection and alerting interventions primarily target ADEs related to the medication ordering process. One study of preventable inpatient ADEs in adults demonstrated that 56% occurred at the stage of ordering.¹³ Among the 6.5 ADEs per 100 admissions in this study, 28% were judged preventable, principally by changing the systems by which drugs are ordered and administered.¹⁴ In one study of computerized ADE detection, the ADEs identified by computerized monitoring were significantly more likely to be classified as severe than those identified by chart review (51% vs. 42%, p=0.04).⁵ Thus, monitors may capture a subset of events with the most potential for patient injury.

Injuries due to drugs have important economic consequences. Inpatients that suffer ADEs have increased lengths of stay of nearly 2 days and added hospital costs of more than $2000.^9,15 Bootman has estimated the annual cost of drug-related morbidity and mortality within the United States at $76.6 billion, with the majority ($47 billion) related to hospital admissions due to drug therapy or the absence of appropriate drug therapy.¹⁶

Opportunities for Impact

Unfortunately, there are no good data as to how many hospitals have integrated lab and medication systems, which are required for many of the triggers used in computerized ADE monitors.

Study Designs

We found 5 studies of computerized ADE alert systems that were Level 3 or better (see Table 8.1). Four studies^17-20 detected potential ADEs using computerized monitoring and then alerted physicians or pharmacists about the event. One additional study²¹ both alerted the monitoring clinician and made recommendations for actions relating to the suspect condition. Four studies^17-20 were in the inpatient setting and one was in the ambulatory setting.²¹

Study Outcomes

All of the studies reported Level 1 or 2 outcomes. Level 1 outcomes were the rate of ADEs^18,19 and renal impairment (as reflected by rises in creatinine).²⁰ Level 2 outcomes included percent of time recommended actions were taken and time to respond to an event.^17-20

Evidence for Effectiveness of the Practice

One study demonstrated significant decreases in adverse clinical outcomes with the alert systems.¹⁸ This decrease included a surprisingly large reduction in allergic reactions (ones not previously known); it is not clear how the computer alert or decision support system could have contributed to this result.¹⁸ The second study¹⁹ showed no significant difference in the number of adverse events in the intervention and control groups. This study and 3 others revealed significant improvements in response times concerning lab values,^17-20 and one study found a significant decrease in the risk of serious renal impairment.²⁰ Finally, one study demonstrated significant changes in physician behavior/modification of therapy based on alerts with recommended actions.²¹ The effect sizes shown in these studies are listed in Table 8.1.

Potential for Harm

None of the studies discuss any potential for harm associated with the monitor and alerts. It is certainly possible that alerts could be erroneous, but it is doubtful that this would lead to any direct patient harm. As in studies of safety in other industries, one possible source of harm could be too many false positives. Large numbers of clinically insignificant warnings for patients would interfere with routine care, and might result in providers ignoring all warnings, even clinically meaningful ones.

Costs and Implementation

In general, implementation of alert monitors requires computer systems that can link laboratory and medication information. Integrating this information requires the creation of interface between the drug and laboratory databases, with costs that will vary depending on the nature of the existing information systems. In addition, alert methods vary, with some applications directly contracting physicians (which requires further integration of coverage and pager databases) and others using pharmacist intermediaries. The cost of pharmacist review of triggers was less than 1 FTE per hospital in 2 studies^5,7; one of them reported an annual cost savings of up to 3 million dollars by reducing preventable ADEs with this alerting technique.⁷

Studies thus far suggest that physicians view computerized alert systems favorably. Forty-four percent of physician-respondents receiving alerts indicated that the alerts were helpful and 65% wished to continue receiving them (although these alerts went to many physicians because it was unclear who the responsible doctor was). In another study in which alerts were sent only to the responsible physician, 95% of physician-respondents were pleased to receive them.¹⁹

The systems in these studies were all "homegrown" and contained idiosyncrasies that might undermine their implementation elsewhere. Clearly it is important that systems track which physicians are responsible for which patients. In addition, all 4 of the inpatient studies were conducted at tertiary care hospitals and the outpatient study was done at clinics affiliated with a tertiary care center. The translation of this alerting approach to community settings may be difficult. One community teaching hospital has reported success in detecting opportunities to prevent injury (64/1000 admissions) using computerized detection and alerting. This report had only a Level 4 design (no control group), so it was not included in the Table.⁷

Comment

Computerized real-time monitoring facilitates detection of actual and potential ADEs and notification of clinicians. This in turn may aid in the prevention of ADEs or decrease the chances that ADEs will cause harm. The monitors also yield improvements in secondary measures relating to the length of time until response and the quality of response.

The applications in these studies were all "homegrown." Future applications should be evaluated and refined further. In particular, it is important to quantify the yield of collecting these data in terms of patient outcomes, since the start-up costs are significant. If monitors do lead to important clinical benefits, they should become standard features of commercially available hospital computer systems. As this occurs, careful attention will need to be paid to optimizing the response process.

In addition, little has been done to translate these monitoring systems to the outpatient setting, largely because outpatient clinical information is often not computerized or resides in disparate systems. As integrated computerized outpatient records become more common, the systems developed in the inpatient setting should be translatable to the outpatient setting.

Table 8.1. Included studies of computerized systems for ADE detection and alerts*

Study Intervention	Study Design, Outcomes	Resultsa
Kuperman, 199919 Computerized alerts to physicians via paging system	Level 1, Level 1	Median time until initial treatment ordered: 1 vs. 1.6 hours (p=0.003) Median time until condition resolved: 8.4 vs. 8.9 hours (p=0.11) Number of adverse events: No significant difference
McDonald, 197621 Alerts to outpatient physicians with suggested responses to medication related events	Level 1, Level 2	Physicians performed recommended testing: 36% vs. 11% (p<0.00001) Physicians made changes in therapy: 28% vs. 13% (p<0.026)
Evans, 199418 Computerized monitor to detect ADEs (including drug/lab monitors and searches of nursing notes) and then computerized alerts to physicians	Level 3, Level 1	Type B ADEs (allergic or idiosyncratic) and severe ADEs: 0.1 vs. 0.5 per 1000 patient-days (p<0.002) Severe ADEs: 0.1 vs. 0.4 per 1000 patient-days (p<0.001)
Rind, 199420 Computerized alert system to physicians about rising serum creatinine values in patients receiving potentially nephrotoxic medications	Level 3, Level 1	Serious renal impairment: RR 0.45 (95% CI: 0.22-0.94) Mean serum creatinine lower after an event (0.16 mg/dL lower on Day 3, p<0.01) Dose adjusted or medication discontinued an average of 21.6 hours sooner after an event (p<0.0001)
Bradshaw, 198917 Computerized alerts integrated into result review and alerts by flashing light	Level 3, Level 2	Response time to alert condition: 3.6 (±6.5) vs. 64.6 (±67.1) hours

* ADE indicates adverse drug event; CI, confidence interval; and RR, relative risk.

^a Results reported as rates with intervention vs. control (Level 1 study designs) or after intervention vs. before intervention (Level 3 study designs).

References

1. Kellaway GS, McCrae E. Intensive monitoring for adverse drug effects in patients discharged from acute medical wards. N Z Med J 1973;78:525-528.

2. Hutchinson TA, Flegel KM, Kramer MS, Leduc DG, Kong HH. Frequency, severity and risk factors for adverse drug reactions in adult out-patients: a prospective study. J Chronic Dis 1986;39:533-542.

3. Cullen DJ, Bates DW, Small SD, Cooper JB, Nemeskal AR, Leape LL. The incident reporting system does not detect adverse drug events: a problem for quality improvement. Jt Comm J Qual Improv 1995;21:541-548.

4. Classen DC, Pestotnik SL, Evans RS, Burke JP. Computerized surveillance of adverse drug events in hospital patients. JAMA 1991;266:2847-2851.

5. Jha AK, Kuperman GJ, Teich JM, Leape L, Shea B, Rittenberg E, et al. Identifying adverse drug events: development of a computer-based monitor and comparison with chart review and stimulated voluntary report. J Am Med Inform Assoc 1998;5:305-314.

6. Bowman L, Carlstedt BC, Black CD. Incidence of adverse drug reactions in adult medical inpatients. Can J Hosp Pharm 1994;47:209-216.

7. Raschke RA, Gollihare B, Wunderlich TA, Guidry JR, Leibowitz AI, Peirce JC, et al. A computer alert system to prevent injury from adverse drug events: development and evaluation in a community teaching hospital. JAMA 1998;280:1317-20. Published erratum appears in JAMA 1999 Feb 3;281:420.

8. Department of Health and Human Services. Health care financing administration. Fed Regist. 1997; 62.

9. Classen DC, Pestotnik SL, Evans RS, Lloyd JF, Burke JP. Adverse drug events in hospitalized patients. Excess length of stay, extra costs, and attributable mortality. JAMA 1997;277:301-306.

10. Cullen DJ, Sweitzer BJ, Bates DW, Burdick E, Edmondson A, Leape LL. Preventable adverse drug events in hospitalized patients: a comparative study of intensive care and general care units. Crit Care Med 1997;25:1289-1297.

11. Adverse drug events: the magnitude of health risk is uncertain because of limited incidence data. Washington, DC: General Accounting Office; 2000. GAO/HEHS-00-21.

12. Gandhi TK, Burstin HR, Cook EF, Puopolo AL, Haas JS, Brennan TA, et al. Drug complications in outpatients. J Gen Intern Med 2000;15:149-154.

13. Bates DW, Cullen DJ, Laird N, Petersen LA, Small SD, Servi D, et al. Incidence of adverse drug events and potential adverse drug events. Implications for prevention. ADE Prevention Study Group. JAMA 1995;274:29-34.

14. Leape LL, Bates DW, Cullen DJ, Cooper J, Demonaco HJ, Gallivan T, et al. Systems analysis of adverse drug events. ADE Prevention Study Group. JAMA 1995;274:35-43.

15. Bates DW, Spell N, Cullen DJ, Burdick E, Laird N, Petersen LA, et al. The costs of adverse drug events in hospitalized patients. Adverse Drug Events Prevention Study Group. JAMA 1997;277:307-311.

16. Johnson JA, Bootman JL. Drug-related morbidity and mortality. A cost-of-illness model. Arch Intern Med 1995;155:1949-1956.

17. Bradshaw KE, Gardner RM, Pryor TA. Development of a computerized laboratory alerting system. Comput Biomed Res 1989;22:575-587.

18. Evans RS, Pestotnik SL, Classen DC, Horn SD, Bass SB, Burke JP. Preventing adverse drug events in hospitalized patients. Ann Pharmacother 1994;28:523-527.

19. Kuperman GJ, Teich JM, Tanasijevic MJ, Ma'Luf N, Rittenberg E, Jha A, et al. Improving response to critical laboratory results with automation: results of a randomized controlled trial. J Am Med Inform Assoc 1999;6:512-522.

20. Rind DM, Safran C, Phillips RS, Wang Q, Calkins DR, Delbanco TL, et al. Effect of computer-based alerts on the treatment and outcomes of hospitalized patients. Arch Intern Med 1994;154:1511-1517.

21. McDonald CJ. Use of a computer to detect and respond to clinical events: its effect on clinician behavior. Ann Intern Med 1976;84:162-167.

Return to Contents
Proceed to Next Chapter