Executive Summary
Evidence Report/Technology Assessment: Number 201
The Agency for Healthcare Research and Quality (AHRQ), through its Evidence-based Practice Centers (EPCs), sponsors the development of evidence reports and technology assessments to assist public- and private-sector organizations in their efforts to improve the quality of health care in the United States. The reports and assessments provide organizations with comprehensive, science-based information on common, costly medical conditions and new health care technologies. The EPCs systematically review the relevant scientific literature on topics assigned to them by AHRQ and conduct additional analyses when appropriate prior to developing their reports and assessments.
AHRQ expects that the EPC evidence reports and technology assessments will inform individual health plans, providers, and purchasers as well as the health care system as a whole by providing important information to help improve health care quality.
Select for the print version of this summary (PDF File, 120 KB; Plugin Software Help).
Background / Methods / Results / Discussion / Full Report / References
Background
Medication management is a continuum that covers all
aspects of prescription medications. Medication management includes prescribing
and ordering, order communication (or order transmission) between prescribers
and pharmacists, dispensing, administering, and monitoring, as well as
reconciliation, adherence, and education.1 Medication management is complex and costly and enhances the health and well-being of more than half of the population in the developing world. Health information technology (health IT) holds great promise to improve the quality of health care and reduce potential and real errors in medication management while at the same time providing cost-effective care. The Agency for Healthcare Research and Quality (AHRQ) is committed to summarizing and providing the evidence base for health IT. It has produced evidence summaries on health IT related to costs and benefits;2 barriers and drivers of health IT for the elderly, chronically ill, and underserved;3 the impact of consumer informatics applications;4 and telemedicine.5 AHRQ also has contracted for evidence summaries on the use of health IT in decisionmaking,6 patient-centered care,7 and decision support for health care decisionmaking.8 The contracted reports will be available through http://www.healthit.ahrq.gov in mid-2011.
Although these reports often mention medication management, the body of
published evidence on all aspects of the medication management process and how
it is affected by multiple health IT systems has not been consolidated. A
single document is needed to summarize the evidence evaluating the effects of
health IT on the medication management process across providers, settings,
patients, and research methods.
The objectives of this report are to:
- Review the literature on the effects of health IT on medication management.
- Synthesize available evidence regarding the effectiveness and effects of health IT in all phases of medication management as well as reconciliation and education.
- Identify gaps in the literature.
- Make recommendations for future research.
For the purposes of this review, medication management
includes the processes that encompass the five phases of the medication process
(i.e., prescribing and ordering, order communication, dispensing,
administering, and monitoring) across groups of health professionals, patients,
and their informal caregivers, and two aspects of quality with respect to
medication management across the five phases of medication management
(medication reconciliation and education, both postprofessional education of
training and patient education related to medication management). Medication
management can also include procurement, storage, and reporting from the first
assessment of patients to determine their need for drugs through to optimal
care and monitoring after the drugs are prescribed. The organization of the
information in this report is based on the Bell framework of the five phases
across the continuum of medication management and reconciliation and education.1
To address the goals of this report, we further define
medication management health IT (MMIT) applications as electronic systems that (1)
collect, process, or exchange health information about patients; (2) are
integrated with existing health IT systems such as electronic health records or
electronic medical record (EMR) systems; and (3) provide advice or suggestions
to either the health care provider or the patients and their families on issues
or decisions related to medication management. We recognize that functional
elements of the MMIT will vary across particular implementation approaches
within a given phase of medication management. Many of the MMIT applications we
found were designed to encompass more than one phase of medication management.
The sophistication of the systems, degree of integration of the health IT into
workflow systems, and the broad range of settings in which a particular health
IT is implemented and used are also complex and varied. Many health professionals,
support staff, patients, and patients' families were involved in medication
management in the studies assessed.
The evidence assessing MMIT is large, diffuse, and
published across many disciplines. People who can benefit from the knowledge in
this report include health professionals, researchers, administrators, and
other decisionmakers and those who develop and implement health IT
applications. This report is timely because of the Federal emphasis on the use
of health IT to improve health care while at the same time making health and
wellness care more cost effective and safer. Seven questions structure this
evidence report. Within reporting related to the questions, sections are based
on phases of medication management. Reporting is done to address the multiple
settings where medication management is important, the range of health care
providers who deliver and support care using medications, and classes of
medications, specific drugs, or a broad spectrum of medications.
Key Questions (KQs)
KQ1:
Effectiveness
Within all phases of the medication management continuum,
what evidence exists that health IT applications are effective in improving:
- Health care processes?
- Other intermediate outcomes (e.g., satisfaction with system, usability, knowledge, skills, and attitude)?
- Costs and economic outcomes?
- Clinical outcomes for patients?
- Population level outcomes?
- Composite outcomes?
- To what extent does the impact of health IT on improving health care processes, other outcomes, costs and economics, and clinical outcomes vary depending on the type of medication (controlled or noncontrolled substance) or the form of the medication (e.g., oral, injection, intravenous)?
KQ2: Gaps in Knowledge or Evidence
What knowledge or evidence
deficits exist to support estimates of cost, benefit, impact, and net value
with regard to health IT applications in all phases of medication management?
KQ3: Value Proposition for Implementers and Users
What critical information
regarding the impact of health IT applications implemented to support the
phases of medication management is needed to give clinicians, health care
facility administrators, patients, and their families a clear understanding of
the value proposition particular to them?
KQ4: System Characteristics
What evidence supports or refutes
the impact of any of: open source, homegrown, proprietary, local configuration
ability, system configuration ability, conformity with standards being
Certification Commission for Healthcare Information Technology (CCHIT)
certified, system architecture, or feature set on the decision to purchase,
implement, or use health IT in medication management systems?
KQ5: Sustainability
What factors influence
sustainability of health IT applications that support a phase of the medication
management continuum?
- What
evidence exists to demonstrate that health care settings (ambulatory, long-term
care, etc.) influence implementation, use, and effectiveness of such health IT
applications?
- What
is the impact (challenges, merits, costs, and benefits) of having electronic
access to patient data on the quality and safety of care provided by health IT
applications that support at least one phase of the continuum of medication
management?
KQ6: Two-Way Prescription Electronic Data Interchange (EDI)
In a two-way electronic data
interchange (EDI) between the prescribers and pharmacists:
- What
evidence exists demonstrating the barriers and drivers of implementation of
complete EDI that can support the prescription, transmittal and receipt, and
perfection process of e-Prescriptions?
- How
do barriers, facilitators, and economic incentives vary across pharmacists,
physicians, and other relevant stakeholders with respect to adoption and use of
complete EDI (e-Prescribing/ordering with e-Transmission)?
KQ7: Randomized Controlled Trials (RCTs) of Clinical
Decision Support Systems (CDSS)
What evidence exists regarding the
extent of integration of electronic clinical decision support in a health IT
system for the prescribing, dispensing, and administering of medications, and
to what extent does the use of clinical decision support systems impact the
various outcomes (e.g., health care process, intermediate, cost and economics,
and clinical) of interest?
Return to Contents
Methods
We anticipated finding few RCTs
across all phases of medication management and MMIT applications. Studies that
employ other research methods can also provide valuable evidence for
understanding MMIT applications. We therefore included studies employing a
range of research methodologies. We restricted our analysis to
hypothesis-driven studies with group comparisons and appropriate statistical
analysis in addition to qualitative studies with explicit methods for KQ1:
Effectiveness. The only methodological limit was for assessment of the effect
of CDSSs on prescribing, for which sufficient RCTs were available to provide
evidence for synthesis.
Through consultation with our
internal team and AHRQ, we determined that the answers to KQ2: Gaps in Knowledge
or Evidence and KQ3: Value Proposition for Implementers and Users would become
evident from our review of the evidence in KQ1: Effectiveness. We supplemented
these articles with other studies addressing values propositions by
stakeholders. KQ4: System Characteristics addresses the impact of MMIT
application features on the likelihood that the systems will be purchased,
implemented, and used. The evidence for this question comes from studies of all
designs that measure implementation, use, and purchasing decisions. KQ5:
Sustainability addresses the factors influencing the sustainability of MMIT
applications, specifically the impact of the setting and access to other
electronic data within integrated systems on health care quality and safety. To
identify articles that addressed this question, the team, in consultation with
AHRQ, used the definition of sustainability by Humphreys et al.,9 which restricted our choice of articles to only a few. Their definition of sustainability was the ability of a health service to provide ongoing access to appropriate quality care in a cost- and health-effective manner. KQ6: Two-Way Prescription EDI relates to the barriers and facilitators to complete EDI between prescribers and pharmacies during the time between prescription writing and dispensing and how these vary across stakeholders. The best evidence available for KQ6 is found in articles studying EDI between prescribers and pharmacies that include original data (qualitative or quantitative). Because insufficient evidence was found on two-way EDI, we included one-way EDI as well. KQ7: RCTs of CDSS addresses the extent to which CDSS systems are integrated into health IT systems for medication management and the impact on outcomes as described in KQ1: Effectiveness. As a team we felt that adequate evidence was available to address this issue so that we could limit our scope to RCTs.
Given the broad range of questions and outcomes
addressed, we searched peer-reviewed electronic databases by first using
textwords relating to the various types of health IT applied to medication
management (Appendix A of the full report). These searches were then combined
with a search using subject headings related to the five medication management
phases plus reconciliation and education as well as specific health IT
application terms (e.g., CDSS). We combined these medication management terms
with computer and technology terms. When possible, we excluded letters,
editorials, commentaries, and animal studies. Because our interest was in all
study designs, we did not limit based on methodology. We also put no limits on
language or time to capture the global literature and early studies.
Databases searched included MEDLINE®, Embase, CINAHL
(Cumulated Index to Nursing and Allied Health Literature), Cochrane Database of
Systematic Reviews, International Pharmaceutical Abstracts, Compendex, Inspec
(which includes IEEE Xplore), Library and Information Science Abstracts,
E-Prints in Library and Information Science, PsycINFO, Sociological Abstracts,
and Business Source Complete. We also looked for eligible studies by reviewing
grey literature sites, performing hand searches of pertinent reviews, querying
our experts, and by reviewing the AHRQ National Resource Center for Health IT Knowledge
Library resources (available at: http://healthit.ahrq.gov/portal/server.pt/community/knowledge_library/653).
The search results were downloaded into Reference Manager®
version 10 (ISI ResearchSoft) and uploaded into a customized systematic review
management system (Health Information Research Unit, McMaster University).
Studies were
eligible for inclusion if they used health IT in any aspect of the medication
management process. We included
articles on MMIT only if the system was integrated with at least one existing
health IT system and if they processed patient-specific information and
provided advice or suggestions. A critical inclusion requirement was the
integration of information.
Personal digital assistants (PDAs), which integrated
patient-specific information provided by either the clinicians or the patients,
were analyzed to assist in medication management decisions (by request of
AHRQ). This exception is made because PDAs and hand-held devices are considered
an important, and perhaps unique, means of improving health care quality in
relation to medications. The use of PDAs to manage medications is especially
important for clinicians and patients who are in settings that do not have large,
sophisticated, and integrated information systems. Other stand-alone devices
with no integration of information with another health IT were excluded.
Articles on all five phases of the management process plus medication
reconciliation and postprofessional education related to MMIT were included.
Once we tagged the articles for content, we assessed whether those that passed
our inclusion criteria were pertinent to specific key questions. Many articles
were analyzed in several phases of medication management and sections of the
report.
Studies were classified as being observational,
case-control, cohort, or RCTs. The quality of included studies was assessed using
the same criteria employed by Jimison et al. in their AHRQ report.3 RCT scoring was based on the Delphi consensus work done by Verhagan and colleagues.10 This scale is referred to in this report as the Verhagen/AHRQ RCT quality scale. Observational studies with before–after, time series, surveys, or qualitative methods were not assessed for quality because few well-validated instruments exist. Bibliographies of systematic and narrative reviews were examined to identify studies, and select reviews were integrated into sections of the report.
Data were abstracted from relevant articles and tagged
for applicability to the various key questions. Given the range of questions
addressed, data abstraction was performed by a core group of staff and entered
into online data abstraction forms. One reviewer did the abstraction, and a
second, senior reviewer checked its accuracy. The authors of this report
performed a final check on the abstracted data. The reviewers were not blinded
to the identity of the article authors, institutions, or journal. Data abstraction
was difficult in many instances because of the lack of accepted definitions and
absence of important features of the study or MMIT application. For example, we
identified problems with the differences between computerized provider order
entry (CPOE) for ordering and e-Prescribing systems. Definitions for medication
errors and related terms were often inconsistently used. To make data abstraction
easier, we established working definitions, which can be found in Appendix F of
the full report.
Meta-analysis was not performed on any data because of
the heterogeneity of the studies in terms of interventions, populations,
technologies used, and outcomes measured, as well as the presence of mostly
descriptive and observational studies.
Throughout the project, the core team sought feedback
from the internal advisors, our Task Officer from AHRQ, and the Technical Expert
Panel.
Return to Contents
Results
Our literature search retrieved 40,582 articles. After
duplicates were removed, 32,785 articles were screened at title and abstract
stage. From a full-text screen of 4,578 articles, we identified 789 articles
that were eligible for inclusion in this report. Of these articles, 361 met
only our inclusion criteria for content and did not have group comparisons,
hypothesis testing, or appropriate analysis. These are listed in the
bibliography of the report. Across the seven key questions, we synthesized the
information from 428 articles.
KQ1: Effectiveness
All outcomes. KQ1: Effectiveness contains 379
studies assessing changes in process, intermediate outcomes, clinical outcomes,
and economic and cost outcomes. The majority of studies were observational,
with a fair number of RCTs for prescribing and monitoring phases (Table A). Fifty-three
qualitative studies are included in this total. Prescribing and monitoring were
the most frequently studied phases of medication management (Table A), with
hospital and ambulatory care settings well-represented to the near exclusion of
long-term care, home, and community (Table B).
Though dealing with prescriptions and medications,
pharmacists were poorly represented in studies, most focused on physicians
(Table C). CDSS and CPOE systems were the most often studied MMIT technologies
(Table D).
The results from this section suggest that care processes
such as medication errors, time for tasks, workflow and knowledge, skills, and
attitudes can be improved with the use of MMIT. The evidence is strongest
specifically during the prescribing and monitoring phases. Few studies
evaluated clinical outcomes associated with the use of MMIT. Those that did
often did not show statistically significant improvements in clinical outcomes.
Most of the studies with statistically significant differences in clinical
outcomes found small differences. The small number of articles with data on
clinical outcomes is probably due, at least in part, to the difficulty in
evaluating and establishing a direct association between the use of MMIT and
clinical outcomes. This difficulty arises because of the distant nature of the
outcome compared with the application of the health IT. Other contributing
factors could also be considered.
Much of the relatively new research is addressing the
type of research needed to come to a realistic and useful assessment of MMIT:
pilot and demonstration projects and quantitative studies. Limited evidence
suggests that MMIT can likely be cost effective, although most of the economic
data come from cost analyses, which were often incomplete and seldom from
head-to-head cost-effectiveness, cost utility, or cost-benefit trials.
A substantial body of qualitative literature indicates
support for the use of health IT in the various phases of medication management
by a number of health care providers and patient groups. Survey studies of
satisfaction and use reflect similar findings of acceptance and satisfaction,
although most indicated room for improvement. Issues relating to changing care
practices and workflow are frequently mentioned. The studies also provide
useful summaries of unintended consequences of MMIT applications, which are
discussed in detail in the full report.
Process changes. Most of the studies evaluating
MMIT applications provided data on changes in process (225 of 378).
Distribution in the number of studies across the five phases, plus
reconciliation and education, was not equal. Prescribing was studied in 174
studies, order communication in 16 studies, dispensing in 9 studies,
administering in 19 studies, and monitoring in 47 studies. Four studies
evaluated reconciliation and one studied patient education. Studies often
evaluated more than one phase.
Prescribing. The prescribing phase is well studied (174 studies), especially in
hospital (61 percent of studies) and ambulatory care settings (39 percent).
Long-term care centers (one study) and community and home settings (no studies)
are not well studied. Physicians are by far the most studied group of health
professionals. More studies are needed that evaluate nonphysician use of MMIT,
specifically pharmacists, mental health professionals, nurses, and other
nonphysician prescribers, as well as patients and their caregivers. Many of the
studies of health care providers who were not physicians were purely
descriptive of the people involved with them, and the systems themselves.
Based on the studies of process changes, CDSS and CPOE
systems can play an important role in making prescribing and ordering more
accurate, improving record keeping, and speeding up and improving
communication. Both systems, either alone or, more often, integrated, are well studied
(multiple studies with strong methods). Other MMIT applications lack evidence,
especially those that involve nurses, pharmacists, and patients and their
families.
MMIT in prescribing is associated with improvements in
patient safety-related processes of the prescribing process, especially in
hospital-based studies (87 percent, 52 of 60 studies), and somewhat less in
ambulatory-based studies (68 percent, 28 of 41 studies). Errors related to
prescribing and ordering were reduced in hospital-based studies (68 percent, 15
of 22 studies), but prescribing errors were not studied as often in ambulatory
settings (two of two studies were positive). Reductions in time were related to
the time taken to order or prescribe or the speed of the
prescribing-to-administering processes. Most reductions in time were not seen
as often in hospital-based studies (four of seven studies positive), but were
positive more often in ambulatory settings (four of five studies). Adherence to
treatment guidelines, reminders, and recommended practice was improved in
hospital studies (83 percent, 19 of 23) and to a lesser but still significant
extent in ambulatory studies (64 percent, nine of 14 studies). Workflow was not
evaluated in these studies of changes in process, although issues of workflow
are addressed in qualitative studies in other sections of this report.
Order
communication. Order communication, like dispensing, is one of the two
medication management phases with the least number of studies—only 16 were
identified. Two-way EDI holds promise to increasing the effectiveness of
perfecting the prescription/order interactions between clinician prescribers
and pharmacists. Currently, evidence on one-way communication predominates. The
changes in process were also varied (two studies of errors, two of prescribing
changes, five on time considerations, and three on workflow). Most studies were
done using quantitative observational methods and all showed positive results.
Dispensing. Nine studies (three RCTs) assessed
process improvements in dispensing. All process changes that were evaluated
were found to be positive: four on modifications of the drugs that the
pharmacists dispensed, three on errors, two on workflow, and one on adherence
to good practice. With these few studies and multiple outcomes, evidence is
limited on the role of MMIT in improving dispensing. This supports the findings
of a Canadian health technology assessment report on MMIT that evaluated
hospital dispensing and administering medications in hospitals.11
Administering. Many articles dealing with
administering medications were not included in this report because they were
descriptive and did not include comparative data. Nineteen studies, 1 RCT, 1
cohort study, and 17 quantitative observational studies, were included. All
studies were set in hospitals and included nurses. The MMIT systems were well integrated
into multiple hospital IT systems. Error-reduction goals were common in the
studies and almost always found to be improved (8 of 13 studies of errors).
Errors were mixed, as some related to transcription and some to timing of
administration, while some identified more serious errors. Four studies showed
no improvement in errors while one study showed increases in errors, mostly
related to timing of administration.12 Four of five studies showed reductions in time from ordering to administering medication. Two studies evaluated the allocation of nursing time: one showed change and one did not in the proportion of time spent on various nursing tasks, including direct patient care, with the introduction of integrated MMIT for medication administering.
Monitoring. In our analysis, 70 percent (33 of 47
studies) of the included studies were associated with a 50 percent improvement
in half or more process measures. Of these studies, most targeted physicians
exclusively (34 studies), were conducted in academic institutions (33 studies),
were developed for use in the ambulatory care setting (28 studies), focused on
the adult population (36 studies), and provided CDSS with alerts or reminders
to support chronic disease management (12 studies).
Studies that involved laboratory-based medication
monitoring were most likely (76 percent of the time) to be associated with a
greater than 50 percent improvement in a process outcome(s) than sign- or
symptom-based medication monitoring. The most successful types of studies
focused on changing prescriber behavior, improving response time to generated
alerts, and improving the diagnosis and management of chronic diseases.
Reconciliation. Two systematic reviews and
four studies provided evidence for improved reconciliation of medications with
health IT. Reconciliation is the matching of medication lists over time, from
different health care systems or from different prescribers. The evidence on
reconciliation of medication lists is sparse, especially for systems that are
fully integrated and capable of providing electronic comparisons of historical
and current medications for individual patients at hospital discharge or on
transfer to other facilities. All four studies showed improvements in agreement
among lists of medications and two extended the evaluation to show improved
prescribing13 and reduced errors.14
Unintended consequences. Eighteen studies provided
data on adverse effects or unintended consequences. Two qualitative studies
identified classes or categories of unintended consequences of health IT, many
of which apply to MMIT applications. Some unintended consequences are minor,
and some are major. In addition, some are seen to be positive and helpful. Some
consequences are serious. For example, a small but statistically significant
increase in mortality was seen in a children's hospital that installed a CPOE
system that did not match workflow needs.15 A similar study showed another children's hospital that did not see the same increase in mortality in admitted children after their careful planning and implementation of health IT.16 Several authors contend that all health IT has unintended consequences. Formal evaluations of health IT installations should seek these unintended consequences and report them in their publications related to the evaluation. The importance of unintended consequences of MMIT also depends on the severity of the event, the degree of invasiveness of the MMIT, and the extent to which the use of the MMIT system disrupts existing workflow and processes. Consideration of formal reporting of serious unintended consequences might benefit all involved in development and implementation of MMIT systems. The qualitative studies in this report supplied a richer understanding of the adverse effects of MMIT, and they can form a strong base for more qualitative and quantitative studies of unintended consequences.
Education. Education related to MMIT centers on
three aspects: formal informatics
training during professional education or after graduation, training to use the
MMIT systems, and improved outcomes based on knowledge and skills because of
the use of the MMIT systems for health care providers, patients, and their
families. This report does not include preprofessional or professional
education related to the use and understanding of MMIT systems or certification
in informatics or eHealth, all important aspects of MMIT application
development and integration. Although we sought articles assessing
postprofessional education related to changes in process associated with MMIT
systems, we did not identify any articles that met our criteria. Training in
the use of systems was often mentioned in articles but was not evaluated. Only
one article was related to the educational component of MMIT systems for
patient and family use, and it was associated with improved clinical outcomes.
More information on health care professional and patient education is included
in the sections of this report dealing with intermediate outcomes.
Intermediate outcomes. Intermediate outcomes deal
with use, usability, education, knowledge, skills, and attitudes. Most studies
with intermediate main endpoints focused on measuring use, correlates of use,
perceptions, and satisfaction in the prescribing phase (26 of 42 studies). As
for changes in process, clinicians and prescribing were well-studied. Use,
perceptions, and satisfaction were reported to be improved. Factors such as
ease of use, perceived usefulness, and improved quality of care predominated.
Satisfaction and attitudes varied depending on the role of the health care
provider. Variation in needs and roles of health professionals with respect to
use of health IT are real and should be considered when choosing or
implementing any new IT system. Usability studies with comparison groups are
sparse but can provide useful suggestions to improve systems. Usability studies
are often difficult to generalize or transfer across settings, in part because
MMIT effectiveness is linked strongly to the culture, institutional leadership,
and other situation specific factors. Therefore, applicability of findings
related to usability is problematic in MMIT applications.
Economic outcomes. Five of 31 articles dealing
with costs conducted comprehensive economic evaluations (costs and
consequences). Two evaluated a CPOE system and three evaluated CDSS. Most of
the studies that included monetary data (22 of 31 studies) were partial
economic evaluations in the form of cost analyses (assessing costs of
alternatives without analysis of effectiveness or efficacy). Most of these
partial economic analyses assessed costs of prescribed medications with the
MMIT system compared with not having the MMIT system.
Several studies found that health IT interventions may
offer cost advantages despite their increased acquisition costs. These studies
showed that over time, a net benefit accrued based on cost reductions resulting
from the MMIT (such as lower adverse drug events (ADEs), drug costs, and
laboratory test usage). However, given the uncertainty that surrounds the cost
and outcomes data, and limited study designs available in the literature, it is
difficult to reach any definitive conclusion as to whether the additional costs
and benefits represent value for money.
Clinical outcomes. A total of 76 studies sought to
measure improvement in clinical outcomes or reduction in ADEs, of which 26 (34
percent) reported significant benefits of health IT. One reported harm—a small
but clinically important increase in mortality when an inflexible CPOE was
implemented in a children's hospital.15 Because of the seriousness of the implications of this study, many people reviewed this article and its methods.17 A later and similar study showed that with careful planning another children's hospital did not see the same increase in mortality in admitted children after the implementation of a health IT.16
An additional two studies implemented CDSSs to reduce
costs and assessed whether reductions in drug use increased mortality15 and length of stay.18 Both studies lacked sufficient power to conduct a valid assessment.
Studies that used laboratory-, sign- and symptom-based
monitoring approaches were mostly clinician based. If the MMIT monitoring was
used to identify and intervene with patients with actual problems (e.g., excess
blood pressure) or needed care (e.g., hemoglobin A1c monitoring), this appears
to be more effective than CDSS approaches that identified theoretical problems
(potential for ADEs), particularly if patients are also sent reminders and
decision support recommendations.
Highly targeted interventions, which focused on specific
problems that provide problem-related specific interventions, appear to be more
effective than more diffusely focused systems such as CDSS and CPOE. Some of
these highly targeted interventions involved CDSS tools for improving the
effectiveness of anticoagulants (proportion of days with blood clotting
parameters within the therapeutic range), improving the choice, route, and
duration of antibiotics, and reducing ADEs related to antibiotic use, and most
were successful.
Studies that have been successful in improving patient
outcomes target high risk and vulnerable populations who have poor disease
control, lack sufficient access to health care providers to manage their
condition or subpopulations with sufficient economic resources to respond to
the CDSS intervention. The effect of similar CPOE systems on mortality can vary
substantially as a function of the extent to which implementation strategies
disrupt or delay critical activities in the clinical setting and demand
additional time for order entry from clinical staff. Critically ill patients
(i.e., those who are most vulnerable) are most likely to be affected by dysfunctional
technology and implementation strategies.
Qualitative studies. Qualitative studies seek to
understand phenomena and answer questions of why and how as well as to gain
insights into real life situations. They often study the more human or "soft"
side of health and health care. The preceding sections concentrated on studies
with quantitative outcomes. Fifty-three qualitative studies are included in
this section. Patient safety was the main health aspect evaluated in
qualitative studies. Before MMIT implementation most studies found that
clinicians expected that MMIT would improve patient safety and once implemented
most clinicians felt that MMIT had improved safety.
The qualitative studies focused on system design
including workflow changes, challenges with the system interface, and new
communication processes—all of which can generate new kinds of medical errors,
which in some cases were detrimental to patient safety.
Early implementers associated MMIT with a lot of
self-reported "hard work" by those who were expected to use the new
systems. These people, most often health professionals, struggled, often
independently, with limited guidance with respect to planning and
implementation tactics during preparation for and implementation of the MMIT applications.
During planning and early implementation, the users often experienced
unanticipated effects. Frequently, the initial stage was disruptive and,
consequently, clinicians found provision of care to be more challenging with
the MMIT system than without. However, after the initial stage was over, the
attitudes of the care providers changed, and the potential benefits of the
system become clearer to most. Of special note is that the implementation of
MMIT systems generated emotional responses in a broad range of health
professionals, both positive and negative. For example, strong feelings were
associated with reminders and alerts19 and CPOE.20
MMIT implementation did not just mean that a clinician
needed to learn a new IT system, but the implementation also affected most of
the other parts of the delivery of care processes, including how the
interdisciplinary care team worked together.
KQ2: Knowledge and Evidence Gaps
We identified gaps in the report, some that we expected
and some that we did not. We address the question of knowledge deficits across
phases and outcomes, settings and participants, grouping similar gaps together.
Phases of medication management. Because of the
preponderance of publications on the prescribing and monitoring phases, they
are less in need of more study than the other phases of order communication,
dispensing and administering, and medication reconciliation. In addition, the
educational or training requirements for effective use of MMIT applications by
health professionals need to be studied as well as education related to
patients as new MMIT applications are developed for their use.
Research methods. MMIT applications are complex
interventions and need to be studied in pragmatic (i.e., does it work in real
settings?) evaluation projects and using complex interventions methods. The
applications also should ideally be studied by teams of researchers with, or
teams that seek consultation from, those who have experience in clinical
practice, research methods, statistical analysis, and informatics training and
experience. Qualitative studies are also vital to understand the complex nature
of how systems are used and valued, especially across groups of health
professionals who often have different needs and expectations.
Health care providers. Physicians are well studied.
Nurses, midlevel practitioners (nurse practitioners, physician assistants,
midwives), pharmacists, other prescribers such as dentists and mental health
practitioners, and hospital administrators need studies directed at their
needs, practice patterns, and health IT tools.
Patients. Many studies included data related to
patients, usually in the measurement and reporting of process changes and other
outcomes. Few studies, however, concentrated on how the MMIT systems directly
affected patients and clinical outcomes important to them. Traditionally, MMIT
systems were developed as clinician and administrator tools. Patient and family
use of MMIT systems is becoming more important, and this gap in our understanding
needs to be addressed.
Settings. Hospitals and ambulatory care settings
are well studied. Gaps exist in our knowledge of the effectiveness of MMIT in
long-term care facilities, the community, and homes. Long-term care facilities
most need strong qualitative and quantitative studies because theyrely heavily
on medication. Homes, schools, and other community settings will also become
more important with shifting care to more self-reliance in relation to
wellness care and chronic disease management.
Health IT. Much research has gone into evaluating
CDSS and CPOE systems, either alone or integrated. For example, 77 of 88 RCTs
evaluated some aspect of CDSSs. Other MMIT applications, especially those that
are used by nonphysicians or outside the prescribing and monitoring phases,
lack evidence. Examples with little evidence on effectiveness are bar coding
for administering and dispensing, pharmacy information systems, electronic
medication administration record systems, and fully integrated comprehensive
information systems.
Process changes. Patient safety processes such as
error reductions and improvement in prescribing have a strong evidence base.
Issues related to workflow, communication changes, and unintended consequences
are understudied. More study of laboratory-based monitoring of medications,
especially in facilities that have highly integrated information systems, is
important. More qualitative and controlled studies are needed as well as multicenter
studies and those that use methods developed by groups focusing on health
technology assessment (HTA). These HTA methods include integrated reports that
bring together research syntheses, modeling of processes and full economic
reports, and cost studies. Often these HTA reports do not, but can, involve additional
collection of evidence.
Intermediate outcomes. More study is needed on the
importance of usability testing in all stages of development and use. This must
be done with all users and not just segments of those involved in using MMIT.
Usability studies have not traditionally been generalizable or transferrable
but more limited to a specific setting. AHRQ might consider a research program
in how to make these usability studies more applicable to multiple
institutions, training in usability methods, collection of usability tools and
completed studies, and research into the need for standards of usability
testing for new or modified systems. Usability studies must also include all
users of systems. For example, systems that have been optimized only for
physician users are usually systems that nurses and other health professionals
have difficulty using. Workarounds have often been unofficially implemented by
users instead of system modifications and improvements.
Clinical outcomes. Findings associated with
improvement in clinical outcomes are still equivocal. These studies are
difficult to do well, expensive, and time consuming, but they must be done. Multicentered
trials planned by strong teams of experienced people from multiple backgrounds
are vital.
Cost and economic outcomes. Although many studies
exist that list costs and outcomes, few comprehensive and definitive studies of
the economic value of MMIT applications exist. Both the potential for
improvement and the costs of implementing and maintaining these systems are
huge. Again, well-planned studies with broad input from many stakeholders are
necessary for understanding the true worth of MMIT applications. HTA or other
studies that integrate costs and consequences of MMIT systems would be ideal.
Qualitative. Qualitative studies have provided
much valuable information about MMIT. Gaps in qualitative knowledge center on
the lack of qualitative studies that address the effects of MMIT on health
outcomes. In addition, very few qualitative studies examined the effects of
MMIT from the perspective of the patient.
KQ3: Value Proposition for Implementers and Users
Value proposition is determined from a balance of financial,
clinical, and organizational benefits. A clear assessment of each of these from
the viewpoint of each stakeholder is needed to make a clear value judgment. For
each stakeholder—and many are involved with MMIT implementation—the relative
importance of these three elements is different. Values will also vary
depending on the setting and the type of technology employed. Multiple
stakeholders, some of whom may be distant from the MMIT, need to be considered
in any value proposition study. Based on the evidence in KQ1: Effectiveness,
knowledge about the three elements needed to make value judgments is slowly
accumulating. We cite only 31 papers in this section, although some of our
assessments come from sections of this report that have included more studies.
Gains in productivity and process of care outcomes have been shown, but good
evidence of improvement in patient outcomes with MMIT is weak or lacking. The
body of economic literature is still sparse and lacks vigorous study. We found
little theoretical work or actual studies that were done to determine what each
stakeholder takes into account to reach value proposition judgments related to
MMIT.
KQ4: System Characteristics
Few studies (n = 21) demonstrated evidence of the impact
of the characteristics of MMIT applications on the likelihood to purchase,
implement, and use such IT applications. No studies assessed open-source health
IT applications, with only one study each on conformity with standards and
CCHIT-certified systems. Twenty of the articles related to the prescribing and
ordering phase. Almost all of the articles suggest that feature sets of health
IT applications have been instrumental in reaching decisions to adopt MMIT
applications. Certain features of systems improve the likelihood of purchase,
implementation, and use of MMIT. The literature, however, is sparse and
observational in nature. Most often authors described barriers and concerns
toward implementation and acceptance rather than characteristics of MMIT that
could facilitate implementation, purchase, and use of such systems. Authors
seldom provided enough details about the technology to form conclusions about
the value of feature sets and system characteristics. Head-to-head comparisons
of systems differing in their features were not found.
KQ5: Sustainability
Our literature review revealed three important findings:
sustainability is frequently mentioned in the core biomedical informatics
literature, it is poorly defined, and none of the articles included in this
evidence report explicitly studied sustainability. These findings are not
entirely surprising. A previous AHRQ-sponsored evidence report that assessed the
costs and benefits of health IT in pediatrics found only one article that
explicitly discussed sustainability.21
Future research would be beneficial for many if a study
or group would develop an operational definition of sustainability that could
be used to study its determinants. Moreover, it is likely that the Health
Information Technology for Economic and Clinical Health (HITECH) Act of 2009
will lead to improvements and sustainability of health IT applications that
specifically support the medication management continuum through meaningful
use.
We have summarized a body of literature that uses surveys
to detect patterns in the characteristics of people and organizations that are
more likely to implement various technologies. These surveys are often the
basis for further study into barriers and facilitators to increasing uptake and
adoption.
Integration of MMIT with other systems was an inclusion
criterion for our report (except for PDAs that analyzed patient-specific data).
Some technologies were integrated with a greater number of components than
others. Frequently, the descriptions of the systems were inadequate to fully
determine how the systems were connected. Access to various other information
sources, most notably laboratory reports, enhanced the performance and
acceptance of the MMIT applications.
KQ6: Complete Two-Way Electronic Data Interchange
No reports documenting the use of complete two-way EDI
systems were found. Evidence from the limited set of one-way, e-Prescribing
studies was extrapolated to identify possible key facilitators and barriers to
completely electronic, two-way, e-Prescribing systems. Possible facilitators
include monetary or other incentives to providers, a permissive regulatory
environment, and the existence of an established standard for prescription EDI.
Barriers included the low rate of EMR adoption in the United States, regulatory
and legal uncertainties, and inadequate consideration of the effects of
e-Prescriptions on pharmacists and pharmacies and their processes. While
answering this question, we found that the Bell model does not represent the
two-way communication between pharmacists and prescribers—it shows only a
one-way linear movement of information.
KQ7: Effectiveness of CDSS
Seventy-seven RCTs were designated as primarily studying
CDSS related to medication management and integrated with other health IT.
These studies involved 4,709 providers and 828,441 patients in total. All
studies assisted with at least the prescribing or monitoring phases of
medication management. Overall, we found a lack of RCTs addressing electronic
decision support integrated with other types of health IT. Statistically
significant process changes were often shown in these RCTs. Only a small
minority of these focus on clinical outcomes, however. Studies with clinical
outcomes are those that are most important to guide decisionmaking of patients'
providers and policymakers about the usefulness and need for MMIT
interventions. A very small number of studies reported improvement in clinical
outcomes.
Return to Contents
Proceed to Next Section
540 Gaither Road Rockville, MD 20850 