This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: https://info.ahrq.gov. Let us know the nature of the problem, the Web address of what you want, and your contact information.
Please go to www.ahrq.gov for current information.
Full Title: Criteria for Distinguishing Effectiveness From Efficacy Trials in Systematic Reviews
April 2006
View or download Report
Structured Abstract
Objectives: To propose and test a simple instrument based on seven criteria of study design to
distinguish effectiveness (pragmatic) from efficacy (explanatory) trials while conducting
systematic reviews.
Design: Currently, no validated definition of effectiveness studies exists. We asked the directors
of 12 Evidence-based Practice Centers (EPCs) to select six studies each: four that they
considered to be examples of effectiveness trials and two considered efficacy studies. We then
applied our proposed criteria to test the construct validity using the selected studies as if they had
been identified by a gold standard.
Results: Based on the rationale to identify effectiveness studies reliably with minimal false
positives (i.e., a high specificity), a cut-off of six criteria produced the most desirable balance
between sensitivity and specificity. This setting produced a specificity of 0.83 and a sensitivity of
0.72.
Conclusions: When applied in a standardized manner, our proposed criteria can provide a valid
and simple tool to distinguish effectiveness from efficacy studies. The applicability of systematic
reviews can improve when analysts place more emphasis on the generalizability of included
studies. In addition, clinicians can also use our criteria to determine the external validity of
individual studies given an appropriate population of interest.
Download Report
Criteria for Distinguishing Effectiveness From Efficacy Trials in Systematic Reviews
Evidence-based Practice Center: Research Triangle Institute/University of North Carolina at Chapel Hill (RTI/UNC-CH)
Topic Nominator: Agency for Healthcare Research and Quality
Current as of April 2006
540 Gaither Road Rockville, MD 20850