Skip nav
 
  •  

Home > About NREPP > FAQs

Frequently Asked Questions (FAQs)

General Information

Using the NREPP Site

Information for Developers Interested in Submitting an Intervention for Review

Information for Developers Whose Intervention Has Been Accepted for Review and/or Posted


General Information

What is NREPP?

The National Registry of Evidence-based Programs and Practices (NREPP) is a searchable online registry of interventions that support mental health promotion, substance abuse prevention, and mental health and substance abuse treatment. Through NREPP, users can identify and learn more about interventions that may meet their needs. All interventions that are included in NREPP have been voluntarily submitted, have met a set of minimum requirements, and have been assessed by independent reviewers.

back to top

What does Quality of Research mean?

Quality of Research (QOR) is one of two main categories of NREPP ratings. The QOR category contains six criteria (reliability of measures; validity of measures; intervention fidelity; missing data and attrition; potential confounding variables; and appropriateness of analysis), which are used in rating the strength of evidence supporting the results or outcomes of the intervention. Because interventions may target multiple outcomes, and the evidence supporting the different outcomes may vary, each outcome is rated separately. Each intervention summary contains QOR ratings as well as a description of the study strengths and weaknesses that may have contributed to those ratings. For more information on the QOR ratings, see the Quality of Research page.

back to top

What does Readiness for Dissemination mean?

Readiness for Dissemination (RFD) is one of two main categories of NREPP ratings. The RFD category contains three criteria (availability of implementation materials; availability of training and support resources; and availability of quality assurance procedures), which are used in rating the quality and availability of an intervention's training and implementation materials, that is, how easily the intervention can be implemented with fidelity using the materials and services that are currently available to the public. Each intervention summary contains RFD ratings as well as a description of the dissemination strengths and weaknesses that may have contributed to those ratings. For more information on the RFD ratings, see the Readiness for Dissemination page.

back to top

What is comparative effectiveness research?

The Federal Coordinating Council for Comparative Effectiveness Research defines comparative effectiveness research, or CER, as follows:

Comparative effectiveness research is the conduct and synthesis of research comparing the benefits and harms of different interventions and strategies to prevent, diagnose, treat, and monitor health conditions in "real world" settings. The purpose of this research is to improve health outcomes by developing and disseminating evidence-based information to patients, clinicians, and other decision-makers, responding to their expressed needs, about which interventions are most effective for which patients under specific circumstances.

To provide this information, comparative effectiveness research must assess a comprehensive array of health-related outcomes for diverse patient populations and subgroups. Defined interventions compared may include medications, procedures, medical and assistive devices and technologies, diagnostic testing, behavioral change, and delivery system strategies. This research necessitates the development, expansion, and use of a variety of data sources and methods to assess comparative effectiveness and actively disseminate the results.

This definition was published in the Federal Coordinating Council's June 30, 2009, Report to the President and the Congress (PDF, 3.6MB).

Each intervention summary included in NREPP indicates whether or not the intervention was evaluated in comparative effectiveness research studies (i.e., whether or not at least one of the studies included in the NREPP review met the definition of comparative effectiveness research given above).

back to top

Are environmental interventions/community-level interventions included in NREPP?

Yes.

back to top

What is the recommended format for referencing an NREPP intervention summary?

NREPP recommends the following format for references:

Substance Abuse and Mental Health Services Administration. ([year, month of review]). [Program name]. Retrieved from the National Registry of Evidence-based Programs and Practices Web site, [URL of summary]

For example:
Substance Abuse and Mental Health Services Administration. (2007, December). Motivational Interviewing. Retrieved from the National Registry of Evidence-based Programs and Practices Web site, http://www.nrepp.samhsa.gov/ViewIntervention.aspx?id=130

back to top

Who rates the interventions reviewed?

NREPP contract staff, not SAMHSA, assign two sets of two reviewers to independently review each intervention: one set for the Quality of Research review and one set for the Readiness for Dissemination review. The ratings provided in the intervention summary are the average reviewer ratings. For more information, see the Selection and Training of Reviewers page.

back to top

How do I become a reviewer for NREPP?

Potential reviewers are recruited, selected, and approved by SAMHSA on the basis of their experience and areas of expertise. All potential reviewers must meet minimum requirements, which differ for Quality of Research and Readiness for Dissemination; each category requires different kinds of expertise. Potential reviewers also must complete training on the procedures and criteria they will use to rate interventions. For more information, see the Selection and Training of Reviewers page.

back to top

How are interventions reviewed?

Each intervention is reviewed by two sets of reviewers who have been selected to participate on the basis of their qualifications. One set reviews Quality of Research, and one set reviews Readiness for Dissemination. NREPP staff send a review packet to each reviewer, who independently conducts a review using either the Quality of Research or Readiness for Dissemination review criteria. The reviewers submit their ratings and comments to NREPP staff, who include this information in the intervention summary that is posted on the NREPP Web site.

back to top

How do I contact NREPP with a question or comment?

NREPP staff are available to answer any questions by email, phone, or online contact form. See the Contact Us page for details or to send a comment or question through the contact form.

back to top

Using the NREPP Site

How do I decide which intervention to use?

First, identify your community's needs so you can determine which type of intervention will be the most appropriate. Next, assess your organization's capacity (e.g., financial resources, organizational commitment, community buy-in) to implement an intervention with fidelity. Once you have this information, search NREPP or another registry to find an intervention that matches your community's needs and that can be implemented given your organization's capacity. Information on selecting and implementing an intervention can be found in NREPP's Learning Center — A Road Map for Implementing Evidence-Based Programs.

back to top

How do I find an intervention?

If you know the name of the intervention, you can either type it in the keyword Search box or select it from the View All Interventions list. From these pages on the Web site, you can access summaries describing interventions that have been reviewed. You can also view a list of interventions that have been Accepted for Review. However, the interventions listed in Accepted for Review have not been summarized on the Web site yet because their review process is not complete. If you sign up for e-updates, you will receive announcements with links to interventions recently added to the site.

If you do not know the name of the intervention, you can find relevant interventions by typing keywords in the Search box or by selecting criteria through the Advanced Search to narrow the list of relevant interventions by area of interest, outcome category, geographic location, age, race/ethnicity, setting, gender, and/or keyword.

If you have further questions, please call 1-866-436-7377, email us at nrepp@samhsa.hhs.gov, or use the online contact form. See the Contact Us page for details or to send a comment or question through the contact form.

back to top

How do I get more information on an intervention I found on the site?

To obtain more information about an intervention listed on this site, please contact the developer directly, using the contact information provided in the intervention summary. To assist you in gathering additional information, NREPP has compiled a list of Questions To Ask as You Explore the Possible Use of an Intervention (PDF, 54KB).

back to top

Why are only select intervention summaries translated into Spanish?

Spanish translations are provided for summaries of interventions for which the materials needed to deliver the intervention (e.g., treatment manuals, curricula, training guides, information for administrators) are available for dissemination in Spanish. To see a list of the summaries that have been translated into Spanish, go to the View All Interventions page, click Language Translations in the list of search criteria on the left, select Spanish, and click Search. You also can use the Language Translations criterion to refine a search after using the Basic Search or Advanced Search.

back to top

Why do some intervention summaries contain two review dates?

In 2011, NREPP began the process of reviewing new research and/or dissemination materials for select interventions that had already been reviewed by NREPP at least 5 years ago and summarized on the Web site. Summaries for these interventions contain information from the original Quality of Research (QOR) and Readiness for Dissemination (RFD) review as well as information from a second QOR and/or RFD review, depending on the research conducted or materials developed since the original review. The summary uses review dates to present the information stemming from each review.

back to top

Information for Developers Interested in Submitting an Intervention for Review

How do I find out if my intervention qualifies for submission?

Minimum requirements and other details regarding submission are provided in a Federal Register notice that is posted the summer before each open submission period. The open submission period is generally held annually from November to February, and submission requirements sometimes differ for each open submission period. To receive announcements about future open submission periods, please sign up for our Submissions mailing list. The current Federal Register notice, as well as those for past submission periods, is available on the NREPP News page.

back to top

Is there any cost to submit an intervention to NREPP?

There is no cost to submit an intervention to NREPP. However, applicants must be prepared to send all relevant intervention materials and three copies of all dissemination materials to be included in the review at their own expense.

back to top

Where can I find information on preparing a submission to NREPP?

This information can be found in NREPP's Learning Center — Preparing an Intervention for NREPP Submission and Potential Review. This course was created for developers of mental health or substance abuse programs, including principal investigators of relevant research studies, who are working toward submission to NREPP.

back to top

Will every intervention that is submitted be reviewed?

No. Each submission undergoes a preliminary assessment to determine whether it has met the minimum requirements outlined in the Federal Register notice published the summer before each submission period. Submissions that do meet the minimum requirements may receive an invitation for review from SAMHSA, depending on the availability of current funding. In addition, SAMHSA may choose to give special consideration to interventions that meet one or more conditions, as outlined in the Federal Register notice.

back to top

Will there be another application cycle?

Yes. Open submission periods are generally held annually from November to February. To receive announcements about future open submission periods, please sign up for our Submissions mailing list.

back to top

Can I submit multiple interventions for review?

Yes. However, each intervention must be submitted separately.

back to top

Can interventions developed outside the United States be reviewed by NREPP?

Yes, if they meet the minimum requirements and have implementation materials that can be disseminated in the United States.

back to top

Why aren't prevention and treatment interventions considered under separate criteria?

The developers and Federal sponsors of NREPP believe that the scientific standards for the evidence-based practice assessments contained within NREPP apply equally to the prevention and treatment components of behavioral health care.

back to top

Information for Developers Whose Intervention Has Been Accepted for Review and/or Posted

My intervention has been accepted for review. How can I refer to its status, as well as NREPP, in my materials?

Intervention materials, including Web sites, can state that the intervention has been accepted for review by NREPP. NREPP maintains a list of interventions that have been Accepted for Review, and an intervention's inclusion on this list can be noted. However, acceptance for review does not constitute an endorsement, promotion, or approval of the intervention by NREPP or SAMHSA.

back to top

Once my intervention has been reviewed by NREPP, am I required to post the results on the NREPP Web site?

Although NREPP is a voluntary registry, each NREPP review represents a considerable investment of time and public funds. Any individual or organization that applies for an NREPP review is expected to authorize publication of the results in the form of an intervention summary. In rare cases, a program developer may choose to decline the posting of a summary. To provide full disclosure to all users of this site, NREPP will publish the names of these reviewed, but unlisted, interventions on the NREPP Web site, and note that the developer declined to post the intervention summary.

back to top

After my intervention has been reviewed, will I be able to approve the intervention summary before it is posted on the Web site?

NREPP works closely with the program developer throughout the review and posting process. After the intervention summary has been completed, the developer is given the opportunity to review the summary and provide comments. The intervention summary is posted on the NREPP Web site after it has been approved by both the developer and SAMHSA.

back to top

My intervention has been reviewed and posted on the site. How can I refer to NREPP in my materials?

Intervention materials, including Web sites, can state that the intervention has been reviewed and posted on the NREPP Web site. Shortly after the intervention summary has been posted on the NREPP Web site, the program developer will receive an email with image files of the NREPP logo. The logo also can be used in the intervention materials. However, the posting of an intervention summary on the NREPP Web site does not constitute an endorsement, promotion, or approval of the intervention by NREPP or SAMHSA.

back to top

Does my intervention qualify for a second review?

A second review may be conducted for an intervention summarized on the NREPP Web site if the original review occurred at least 5 years ago. The second review may be limited to Quality of Research (QOR) or Readiness for Dissemination (RFD), or it may include both, depending on the following requirements:

  • To qualify for a second QOR review, developers need to have (1) a new study (or studies) of the intervention that evaluated the same intervention with the same target population or a slightly modified intervention with the same or different target population or (2) updates to the study (or studies) already reviewed by NREPP, with results from secondary analysis of the original dataset and/or analysis of longitudinal data. In either case, the studies must meet NREPP's minimum requirements for review: the studies have produced statistically significant findings related to substance abuse or mental health behavioral outcome(s) at the p ≤ .05 level, have used a quasi-experimental or experimental design, and have been described in a published form (journal article and/or comprehensive evaluation report).
  • To qualify for a second RFD review, the intervention's dissemination materials must have changed markedly--or new materials must have been developed--since the original review was conducted.

NREPP has already extended invitations for a second review to several cohorts of interventions, and new cohorts will be invited depending on available funding.

back to top