IV. Findings

To explore the adoption, integration, delivery, and dissemination of the USPSTF recommendations, we interviewed a total of 42 health plan staff members from four health plans across the U.S. The health plans differed in scope (number of lives covered and provider sites), geography (rural versus urban), organization and management structure, and technological sophistication. The closed-panel plan operates 31 medical centers and serves over 500,000 members regionally, while the hybrid plan operates a large ambulatory program that serves more than 245,000 members across 17 counties in one state. The governmental plan is unique in that it operates in concert with a larger public health system. The open-panel plan serves both urban and rural patient populations, and operates a PPO and large Medicaid managed care program. Before describing the findings, Exhibit 5 below provides an overview of key plan features to illustrate the differences among participating sites.

Exhibit 5: Features of Health Plans

Health Plans	Open-Panel Plan	Closed-Panel Plan	Hybrid Plan	Governmental Plan
Membership Size (< 300,000)			x
Membership Size (300,001-500,000)		x
Membership Size (> 500,001)	x			x
Service area is one state	x		x	x
Service area is regional		x
Serves a large rural population	x		x	x

Next, we present project findings related to: 1) adoption of the USPSTF recommendations, including familiarity with the recommendations, and the process of accessing, adopting and integrating the recommendations; 2) integration and delivery of the USPSTF recommendations, including incorporating the recommendations in to practice, major uses of health IT and quality improvement, the impact of reimbursement structures and other factors, and barriers to integration and delivery of the recommendations; and 3) improving dissemination of the USPSTF recommendations in health plans.

Return to Contents

I. Adoption of the USPSTF Recommendations in Health Plans

The synthesis of findings begins with the adoption of the USPSTF recommendations in the health care plans. We discuss respondents' familiarity with the recommendations, the methodology underlying the recommendations, and the "A" and "B" USPSTF recommendations. We then examine the process for adopting the USPSTF recommendations and the barriers to adopting the USPSTF recommendations. Finally, we examine the methods for accessing the recommendations.

1.1. Familiarity with the Recommendations

Familiarity with the USPSTF Recommendations. Study participants were asked: "How familiar are you with the USPSTF recommendations for clinical preventive services?" Overall, we found that the respondents were familiar with the USPSTF recommendations. However, reported familiarity varied dramatically. The majority of respondents indicated that they are "somewhat" or "fairly" familiar with the recommendations. Many respondents indicated that they have heard of the USPSTF recommendations, but do not know them well enough to cite specific examples. A few respondents indicated that they had not heard of the USPSTF recommendations at all prior to our interview. For analysis purposes, we depict respondents as "familiar plus" (indicating they have at least some familiarity with the recommendations) and "not familiar" (indicating they are not familiar with the recommendations). A small number of respondents also chose not to answer this question. Our results are presented in Exhibit 6.

Exhibit 6: Familiarity with the USPSTF Recommendations Across Health Plans

Familiarity with the USPSTF Recommendations	Familiar Plus	Not Familiar	No Answer
Closed-Panel (n=12)	9 (75%)	3 (25%)	0 (0%)
Open-Panel (n=10)	9 (90%)	0 (0%)	1 (10%)*
Governmental (n=11)	5 (45%)	5 (45%)	1 (9%)*
Hybrid (n=9)	7 (78%)	2 (22%)	0 (0%)
Total (n=42)	30	10	2

Note: Respondents were asked: "How familiar are you with the USPSTF recommendations for clinical preventive services?" Respondents categorized as "Familiar plus" had at least some familiarity with the USPSTF recommendations, while those categorized as "Not Familiar" had no familiarity with the recommendations.
*These respondents were a Director of Health IT and Health IT staff member, respectively.

Across plans, the closed-panel and open-panel plans had the largest number of respondents that were at least somewhat familiar with the USPSTF recommendations, though familiarity ranged from strong familiarity with the USPSTF recommendations to only having heard of the USPSTF recommendations. The government plan had the fewest number of respondents that were familiar with the USPSTF recommendations, with half of the respondents indicating that they had never heard of the USPSTF recommendations. The hybrid plan fell somewhere in between, with the majority of respondents having some familiarity with the USPSTF recommendations.

The respondents who were most familiar with the USPSTF recommendations for the open-panel plan, closed-panel plan, and the governmental plan fulfilled a similar role: the Clinical Preventive Services Guidelines Coordinator.⁵⁵ The CPS Guidelines Coordinator works within the departments of quality improvement or medical policy management to spearhead the adoption of CPS recommendations for each plan. As a result, the CPS Guidelines Coordinator had a very strong familiarity with recommendations from the USPSTF and other sources. Overall, the Directors of Quality Improvement and Clinical Advisors also tended to be more familiar with the USPSTF recommendations than Health IT Directors and Health IT Staff respondents.

Approximately one quarter of the respondents indicated that they were not familiar with the USPSTF recommendations or had not heard of the USPSTF prior to the interview. However, it is quite possible that this could be an underestimate. Interviewees were provided with a description of the project when we requested their participation, which included a link to the USPSTF recommendations. As a result, respondents indicating that they are "somewhat" or "fairly" familiar with the recommendations may have not been familiar prior to the interview.

Exhibit 7 explores which types of health plan respondents were unfamiliar with the recommendations across health plans. For each health plan, we indicate the number of respondents who were not familiar with the USPSTF recommendations. We also identify their roles in the health plans. Health IT Staff respondents were least familiar with the USPSTF recommendations, followed by Quality Improvement Staff and Clinical Advisors.

Exhibit 7: Respondents Who Were Not Familiar with the USPSTF Recommendations

Not Familiar with the USPSTF Recommendations	Open-Panel	Closed-Panel	Hybrid	Governmental	Total	Total/n
Medical Director (n=3)					0	0%
Director of QI (n=12)		1			1	8%
Director of Health IT (n=7)			1	1	2	29%
Clinical Advisor (n=9)				2	2	44%
QI Staff (n=4)				1	1	50%
Health IT Staff (n=6)		2	1	1	4	67%
Other (n=1)					0	0%
Total Unfamiliar (n=42)	0	3	2	5	10	24%

Note: Respondents were asked: "How familiar are you with the USPSTF recommendations for clinical preventive services?" Respondents categorized as "Familiar Plus" had at least some familiarity with the USPSTF recommendations, while those categorized as "Not Familiar" had no familiarity with the recommendations.

Familiarity with the Methodology Underlying the Recommendations. The USPSTF has a distinct process for selecting and prioritizing its recommendations. After reviewing the evidence and estimating the magnitude of benefits and harms for each preventive service, the USPSTF grades the strength of the evidence from "A" (strongly recommends) to "D" (recommends against) or "I" (insufficient evidence to recommend for or against). We did not explicitly ask respondents to describe their familiarity with the methodology underlying the USPSTF recommendations. However, our conversations with respondents provide evidence to suggest that many were unfamiliar with this methodology.

When asked the question, "are all or some of the services that are recommended by the USPSTF, the "A" and "B" recommendations, being delivered throughout your healthcare system," approximately 15% of respondents (6 out of 41 respondents) indicated that they could not answer because they were unfamiliar with the USPSTF's grading terminology.^c These respondents were Directors of Quality Improvement, Quality Improvement Staff, and a Clinical Advisor.

While some respondents were unfamiliar with the methodology underlying the recommendations, approximately 40% of all respondents indicated that they would like to know more about the process behind the USPSTF recommendations. We asked respondents whether it would be helpful to know more about the process the USPSTF uses to select and prioritize the recommendations: "Do you believe it would be helpful if the Task Force did more to disseminate information about its process for selecting and prioritizing recommendations? By that, I mean, would knowing more about the process behind the recommendations affect the way in which [the health plan] adopts and integrates the Task Force recommendations?" Seven out of the 12 Directors of Quality Improvement and three out of the four Quality Improvement staff members indicated that they would like to know more information about the process used by the USPSTF. Medical Directors were also interested in learning more about the process behind the USPSTF recommendations. Interestingly, Directors of Quality Improvement from both the open- and closed-panel plans indicated that providers would be the most likely people to desire a better understanding of the process underlying the recommendations. We did not find this to be the case.

Only about half of the Clinical Advisors were interested in receiving more information about the process used by the USPSTF. Clinical Advisor respondents indicated that they already understand the methodology underlying the recommendations, or they "have trust in the process used by the USPSTF."

Another interesting finding is that, from our conversations with respondents, it appears that many people do not understand that the USPSTF utilizes a rigorous cost-benefit methodology to grade its recommendations. For example, one respondent at the closed-panel plan responded that the USPSTF does not specifically provide information about the cost-benefit analysis that underlies the recommendations:

"I'll tell you what most frontline physicians don't understand—the Chlamydia screening and that there is an upper age limit. They want to screen 50 year olds when the benefits or lack thereof are poorly understood. One of the things that would be very helpful would be if the Task Force completed an analysis of the costs and benefits as one moves out of the arena of routine screening. I've looked closely at Chlamydia. The positivity rate of [screening people] 45 years and older approaches the false-positivity rate of the test. People in public health understand that, but I don't think that's out there explicitly as part of a recommendation."

This respondent indicated that it would be helpful to know more about the methodology employed by the USPSTF.

Familiarity with USPSTF Tools and Products. Many respondents were not aware that AHRQ disseminates a line of tools and products that incorporate the USPSTF recommendations, such as the Put Prevention into Practice materials, the Electronic Preventive Services Selector (ePSS), a pocket manual of recommendations, and email updates. Health IT Directors and Staff, in particular, were unaware of the products disseminated by the Task Force. This finding became apparent when a number of respondents in various positions recommended that AHRQ create "new" tools and products that already exist.

• A Clinical Advisor for the governmental plan recommended that AHRQ would improve dissemination of the USPSTF recommendations by creating a small hand-held manual of the recommendations for clinicians. Given that a "pocket guide" for clinicians is already available (downloadable from the AHRQ Web site), it is clear that this Clinical Staff member was not familiar with the USPSTF prevention tools available from AHRQ.
• A Director of Quality Improvement from the closed-panel plan recommended that AHRQ categorize the recommendations according to patient characteristics. The respondent indicated that the recommendations are only user-friendly "if you want to look up recommendations for a specific disease" but not user-friendly "if you want to look up the recommendations for a specific patient." The respondent was not aware that AHRQ already has tools such as the Electronic Preventive Services Selector (ePSS), which enables clinicians to search the recommendations by specific patient characteristics, such as age, sex, and selected behavioral risk factors.⁵⁶
• A Medical Director respondent from the governmental plan suggested that AHRQ could disseminate more information about the process the USPSTF uses to select and prioritize the recommendations by including a link about the Task Force's methodology, accessible via the Web site. The USPSTF Web site currently has a link about the methodology as well as information on the questions and answers page that addresses the process used to select and prioritize recommendations.
• When asked if there are any specific tools or information that AHRQ could provide to improve the integration of the Task Force recommendations at the systems-level, a Director of Quality Improvement from the closed-panel plan asked: "Do they put out a newsletter? A lot of organizations have newsletters or links that can be sent out to people who are interested in being subscribers to their journals. Then, you can read all you want. I don't remember seeing anything from [the Task Force]."
• A Director of Quality Improvement from the closed-panel plan indicated that he/she was unaware that the Task Force offered email updates about new recommendations. The USPSTF does enable interested users to sign up for the AHRQ Prevention Program listserv which circulates new and updated recommendations from the USPSTF and new resources from the Put Prevention into Practice program.
• A Director of Quality Improvement from the open-panel plan indicated that it would be helpful to know more about the Task Force's tools and products. When asked if there are any specific tools or information that AHRQ could provide to improve integration of the recommendations at the systems-level, the respondent told us: "I'm not familiar enough with the tools. I'm only familiar with the written [recommendations]—the full text and summary [recommendations] on the Web site. I did notice that there are personal digital assistants (PDA) downloads available. I don't know if those are available in sections or by different categories."

---

^cThe sample size for this question is 41 respondents rather than 42. An expert interview was conducted with a senior-level manager at the governmental plan's system headquarters. The respondent was not asked this question.

---

Return to Contents
Proceed to Next Section