Evaluation of the U.S. Preventive Services Task Force Recommendations for Clinical Preventive Services: Final Contract Report. Appendix: Interview Protocols

You Are Here: AHRQ Home > About AHRQ > Mission & Budget > Evaluation of the U.S. Preventive Services Task Force Recommendations for Clinical Preventive Services > Appendix: Interview Protocols

Evaluation of the U.S. Preventive Services Task Force Recommendations for Clinical Preventive Services

Appendix: Interview Protocols

Medical Director Protocol
Clinical Advisor Protocol
Director of Quality Improvement Protocol
Director of Health IT or Health IT Staff Protocol (Part I)
Director of Health IT or Health IT Staff Protocol (Part II)

Medical Director Protocol

We are interested in learning more about how clinical preventive services, and especially the USPSTF recommendations are integrated into [Health System].

1. First, how familiar are you with the U.S. Preventive Services Task Force recommendations for clinical preventive services?

2. Are all or some of the services that are recommended by the Task Force ["A" and "B" recommendations] being delivered throughout your healthcare system?

Can you provide a few examples of how the recommendations are integrated?
Are there any examples of screenings that you stopped delivering because of Task Force recommendations against routine screening? ["D" recommendations]
If you don't implement all of the recommendations, what are the reasons or criteria [Health System] uses to select which recommendations will be integrated?
What is your role in implementing clinical preventive services, or ensuring that they are implemented, in your Health System?
Can you describe the process [Health System] uses to review clinical preventive services recommendations, from the Task Force and from other sources, when deciding whether or not those recommendations should be adopted and integrated into your system?
What kinds of staff and staff changes are needed to ensure that the Task Force recommendations are integrated into the system and to provide the services recommended by the Task Force?

3. Is your system engaged in practice change or continuous quality improvement activities?

Can you provide a few examples of activities that were designed to increase the appropriate delivery of a Task Force recommendation? For the purposes of this interview, I'd like you to keep in mind that we're defining "appropriate delivery" as following the Task Force's grading scheme of A, B, C, D, I.
When [Health System] designs a practice change or continuous quality improvement activity to increase the appropriate delivery of a Task Force recommendation, how do you encourage implementation at the practice or clinician level?

Now I'd like to talk a little about reimbursement at [Health System].

4. Does the reimbursement structure at [Health System] reward the appropriate delivery of clinical preventive services?

Can you provide examples of Task Force recommendations that are integrated into the reimbursement structure?
We understand that certain recommendations are more difficult to incorporate into the reimbursement structure. Do you think that there are any disincentives to the appropriate delivery of clinical preventive services in the reimbursement structure?
When [Health System] makes a change in the reimbursement structure to reward the appropriate delivery of clinical preventive services, how long does it generally take to see changes in clinicians' delivery of the service?
Do you believe the reimbursement structure is an effective means for increasing appropriate delivery of clinical preventive services?

We're also interested in learning more about the barriers you face in trying to adopt and integrate the Task Force recommendations at [Health System], so I'd like to ask you a few questions about that now.

5. What barriers do you face to adopting the Task Force recommendations at the systems-level?

Are there fundamental issues with the recommendations themselves—such as the topics or populations the recommendations address—that prevent adoption and implementation at your system?
Are certain types of Task Force recommendations easier to adopt and integrate than others? For example, are screening recommendations easier to adopt and integrate than counseling recommendations, or vice-versa?

I'd like to talk now about how information resources regarding clinical preventive services affect their use at your system.

6. How can AHRQ improve its dissemination of the Task Force recommendations to improve adoption rates at the systems-level?

Who is the primary target audience within your Health System for information about recommendations from the Task Force? In other words, when the Task Force disseminates new information about the recommendations, who should they contact at your Health System?
Are the recommendations "packaged" in a user-friendly way to make review, adoption, and integration as straightforward for you as possible?
Do you believe it would be helpful if the Task Force did more to disseminate information about its processes for selecting and prioritizing recommendations? By that, I mean would knowing more about the processes behind the recommendations affect the way in which [Health System] adopts and integrates the Task Force recommendations? Do you feel it is important to understand those processes?
Do you regularly check for updated evidence and recommendations from the Task Force?
Are there any specific tools or information AHRQ could provide you to improve integration of the recommendations at the systems-level?

7. Other than the Task Force, what are your other sources of information for clinical preventive services?

How do [those/their] recommendations compare to the Task Force recommendations in terms of the ease of adoption and integration?

8. Are the Task Force's prevention priorities aligned with other systems-level variables, such as payer expectations, industry quality indicators, and consumer demand, that you have to manage?

Can you provide examples where the recommendations are and are not aligned with other factors?
Do you have suggestions for what the Task Force can do to better align its recommendations with other factors that you need to weigh when deciding whether or not to adopt a recommendation?

9. Are the Task Force's prevention priorities aligned with federal initiatives, such as the National Health Quality or Disparities reports, and/or state initiatives that you are required to, or choose to, follow?

Can you provide examples where the recommendations are and are not aligned with other factors?
How could the Task Force better align itself with other federal and/or state initiatives that you follow?

To wrap up, we're interested in learning more about the characteristics of organizations where the theme of prevention is integrated across all of its activities. I have two final questions for you.

I. What characteristics should we look for in order to recognize an organization where prevention is considered a "core value" throughout the system?

II. What are the characteristics of organizations that have successfully implemented evidence-based clinical preventive services recommendations?

Return to Appendix Contents

Clinical Advisor Protocol

1. First, how familiar are you with the U.S. Preventive Services Task Force recommendations for clinical preventive services?

We are interested in learning more about how clinical preventive services, and especially the USPSTF recommendations are integrated into [Health System].

2. Are all or some of the services that are recommended by the Task Force ["A" and "B" recommendations] being delivered throughout your healthcare system?

Can you provide a few examples of how the recommendations are integrated?
Are there any examples of screenings that you stopped delivering because of Task Force recommendations against routine screening? ["D" recommendations]
If you don't implement all of the recommendations, what are the reasons or criteria [Health System] uses to select which recommendations will be integrated?
What is your role in implementing clinical preventive services, or ensuring that they are implemented, in your Health System?
Can you describe the process [Health System] uses to review clinical preventive services recommendations, from the Task Force and from other sources, when deciding whether or not those recommendations should be adopted and integrated into your system?
What kinds of staff and staff changes are needed to ensure that the Task Force recommendations are integrated into the system and to provide the services recommended by the Task Force?

3. Is your system engaged in practice change or continuous quality improvement activities?

Can you provide a few examples of activities that were designed to increase the appropriate delivery of a Task Force recommendation? For the purposes of this interview, I'd like you to keep in mind that we're defining "appropriate delivery" as following the Task Force's grading scheme of A, B, C, D, I.
When [Health System] designs a practice change or continuous quality improvement activity to increase the appropriate delivery of a Task Force recommendation, how do you encourage implementation at the practice or clinician level?

4. Does the reimbursement structure at [Health System] reward the appropriate delivery of clinical preventive services?

Can you provide examples of Task Force recommendations that are integrated into the reimbursement structure?
We understand that certain recommendations are more difficult to incorporate into the reimbursement structure. Do you think that there are any disincentives to the appropriate delivery of clinical preventive services in the reimbursement structure?
When [Health System] makes a change in the reimbursement structure to reward the appropriate delivery of clinical preventive services, how long does it generally take to see changes in clinicians' delivery of the service?
Do you believe the reimbursement structure is an effective means for increasing appropriate delivery of clinical preventive services?

We're also interested in learning more about the barriers you face in trying to adopt and integrate the Task Force recommendations at [Health System].

5. What barriers do you face to adopting the Task Force recommendations at the systems-level?

Are there fundamental issues with the recommendations themselves—such as the topics or populations the recommendations address—that prevent adoption and implementation at your system?
Are certain types of Task Force recommendations easier to adopt and integrate than others? For example, are screening recommendations easier to adopt and integrate than counseling recommendations, or vice-versa?

6. How can AHRQ improve its dissemination of the Task Force recommendations to improve adoption rates at the systems-level?

Who is the primary target audience within your Health System for information about recommendations from the Task Force? In other words, when the Task Force disseminates new information about the recommendations, who should they contact at your Health System?
Are the recommendations "packaged" in a user-friendly way to make review, adoption, and integration as straightforward for you as possible?
Do you believe it would be helpful if the Task Force did more to disseminate information about its processes for selecting and prioritizing recommendations? By that, I mean would knowing more about the processes behind the recommendations affect the way in which [Health System] adopts and integrates the Task Force recommendations? Do you feel it is important to understand those processes?
Do you regularly check for updated evidence and recommendations from the Task Force?
Are there any specific tools or information AHRQ could provide you to improve integration of the recommendations at the systems-level?

To wrap up, we're interested in learning more about the characteristics of organizations where the theme of prevention is integrated across all of its activities.

I. What characteristics should we look for in order to recognize an organization where prevention is considered a "core value" throughout the system?

II. What are the characteristics of organizations that have successfully implemented evidence-based clinical preventive services recommendations?

Return to Appendix Contents

Director of Quality Improvement Protocol

1. First, how familiar are you with the U.S. Preventive Services Task Force recommendations for clinical preventive services?

We are interested in learning more about how clinical preventive services, and especially the USPSTF recommendations are integrated into [Health System].

2. Are all or some of the services that are recommended by the Task Force ["A" and "B" recommendations] being delivered throughout your healthcare system?

Can you provide a few examples of how the recommendations are integrated?
Are there any examples of screenings that you stopped delivering because of Task Force recommendations against routine screening? ["D" recommendations]
If you don't implement all of the recommendations, what are the reasons or criteria [Health System] uses to select which recommendations will be integrated?
What is your role in implementing clinical preventive services, or ensuring that they are implemented, in your Health System?
Can you describe the process [Health System] uses to review clinical preventive services recommendations, from the Task Force and from other sources, when deciding whether or not those recommendations should be adopted and integrated into your system?
What kinds of staff and staff changes are needed to ensure that the Task Force recommendations are integrated into the system and to provide the services recommended by the Task Force?

3. Is your system engaged in practice change or continuous quality improvement activities?

Can you provide a few examples of activities that were designed to increase the appropriate delivery of a Task Force recommendation? For the purposes of this interview, I'd like you to keep in mind that we're defining "appropriate delivery" as following the Task Force's grading scheme of A, B, C, D, I.
When [Health System] designs a practice change or continuous quality improvement activity to increase the appropriate delivery of a Task Force recommendation, how do you encourage implementation at the practice or clinician level?

4. Does the reimbursement structure at [Health System] reward the appropriate delivery of clinical preventive services?

Can you provide examples of Task Force recommendations that are integrated into the reimbursement structure?
We understand that certain recommendations are more difficult to incorporate into the reimbursement structure. Do you think that there are any disincentives to the appropriate delivery of clinical preventive services in the reimbursement structure?
When [Health System] makes a change in the reimbursement structure to reward the appropriate delivery of clinical preventive services, how long does it generally take to see changes in clinicians' delivery of the service?
Do you believe the reimbursement structure is an effective means for increasing appropriate delivery of clinical preventive services?

5. What barriers do you face to adopting the Task Force recommendations at the systems-level?

Are there fundamental issues with the recommendations themselves—such as the topics or populations the recommendations address—that prevent adoption and implementation at your system?
Are certain types of Task Force recommendations easier to adopt and integrate than others? For example, are screening recommendations easier to adopt and integrate than counseling recommendations, or vice-versa?

6. How can AHRQ improve its dissemination of the Task Force recommendations to improve adoption rates at the systems-level?

Who is the primary target audience within your Health System for information about recommendations from the Task Force? In other words, when the Task Force disseminates new information about the recommendations, who should they contact at your Health System?
Are the recommendations "packaged" in a user-friendly way to make review, adoption, and integration as straightforward for you as possible?
Do you believe it would be helpful if the Task Force did more to disseminate information about its processes for selecting and prioritizing recommendations? By that, I mean would knowing more about the processes behind the recommendations affect the way in which [Health System] adopts and integrates the Task Force recommendations? Do you feel it is important to understand those processes?
Do you regularly check for updated evidence and recommendations from the Task Force?
Are there any specific tools or information AHRQ could provide you to improve integration of the recommendations at the systems-level?

7. Other than the Task Force, what are your other sources of information for clinical preventive services?

How do [those/their] recommendations compare to the Task Force recommendations in terms of the ease of adoption and integration?

8. Are the Task Force's prevention priorities aligned with other systems-level variables, such as payer expectations, industry quality indicators, and consumer demand, that you have to manage?

Can you provide examples where the recommendations are and are not aligned with other factors?
Do you have suggestions for what the Task Force can do to better align its recommendations with other factors that you need to weigh when deciding whether or not to adopt a recommendation?

Can you provide examples where the recommendations are and are not aligned with other factors?
How could the Task Force better align itself with other federal and/or state initiatives that you follow?

10. Are IT/HIT systems utilized for clinical preventive services integration?

Can you provide some examples of how the Task Force recommendations are integrated into your IT and HIT systems?
Do you provide clinical decision support systems to your physician members to support the appropriate delivery of the Task Force recommendations?
Do you have a central EMR?

[If Yes]: Are clinician reminders used?
[If No]: Do you encourage or provide support for participating clinicians to integrate reminders into their own EMR?
What are the challenges you face in tracking patient data for reminder systems?
What are the particular challenges you face, and what solutions have been implemented, in dealing with issues such as the ages at which services should be delivered and the frequency of service delivery when these are not explicitly addressed in the Task Force's recommendation?

11. Do you utilize your IT systems to measure and monitor the delivery of clinical preventive services for quality improvement purposes or as part of your performance improvement strategy?

What types of data are collected?
What are the processes for tracking and monitoring the data?
How would you compare the quality of data collected for screening recommendations compared to counseling recommendations?
When a recommendation is delivered by a non-physician member of the clinical staff, is it generally captured in [Health System]'s data monitoring system?
How frequently are data evaluated to determine if the [Health System] needs to intervene to increase the appropriate delivery of a recommendation?
What are the challenges you face in gathering and using this data for QI purposes?
Do you report the data on appropriate clinical preventive services delivery rates back to participating providers directly? Can you describe the provider feedback process?
Can you provide examples of how the measurement and monitoring of clinical preventive services data has led to a system-wide intervention, or an intervention targeted to a subset of participating providers?
Are patient outcomes evaluated relative to the delivery of recommended clinical preventive services?
Where quantitative measures don't already exist, how would you suggest they be designed to most effectively measure and track the delivery of a clinical preventive service over the long-term?

I. What characteristics should we look for in order to recognize an organization where prevention is considered a "core value" throughout the system?

II. What are the characteristics of organizations that have successfully implemented evidence-based clinical preventive services recommendations?

Return to Appendix Contents

Director of Health IT or Health IT Staff Protocol (Part I)

1. First, how familiar are you with the U.S. Preventive Services Task Force recommendations for clinical preventive services?

We are interested in learning more about how clinical preventive services, and especially the USPSTF recommendations are integrated into [Health System].

2. What is your role in implementing clinical preventive services, or ensuring that they are implemented, in your Health System?

3. Are IT/HIT systems utilized for clinical preventive services integration?

Can you provide some examples of how the Task Force recommendations are integrated into your IT and HIT systems?
Do you provide clinical decision support systems to your physician members to support the appropriate delivery of the Task Force recommendations?
Do you have an EMR or HIS (Hospital Information System)?

[If Yes]: Are clinician reminders used?
[If No]: Can clinicians choose to integrate their own reminders into the system?
What are the challenges you face in tracking patient data for reminder systems?
What are the particular challenges you face, and what solutions have been implemented, in dealing with issues such as the ages at which services should be delivered and the frequency of service delivery when these are not explicitly addressed in the Task Force's recommendation?

4. Do you utilize your IT systems to measure and monitor the delivery of clinical preventive services for quality improvement purposes or as part of your performance improvement strategy?

What types of data are collected?
What are the processes for tracking and monitoring the data?
If a recommendation is delivered by a non-physician staff member, is it generally captured in [Health System]'s data monitoring system?
How frequently are data evaluated to determine if the [Health System] needs to intervene to increase the appropriate delivery of a recommendation?
What are the challenges you face in gathering and using this data for QI purposes?
Is there a process for reporting data on appropriate clinical preventive services delivery rates back to participating providers? Can you describe the provider feedback process?
Can you give me some examples on the kinds of reports related to clinical preventive services that you routinely submit to your QI or PI department?
Can you provide examples of how the measurement and monitoring of clinical preventive services data has led to a system-wide intervention, or an intervention targeted to a subset of participating providers?
Where quantitative measures don't already exist, how would you suggest they be designed to most effectively measure and track the delivery of a clinical preventive service over the long-term?
When a new guideline or preventive service is accepted for integration into [Health System], what is your process for implementing it in your EMR/HIS? How long does that process usually take?

5. Are there any specific tools or information AHRQ could provide you to improve integration of the recommendations at the systems-level?

I. What characteristics should we look for in order to recognize an organization where prevention is considered a "core value" throughout the system?

II. What are the characteristics of organizations that have successfully implemented evidence-based clinical preventive services recommendations?

Return to Appendix Contents

Director of Health IT or Health IT Staff Protocol (Part II)

1. First, how familiar are you with the U.S. Preventive Services Task Force recommendations for clinical preventive services?

We are interested in learning more about how clinical preventive services, and especially the USPSTF recommendations are integrated into [Health System].

2. Are all or some of the services that are recommended by the Task Force ["A" and "B" recommendations] being delivered throughout your healthcare system?

Can you provide a few examples of how the recommendations are integrated?
Are there any examples of screenings that you stopped delivering because of Task Force recommendations against routine screening? ["D" recommendations]
If you don't implement all of the recommendations, what are the reasons or criteria [Health System] uses to select which recommendations will be integrated?
What is your role in implementing clinical preventive services, or ensuring that they are implemented, in your Health System?
Can you describe the process [Health System] uses to review clinical preventive services recommendations, from the Task Force and from other sources, when deciding whether or not those recommendations should be adopted and integrated into your system?
What kinds of staff and staff changes are needed to ensure that the Task Force recommendations are integrated into the system and to provide the services recommended by the Task Force?

3. Is your system engaged in practice change or continuous quality improvement activities?

Can you provide a few examples of activities that were designed to increase the appropriate delivery of a Task Force recommendation? For the purposes of this interview, I'd like you to keep in mind that we're defining "appropriate delivery" as following the Task Force's grading scheme of A, B, C, D, I.
When [Health System] designs a practice change or continuous quality improvement activity to increase the appropriate delivery of a Task Force recommendation, how do you encourage implementation at the practice or clinician level?

4. Are IT/HIT systems utilized for clinical preventive services integration?

Can you provide some examples of how the Task Force recommendations are integrated into your IT and HIT systems?
Do you provide clinical decision support systems to your physician members to support the appropriate delivery of the Task Force recommendations?
Do you have a central EMR?

[If Yes]: Are clinician reminders used?
[If No]: Do you encourage or provide support for participating clinicians to integrate reminders into their own EMR?
What are the challenges you face in tracking patient data for reminder systems?
What are the particular challenges you face, and what solutions have been implemented, in dealing with issues such as the ages at which services should be delivered and the frequency of service delivery when these are not explicitly addressed in the Task Force's recommendation?

5. Do you utilize your IT systems to measure and monitor the delivery of clinical preventive services for quality improvement purposes or as part of your performance improvement strategy?

What types of data are collected?
What are the processes for tracking and monitoring the data?
How would you compare the quality of data collected for screening recommendations compared to counseling recommendations?
When a recommendation is delivered by a non-physician member of the clinical staff, is it generally captured in [Health System]'s data monitoring system?
How frequently are data evaluated to determine if the [Health System] needs to intervene to increase the appropriate delivery of a recommendation?
What are the challenges you face in gathering and using this data for QI purposes?
Do you report the data on appropriate clinical preventive services delivery rates back to participating providers directly? Can you describe the provider feedback process?
Can you provide examples of how the measurement and monitoring of clinical preventive services data has led to a system-wide intervention, or an intervention targeted to a subset of participating providers?
Are patient outcomes evaluated relative to the delivery of recommended clinical preventive services?
Where quantitative measures don't already exist, how would you suggest they be designed to most effectively measure and track the delivery of a clinical preventive service over the long-term?

6. What barriers do you face to adopting the Task Force recommendations at the systems-level?

Are there fundamental issues with the recommendations themselves—such as the topics or populations the recommendations address—that prevent adoption and implementation at your system?
Are certain types of Task Force recommendations easier to adopt and integrate than others? For example, are screening recommendations easier to adopt and integrate than counseling recommendations, or vice-versa?

7. How can AHRQ improve its dissemination of the Task Force recommendations to improve adoption rates at the systems-level?

Who is the primary target audience within your Health System for information about recommendations from the Task Force? In other words, when the Task Force disseminates new information about the recommendations, who should they contact at your Health System?
Are the recommendations "packaged" in a user-friendly way to make review, adoption, and integration as straightforward for you as possible?
Do you believe it would be helpful if the Task Force did more to disseminate information about its processes for selecting and prioritizing recommendations? By that, I mean would knowing more about the processes behind the recommendations affect the way in which [Health System] adopts and integrates the Task Force recommendations? Do you feel it is important to understand those processes?
Do you regularly check for updated evidence and recommendations from the Task Force?
Are there any specific tools or information AHRQ could provide you to improve integration of the recommendations at the systems-level?

I. What characteristics should we look for in order to recognize an organization where prevention is considered a "core value" throughout the system?

II. What are the characteristics of organizations that have successfully implemented evidence-based clinical preventive services recommendations?

Return to Appendix Contents
Return to Document Contents

AHRQ Publication No. 08-M011-EF
Current as of December 2007

Internet Citation:

Evaluation of the U.S. Preventive Services Task Force Recommendations for Clinical Preventive Services Final Report. AHRQ Publication No. 08-M011-EF, December 2007, Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/about/evaluations/uspstf/