Medical Office Survey on Patient Safety Culture

2008 Preliminary Comparative Results

---

Preliminary benchmarks are provided for the items and safety culture dimensions on the AHRQ Medical Office Survey on Patient Safety Culture to allow medical offices to compare their survey results against the results from 182 medical offices that participated in a pilot test of the survey in the United States in late 2007.

Select to download the print version of this document (PDF File, 550 KB) PDF Help.

---

Contents

Purpose and Use of this Document
Overview
Survey Background
Description of the 182 Pilot Study Medical Offices
Description of Pilot Study Medical Office Respondents
Item-Level Comparative Results
Composite-Level Comparative Results
Composite Percent Positive Scores
Comparative Results Using Percentiles

Purpose and Use of this Document

Preliminary benchmarks are provided for the items and safety culture dimensions on the AHRQ Medical Office Survey on Patient Safety Culture to allow medical offices to compare their survey results against the results from 182 medical offices that participated in a pilot test of the survey in the United States in late 2007.

Note: When comparing your medical office's results against the comparative results provided in this document, keep in mind that these results are from limited numbers of staff and medical offices. The results will provide only a general indication of how your medical office compares with other medical offices in the United States.

At this time, there is no central repository for medical offices to submit data for benchmarking purposes. However, similar to the AHRQ Hospital Survey on Patient Safety Culture Comparative Database (http://www.ahrq.gov/qual/hospculture), AHRQ plans to support an annual U.S. comparative database on the medical office survey that will provide more extensive comparative data. More details will be forthcoming from AHRQ about when data submission will begin and when the new comparative results will be available.

Return to Contents

Overview

Preliminary comparative data are provided for the survey items and for composite scores on the safety culture dimensions based on pilot data obtained from 4,174 staff from 182 medical offices. Basic descriptive data are provided about the respondents and medical offices that participated in the pilot study. This document also contains a description of how to calculate your medical office's composite scores on the Medical office Survey on Patient Safety Culture.

Return to Contents

Survey Background

The Medical Office Survey on Patient Safety is an expansion of AHRQ's Hospital Survey on Patient Safety Culture, which was pilot tested and made available to the public in November 2004 (http://www.ahrq.gov/qual/hospculture). The Medical Office Survey on Patient Safety Culture was specifically designed to measure the culture of patient safety in medical offices from the perspectives of providers and staff.

Safety culture can be defined as the set of values, beliefs, and norms about what is important, how to behave, and what attitudes are appropriate regarding patient safety in a work group or organization. The Medical Office Survey on Patient Safety Culture is intended to help a medical office assess the extent to which its organization's culture emphasizes the importance of patient safety, facilitates teamwork and open discussion about mistakes, and creates an atmosphere of continuous learning and improvement.

The survey design team reviewed the literature on patient safety problems and issues in medical offices. They also interviewed more than two dozen experts and researchers on patient safety in medical offices and asked many medical office providers and staff to identify appropriate survey topics. They then drafted sets of survey dimensions and items for review by experts. The draft survey was pretested with medical office providers and staff to ensure that the questions were easy to understand and answer and that the items were relevant. The pilot test survey was then administered in 182 medical offices, the data were analyzed to examine the survey's psychometric properties (reliability and factor structure), and the length was shortened by dropping items.

The final survey includes 52 survey items that measure the following 12 areas of organizational culture pertaining to resident safety:

1. Communication About Error.
2. Communication Openness.
3. Information Exchange With Other Settings.
4. Office Processes and Standardization.
5. Organizational Learning.
6. Overall Perceptions of Patient Safety and Quality.
7. Owner/Managing Partner/ Leadership Support for Patient Safety.
8. Patient Care Tracking/Followup.
9. Patient Safety and Quality Issues.
10. Staff Training.
11. Teamwork.
12. Work Pressure and Pace.

The survey uses 6-point frequency scales ("Daily" to "Not in the past 12 months"), 5-point frequency scales ("Never" to "Always"), or 5-point scales of agreement ("Strongly disagree" to "Strongly agree"). Most items include a "Does not apply or Don't know" option.

The survey also includes two overall rating questions that ask respondents to rate their medical office in five areas of health care quality (patient-centerd, effective, timely, efficient, equitable) and to provide an overall rating on patient safety. The last section of the survey asks respondents to write any comments they have about patient care and safety in their medical office.

Return to Contents

Description of the 182 Pilot Study Medical Offices

The pilot survey administration was designed to contain a diverse sample of medical offices, but the medical offices that voluntarily participated in the pilot study were not statistically representative of all medical offices in the United States. Completed responses in the pilot study database represent 182 medical offices dispersed across 21 States. Overall response results for the pilot study are summarized in Table 1.

Table 1. Overall statistics for participating medical offices

Total number of participating medical offices	182
Total number of individual survey respondents	4,174
Total number of surveys administered	5,931
Overall medical office response rate	70%
Average number of completed surveys per medical office (range: 3 to 192 surveys)	23
Average number of surveys administered per medical office (range: 5 to 376 surveys)	33
Overall average medical office response rate (range: 33% to 100%)	74%

Medical office characteristics were obtained from a designated point-of-contact in each medical office or by headquarters staff for a health care system. Tables 2 to 5 show the distribution of medical offices by size of office (number of physicians), total number of providers and staff, type of specialty, and number of medical offices by specialty. According to the data in these tables:

• More than half (55 percent) of the 182 medical offices had five or fewer physicians (MD or DO).
• The total number of providers and staff in the pilot study medical offices ranged from 5 to 376. Forty-five percent of offices had 20 or fewer providers and staff.
• Of the 182 medical offices in the pilot study, almost two-thirds were single specialty offices.
• Most of the medical offices reported that they provide primary care, but a wide range of other specialty care was represented.

Note: Column percent totals in the tables may not add to exactly 100 percent because of rounding of decimals.

Table 2. Distribution of medical offices by size of office

Number of Physicians (MD or DO)	All Medical Offices
	Number	Percent
0-2	32	18%
3-5	67	37%
6-10	45	25%
More than 10	38	21%
TOTAL	182	101%

 

Table 3. Distribution of medical offices by total number of providers and staff

Total Number of Providers and Staff	All Medical Offices
	Number	Percent
3-10	27	15%
11-20	55	30%
21-30	37	20%
31-40	22	12%
41-50	16	9%
51-60	7	4%
61-70	4	2%
More than 70	14	8%
TOTAL	182	100%

 

Table 4. Distribution of medical offices by type of specialty

Type of Specialty	All Medical Offices
	Number	Percent
Single specialty	114	63%
Multispecialty with primary care only (e.g., family medicine, internal medicine, pediatrics, OB/GYN, general practice)	29	16%
Multispecialty with primary care and specialty care	28	15%
Multispecialty with specialty care only	11	6%
TOTAL	182	100%

 

Table 5. Number of medical offices by specialty

Number of Medical Offices	Specialty	Number of Medical Offices	Specialty
3	Allergy/Immunology	1	Nephrology
4	Cardiology	5	Neurology
2	Child and Adolescent Psychiatry	18	OB/GYN or GYN
2	Dermatology	1	Ophthalmology
1	Diagnostic Radiology	12	Orthopedics
7	Endocrinology/Metabolism	4	Otolaryngology
68	Family Practice/Family Medicine	49	Pediatrics
4	Gastroenterology	4	Physical Medicine and Rehabilitation
1	General Practice	8	Psychiatry
1	General Preventive Medicine	1	Pulmonary Medicine
10	General Surgery	1	Radiology
6	Geriatrics	5	Rheumatology
6	Hematology/Oncology	4	Surgery (All)
41	Internal Medicine	4	Urology
1	Medical Genetics	14	Other specialty

Medical offices in the overall pilot study also had the following characteristics:

Number of office locations

• 69% had only one office location.

Majority ownership

• For 59%, the majority owner was a hospital or health care system.
• 25% were owned by physicians or providers, or both.
• 14% were owned by a university or medical school or academic medical institute.

Accreditation status

• 75% were not accredited by a national organization.

Patient visits per week

• The average number of patient visits per provider each week was 68.
• The average number of patient visits per week across all providers in a medical office was 515.

Status of electronic tools

• 94% had fully implemented electronic billing of services.
• 21% had fully implemented electronic ordering of medications (45% were in the process of implementing this tool).
• 32% had fully implemented electronic medical records (37% were in the process of implementing EMR).
• Sixteen of the medical offices in the pilot study (9%) had fully implemented electronic billing, appointment scheduling, ordering of medications, ordering of tests, access to patients' test or imaging results, and electronic medical records.

Return to Contents

Description of Pilot Study Medical Office Repsondents

Tables 6 to 8 display distributions of the 4,174 medical office respondents respondents by staff category, tenure in the medical office, and hours worked per week in the medical office.

According to the data on respondent characteristics shown in these tables:

• 20% of respondents were physicians.
• About 60% had worked in the medical office for at least 3 years.
• 75% worked 33 hours or more each week in the medical office.

Note: Individuals who did not respond to a question are shown as missing in the following tables and are excluded from total percentages. Column percent totals in the tables may not add to exactly 100% because of rounding of decimals.

Table 6. Distribution of pilot study respondents by staff category

Staff Category	Number	Percent
Administrative or clerical staff	1,158	28%
Physician (MD or DO)	826	20%
Other clinical staff or clinical support staff	772	19%
Registered Nurse (RN), Licensed Vocational Nurse (LVN), Licensed Practical Nurse (LPN)	655	16%
Management	326	8%
Other position	184	4%
Physician Assistant, Nurse Practitioner, Clinical Nurse Specialist, Nurse Midwife, Advanced Practice Nurse, etc.	169	4%
TOTAL	4,090	99%
Missing: No answer	84
Overall total	4,174

 

Table 7. Distribution of pilot study respondents by tenure in the medical office

Tenure in Medical Office	Number	Percent
Less than 2 months	47	1%
2 to 11 months	572	14%
1 to 2 years	944	24%
3 to 5 years	802	20%
6 to 10 years	834	21%
11 years or more	798	20%
TOTAL	3,997	100%
Missing: No answer	177
Overall total	4,174

 

Table 8. Distribution of pilot study respondents by hours worked per week in the medical office

Hours Worked per Week in the Medical Office	Number	Percent
1 to 4	64	2%
5 to 16	193	5%
17 to 24	303	8%
25 to 32	419	11%
33 to 40	1,813	45%
41 or more	1,193	30%
TOTAL	3,985	101%
Missing: No answer	189
Overall total	4,174

Return to Contents

Item-Level Comparative Results

To compare your medical office's results on any item from the Medical Office Survey on Patient Safety Culture, you first need to calculate your medical office's percentage of positive response on each item.

For positively worded items, this means simply calculating the total percentage of respondents who answered positively—combined percentage of "Strongly agree" and "Agree" responses, or "Always" and "Most of the time" responses, depending on the response categories used for the item.

Note: For Section B (Information Exchange With Other Settings), the percentage of positive response is based on the combined percentage of "Problems once or twice in the past 12 months" and "No problems in the past 12 months."

For negatively worded items, calculate the total percentage of respondents who answered negatively—combined percentage of "Strongly disagree" and "Disagree" responses, or "Never" and "Rarely" responses, since a negative answer on these items indicates a positive response).

Note: For Section A (List of Patient Safety and Quality Issues), all items are negatively worded so the combined percentage of "Once or twice in the past 12 months" and "Not in the past 12 months" represent positive responses.

Once you have calculated your medical office's percentage of positive response on each item, compare your item-level results with the average percentages on the survey's items from the 182 pilot study medical offices, shown in Chart 1.

Use a difference of 5 percentage points as a rule of thumb when comparing your medical office's results to the comparative results. Your medical office's percentages should be at least 5 points higher than the comparative results to be considered "better" (e.g., 75% vs. 70%) and should be at least 5 points lower to be considered "lower" than the comparative results (e.g., 60% vs. 65%).

Keep in mind that this information provides only relative comparisons. Even though your medical office's results may be better than the comparative results, you may still believe there is room for improvement in an absolute sense.

Chart 2 shows the distribution of responses for the Overall Ratings on Quality and Chart 3 shows the results for the Overall Rating on Patient Safety.

Return to Contents

Composite-Level Comparative Results

In addition to comparing your medical office's results on each item, you can obtain a summary view of how your medical office compares with other medical offices by examining composite scores. A composite score summarizes how respondents answered groups of items that all measure the same thing. A composite score summarizes how respondents answered groups of items that all measure the same thing. Composite scores on the 12 patient safety culture survey dimensions tell you the average percentage of respondents who answered positively when looking at the survey items that measure each safety culture dimension. Composite scores allow a summary comparison because you compare against only 12 safety culture dimensions rather than 52 separate survey items.

Return to Contents

Composite Percent Positive Scores

To calculate your medical office's composite score on a particular safety culture dimension, simply average the percent positive response on each item that is included in the composite. Here is an example of computing a composite score for Office Processes and Standardization:

There are four items in this composite—two are positively worded (items C9 and C15) and two are negatively worded items (items C8 and C12). Keep in mind that DISAGREEING with a negatively worded item indicates a POSITIVE response.

Calculate the percent positive response at the item level (Table 9).

Table 9. Example of How To Calculate Item and Composite Percent Positive Scores

Four items measuring Office Processes and Standardization	For positively worded items, the # of "Strongly agree" or "Agree" responses	For negatively worded items, the # of "Strongly disagree" or "Disagree" responses	Total # of responses to the item (excluding NA/DK & missing responses)	Percent positive response on item
Item C9-positively worded:
"We have good procedures for checking that work in this office was done correctly"	24	NA*	52	24/52=46%
Item C15-positively worded:
"Staff in this office follow standardized processes to get tasks done"	26	NA*	50	25/50=52%
Item C8-negatively worded:
"This office is more disorganized than it should be"	NA*	22	48	22/48=46%
Item C12-negatively worded:
"We have problems with workflow in this office"	NA*	28	50	28/50= 56%
* NA = Not applicable	Average percent positive response across the 4 items = 50%

In this example, there were four items, with percent positive response scores of 46 percent, 52 percent, 46 percent, and 56 percent. Averaging these item-level percent positive scores (46% + 52% + 46% + 56% / 4 = 50%) results in a composite score of .50 or 50 percent on Office Processes and Standardization. That is, an average of about 50 percent of the respondents responded positively to the survey items in this composite.

Use a difference of 5 percentage point difference as a rule of thumb when comparing your medical office's results to the comparative results. Your medical office's percentages should be at least 5 points higher than the comparative results to be considered "better" and should be at least 5 points lower to be considered "lower" than the comparative results.

Keep in mind that this information provides only relative comparisons. Even though your medical office's results may be better than the comparative results, you may still believe there is room for improvement in an absolute sense.

Chart 4 shows the average percent positive response from the 182 pilot study medical offices on each of the survey's patient safety culture composites, in order from most positive to least positive.

Return to Contents

Comparative Results Using Percentiles

In addition to comparing percent positive scores from your medical office with the average percent positive scores from the pilot test medical offices, you may also find it useful to examine additional comparative statistics: minimum and maximum scores and percentiles.

Minimum and maximum scores. The minimum (lowest) and maximum (highest) percent positive scores are presented for each composite. These scores provide information about the range of percent positive scores from the pilot test medical offices and are actual scores from the lowest and highest scoring offices. When comparing with the minimum and maximum scores, keep in mind that these scores may represent offices that are extreme outliers.

Percentiles. The 25th, 50th and 75th percentile scores are displayed for the survey composites. Percentiles provide information about the distribution of medical office scores. To calculate percentile scores, percent positive scores from the 182 pilot test medical offices were rank ordered from low to high. A specific percentile score shows the percentage of medical offices that scored at or below a particular score. For example, the 75th percentile is the percent positive score where 75 percent of the pilot test medical offices scored the same or lower, and 25 percent of the medical offices scored higher. Interpret the percentile scores as shown in Table 10.

Table 10. Interpretation of Percentile Scores

Percentile Score	Interpretation
25th percentile This score represents lower-scoring medical offices	25% of the medical offices scored the same or lower 75% of the medical offices scored higher
50th percentile This score represents middle-scoring medical offices	50% of the medical offices scored the same or lower 50% of the medical offices scored higher
75th percentile This score represents higher-scoring medical offices	75% of the medical offices scored the same or lower 25% of the medical offices scored higher

For example: For a survey composite in Table 11, the 25th percentile score is 49 percent positive, and the 50th percentile score is 62 percent positive.

If your medical office's score on the composite is 55 percent positive, it falls above the 25th percentile (but below the 50th percentile), meaning that your medical office scored higher than at least 25 percent of the other medical offices.

If your medical office's score on the composite is 65 percent positive, it falls above the 50th percentile, meaning your medical office scored higher than at least 50 percent of the other medical offices.

Table 11. Sample Percentile Statistics

Survey Composite	Composite Percent Positive Response
	Min	25th Percentile		50th Percentile		75th Percentile	Max
Composite 1	8%	49%	1	62%	2	85%	96%

¹ If your medical office's score is 55%, your score falls here.
² If your medical office's score is 65%, your score falls here.

For each patient safety culture composite, Table 12 shows the average percent positive scores, minimum score, 25th percentile, 50th percentile, 75th percentile, and maximum score for the 182 pilot study medical offices.

Table 12. Composite-Level Minimum, Maximum, and Percentile Results From 182 Pilot Study Medical Offices

Patient Safety Culture Composites		Pilot Study Medical Offices Average % Positive	% Positive Response
			Median/
			Min	25th %ile	50th %ile	75th %ile	Max
1	Teamwork	84%	28%	76%	86%	93%	100%
2	Patient Care Tracking/Followup	78%	41%	71%	79%	87%	100%
3	Organizational,Learning	75%	26%	68%	75%	83%	100%
4	Overall Perceptions of Patient Safety and Quality	75%	34%	67%	75%	86%	100%
5	Staff Training	72%	17%	60%	73%	86%	100%
6	Owner/Managing Partner/Leadership Support for Patient Safety	69%	31%	59%	69%	80%	100%
7	Communication About Error	67%	17%	59%	67%	76%	100%
8	Communication Openness	66%	27%	55%	66%	77%	100%
9	Patient Safety and Quality Issues	62%	22%	54%	63%	71%	95%
10	Office Processes and Standardization	61%	18%	48%	62%	75%	95%
11	Information Exchange With Other Settings	55%	8%	45%	55%	66%	100%
12	Work Pressure and Pace	49%	8%	34%	50%	63%	95%

Return to Contents

Produced by Westat under contract for the Agency for Healthcare Research and Quality (AHRQ)

Current as of March 2009