Module 4: Measuring Quality of Care for Asthma

This module discusses the basic building blocks of quality improvement—measures and data. The module describes the asthma-related data available in the National Healthcare Quality Report (NHQR) and National Healthcare Disparities Report (NHDR) and from other sources that States can use. Each State has a cadre of health statisticians and analysts who should be recruited as part of any quality improvement project aimed at the health care system in the State because they will be familiar with local health data and because they know how to use and interpret data.

Key Ideas in Module 4: Quality improvement begins with measurement, which requires good measures and data for measuring quality of care. Process and outcome measures should be considered together to assess asthma care quality. The NHQR is a starting point for accessing consensus-based measures. Although a consensus on a small core of key asthma measures has not yet evolved, this Resource Guide identifies measures that are available for local quality improvement programs. Before undertaking any extensive data collection, State agencies should identify the questions to be answered and the data available to answer them. There are national and local data sources that can provide relevant data for creating estimates of State performance. State-level baseline estimates for asthma care afford State leaders a broad view of asthma care quality in their State. Analysis of data can answer some key questions for States: What measures should be used to set goals for quality asthma care? What goals should be set as targets for specific measures? What factors influence a State's position among other States?

Return to Contents

Quality Measurement

This section reviews the concept of quality measurement, available asthma-related measures in the NHQR and other sources, and the importance of using multi-dimensional measure sets. All of this is examined from the perspective of States and their role in initiating quality improvement programs.

The Concept of Quality Measurement

The Institute of Medicine defines health care quality as "the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge" (IOM, 2001). That definition suggests a distinction between quality measures and guidelines for quality care:

• Quality measures relate to populations. They include rates that indicate how many members of a population achieved a goal (for example, low emergency room visits for asthma nationwide) relative to a population base (for example, all people with asthma in the United States).
• Guidelines for quality care are recommendations devised via consensus processes of clinical experts that describe standards of care for individual patients. In general, guidelines for individual patient care prescribe what clinicians can do to improve the care that they deliver to their patients with a specific disease or condition. These guidelines also are used as the basis for developing population-based measures that enable analysts to assess and track change in the treatment of a population.

With a specific population in mind, a quality improvement program should consider the dimensions to be measured before embarking on data collection. What is to be measured? What change will be instituted? What quality measure will track the spread of that change? What is the ultimate outcome to be improved and how is that changed measured? What special populations are to be targeted and how will their improvement be documented?

Types of Quality Measures

Quality measures cover a large range, from crude measures (for example, unadjusted mortality rates) to more refined measures (for example, percent using asthma medications to achieve better asthma control). Although a full range of measures is essential for a complete picture of health care quality, specific process measures are needed to guide a health care team in improving quality of care. For example, the number of deaths related to asthma at a hospital can suggest poor quality of treatment at that hospital and in the community, but knowing the number of deaths does not tell the hospital staff or community providers how to improve. Metrics that measure processes of care that reduce deaths or improve other medical outcomes help medical staff know how to change care so that they provide better care.

Most quality improvement efforts focus on two types¹ of measures—process and outcome:

• Process measures often reflect evidence-based guidelines of care for specific conditions. Process measures are generally considered to be within the control of the provider and, therefore, are performance indicators. They also are more likely to reveal actions that can be taken to improve quality (for example, whether a necessary test or medication is given).
• Outcome measures frequently relate to patient health status. Better outcomes are the ultimate objective of quality improvement—for example, lower mortality, lower hospitalization rates, or better test results.

Ideally, improvements in processes yield improvements in associated outcomes, and measures should reflect that. However, the connections may not be that direct. For example, the asthma process measure for inhaled corticosteroid use is included in the NHQR because the evidence-based NHLBI clinical guidelines for asthma care recommend daily use of such medications for asthma patients with persistent asthma. Use of such asthma medications can help control and prevent asthma attacks and thus prevent the need for emergency care and hospitalizations.

The NHQR also monitors the outcome measure of hospitalizations for asthma. In this case, improvement in the process of prescribing inhaled corticosteroids and proper use by patients is expected to decrease the number of such hospitalizations, as diagrammed below. However, other factors (discussed more fully later in this module) are also important. Effective provider and patient education and self-management are crucial components. Without these, improved outcomes might never occur. (Select for Diagram.)

---

¹ A third type of measure is less directly related to quality of care. Structural measures reflect aspects of the health care infrastructure that generally are broad in scope, system wide, and difficult to link to short-term quality improvement (for example, a hospital's staff-to-bed ratio). The NHQR does not use structural measures.

---

Selection of Quality Measures for the NHQR

The selection of quality measures for the NHQR was based on criteria that include the clinical significance of the measure, reliability of available data, and consensus of the experts. The first NHQR, published in 2003, used a consensus process for determining which measures to include in the national tracking of health care quality. That process included issuing a public call for measures and assembled an interagency task force that reviewed and selected measures according to criteria developed by the Institute of Medicine and adopted by the Interagency Work Group for the NHQR.

Criteria for Selecting Asthma Measures: Importance Impact on health: What is the impact on the patient? Meaningfulness: Are providers and patients concerned about this area? Susceptibility to influence by the health care system: Can the health care system meaningfully address this aspect or problem? Scientific soundness Validity: Does the measure actually measure what it is intended to measure? Reliability: Does the measure provide stable results across various populations and circumstances? Explicitness of evidence: Is scientific evidence available to support the measure? Feasibility and usefulness Existence of prototypes: Is the measure in use? Availability of required data across the system: Can information needed for the measure be collected in the scale and time frame required? Cost or burden of measurement: How much will it cost to collect the data needed for the measure? Capacity of data and measure to support subgroup analyses: Can the measure be used to compare different groups of the population? Source: Adapted from Institute of Medicine, Envisioning the National Health Care Quality Report, 2001.

Other Sources of Asthma Measures

The NHQR currently includes only a few asthma measures, but others are available. Some of the major developers of asthma measures are:

• The National Asthma Survey, funded by the Centers for Disease Control and Prevention (CDC) and tested in 2003, is a 15-minute survey that States can use to provide a comprehensive assessment of asthma in the State. The NAS includes questions found in the Behavioral Risk Factor Surveillance System (BRFSS) asthma supplement as well as a more comprehensive set of questions on asthma care. (More information is available at http://www.cdc.gov/nchs/about/major/slaits/nsa.htm).
• The Joint Commission on Accreditation of Healthcare Organizations (JCAHO) Disease-Specific Care Certification Program provides an implementation guide for asthma performance measures for hospitals. A module for children's hospitalizations for asthma is in development. (More information is available at http://www.jcaho.org).
• The HRSA Bureau of Primary Health Care supports Health Disparities Collaboratives for disease-specific conditions, including asthma, for primary health care centers to participate in learning networks to improve quality of care. These learning collaboratives maintain a registry of asthma patients and monitor care for asthma patients on a monthly basis. (More information is available at http://www.healthdisparities.net).
• The National Initiative for Children's Healthcare Quality also develops learning collaboratives for asthma care for children based on the chronic care model for quality improvement.

Return to Contents

Multiple Dimensions of Quality for Asthma Care

One challenge of initiating quality improvement for asthma care from the perspective of a State or local quality improvement team is selecting from measures that assess the process and outcomes of improved care. There are many measures that could be used to assess different aspects of asthma care. Table 4.1 shows important dimensions of asthma quality of care and the measures that have been developed to assess these dimensions for improving care for asthma. The dimensions include provider processes of care, patient self-care processes, and outcomes of care such as quality-of-life factors. In addition, insurance coverage and prevalence and severity of asthma among the population are important factors that will influence the various measures of asthma care quality in any population.

Appendix D lists over 100 measures that are used throughout the country to measure asthma care quality and shows that different organizations evaluate different dimensions of asthma and define measures in different ways. Such variability in measurement makes it difficult, if not impossible, to compare across organizations, settings, and geography. CDC's National Asthma Survey addresses nearly all of the dimensions of quality asthma care, and the Behavioral Risk Factor Surveillance System surveys address the questions of influenza vaccination and smoking cessation counseling. Only a measure of whether the physician assessed the patient's asthma severity appropriately is missing. As noted earlier, the NAS was implemented in 2003 and tested in a few States. It has been adapted for use as a call-back survey in the BRFSS; the call-back data will be merged with the BRFSS core data so that all the measures in BRFSS will be available for analysis with the asthma-specific data. States can use Table 4.1 as a guide to understand how the measures can be used to assess asthma care quality.

Return to Contents

Data Sources for Asthma Quality of Care

Once States have identified the appropriate measures, the next step is locating sources of data for assessing the health system's performance in delivering quality care for asthma. This section describes three data sources for assessing asthma quality of care: the NHQR, the Behavioral Risk Factor Surveillance System (BRFSS), and local data sources.

Asthma-Related Data in the NHQR: Avoidable Hospitalizations

The NHQR asthma-related measures are primarily national or regional in geographic scope. At the State level, one asthma measure for outcomes of three age groups (under 18, 18-64, and 65 and over) appears in the NHQR—avoidable hospitalizations related to asthma. As shown in Module 1, Table 1.2 lists that measure by age group, available for 28 States that participated in the Healthcare Cost and Utilization Project (HCUP) in 2004 and whose hospital data are available online through HCUPnet (http://hcupnet.ahrq.gov/). HCUP is a Federal-State-Industry partnership, sponsored by AHRQ, which standardizes data across States. Table 1.2 shows:

• The State's hospitalization rate adjusted for age and sex differences among the States.
• The difference between the State's rate and the average of the " best-in-class" States—the 10 percent of States that have the lowest admission rates.

By examining the State rate and the difference from the best-in-class rate, a State can determine how far it has to go to reduce hospitalizations to become a top performer.

Hospitalization rates are affected by demographic characteristics of the population such as age, socioeconomic status and race/ethnicity. Although quality improvement efforts do not modify these characteristics, quality improvement initiatives can target subgroups that experience disparities to improve their asthma care quality and improve outcomes such as reducing hospitalizations for asthma.

The NHQR contains State-level rates only for this outcome measure of avoidable hospital admissions. The NHQR currently excludes State-level asthma process measures because no national consensus has, as yet, established the key asthma measures out of the many that have been developed and used by various organizations. As noted previously, over 100 measures for approximately 50 topics related to asthma care quality are listed in Appendix D. Also, results from the new National Asthma Survey, designed by the CDC to overcome limitations in the BRFSS asthma supplement, were not available in 2003 and 2004 for the first two releases of the NHQR and NHDR. Future releases are expected to include asthma measures from the NAS when available nationwide.

Six Asthma Measures in CDC's Behavioral Risk Factor Surveillance System

Currently, the richest source of asthma data nationwide by State is CDC's Behavioral Risk Factor Surveillance System. However, data from BRFSS should be interpreted with care. Due to sample size limitations, estimates may have large standard errors. The estimates reported here are from the 2004 data year, but several years of data could be pooled together for more reliable estimates. Despite limitations, BRFSS asthma data are a valuable starting point for viewing the national landscape of asthma quality care by State.

Table 4.2 summarizes estimates for six measures derived from BRFSS listed in Table 4.1. Each measure is displayed with the three estimates—the national average (reporting States weighted to a national average), the best-in-class average (the 10 percent of States with the best value), and the poorest performing average (the 10 percent of States with the poorest values).

Table 4.2 shows that the gap between the best-performing and poorest-performing States varies by type of measure.² Process measures are practices that clinicians can directly influence. Outcome measures are necessary for State programs and policymakers to assess the effects of changes in processes of care on the outcome of patients with asthma, and thus on the success of the quality improvement program. The spread for all measurements is about 15 to 20 percent.

---

^2. As shown in Table 4.2, the number of States (including DC and U.S. territories) reporting on each measure varies. For more detail on BRFSS estimates and individual State estimates of BRFSS measures, go to Appendix E.

---

Local Data Sources

Finding appropriate data can be a challenge for quality improvement programs. To stimulate interest and start the quality improvement process, States can develop an inventory of local data sources. (Go to Appendix F for a summary of asthma-related data sources.)

Local data (whether by State, county, municipality, or individual health care provider) are essential for quality improvement programs to have a local impact. Local leaders and health care professionals must see their own data compared with those from other providers and with State, regional, and national benchmarks in order to appreciate the importance of their work. By developing a complete inventory of data systems available at the State and local level, States can avoid duplicate data collection and reduce data-related costs. Also, a review of local data in the context of national data should clarify where existing local surveys or data systems could be improved.

Some possible local data sources to consider are listed below. These data sources may or may not be health care specific, but they may afford important opportunities for collaborations with various State or local agencies. It may be possible to add questions to ongoing local surveys to inform quality improvement activities for asthma.

Children and Youth:

• State school health surveys, administered before entering public schools to assess youth health risk behavior, may include questions about asthma prevalence, activity limitations due to asthma, etc.
• The Youth Tobacco Survey, administered by State health departments, may include questions on asthma prevalence to assess health risks and health behavior related to tobacco use.
• The Youth Risk Behavior Surveillance System, administered by CDC and State and local health and education agencies, monitors health-risk behaviors that contribute to unintentional injuries and violence, tobacco use, alcohol and other drug use, unintended pregnancy and sexually transmitted diseases, unhealthy dietary behaviors, and physical inactivity.

Adults:

• Occupational health surveys may provide data on work environment and triggers for asthma, activity limitation, and number of work days missed due to asthma.

Community/Environmental Assessment:

• Community surveys may provide local data on environmental factors that affect asthma and may compare asthma prevalence and outcomes by county, city, or neighborhood levels.

Most States also have ongoing surveys or health data systems that collect data at the State, county, and sometimes provider level. Some of those data systems include:

• State-level BRFSS data, available through the State health department.
• Statewide inpatient hospital discharge systems that collect data on individual discharges from hospitals and can provide county-level and, sometimes, hospital-level data. National benchmarks are available for these types of data through the NHQR.
• State vital statistics include mortality rates by cause of death and race/ethnicity. The National Vital Statistics System, which compiles these State data, can provide uniform national estimates and State estimates.
• Special disease registries focused on asthma may exist within the State, and these provide a rich source of patient-level information on severity and adherence to tracked treatments.
• Other special data collection of State departments of health statistics and other State programs may be modified to address asthma.

Specific data systems for populations that the State supports are also available in most States. These include:

• Medicaid information systems based on health care provider claims for Medicaid reimbursement.
• State employee health benefit claims for reimbursement.
• Patient records from State- or county-operated programs, such as mental health and substance abuse programs, public assistance, or justice systems.

Examples of State-level data sources are available at the National Association of Health Data Organizations Web site at: http://www.nahdo.org/soa/soalist1.asp?Category=State%20Agency.

Other Federal or national asthma surveillance systems compile data that describe State and local populations or health resources. These include:

• NHLBI Web site, a valuable starting place to identify data and become familiar with the network of organizations and individuals associated with asthma data collection on the State and national level (http://www.nhlbi.nih.gov).
• Census population data by State, maintained by the U.S. Bureau of the Census. These data are helpful for describing the demographic characteristics and wealth of local areas (http://factfinder.census.gov/jsp/saff/SAFFInfo.jsp?_pageId=gn10_select_state).
• The Area Resource File, a county-level database of health care resources from several surveys and data sources, compiled by the Health Resources and Services Administration. This resource might be helpful in analyzing the health resources available on a county level.
• Quality of care in managed care organizations, provided through the National Committee for Quality Assurance (go to: http://www.ncqa.org/index.asp). Local managed care organizations can be an important source of local data on health care quality.
• The Henry J. Kaiser Family Foundation Web site (http://kff.org/statepolicy/index.cfm), a rich source of health and other information at the State-level compiled from many public databases and published studies. This may help identify differences among State environments that would explain asthma prevalence or treatment differences across States.
• American Lung Association Web site, which contains patient education materials and tools as well as research on asthma (http://www.lungusa.org).

Return to Contents

Using Benchmarks To Develop State Performance Estimates

Once States have identified measures and acquired relevant data, analysts must develop estimates that gauge State performance.

Benchmarks

Benchmarks are external markers or values against which States can measure performance. The benchmark can represent the national average or best performers. How the State fares depends on where the State estimate falls compared with the benchmark. The NHQR provides a national set of estimates and State estimates that can be used as benchmarks for quality improvement.

Several types of metrics or benchmarks can be used for assessing a State. From more to less stringent, they include:

• The theoretic limit of 100 percent achievement (or 0 percent occurrence for avoidable events), which is an ideal, but often impractical or even impossible goal.
• A best-in-class estimate of the top State or top tier of States that shows what has been achieved (e.g., the top 10 percent of States is used in this Resource Guide).
• A national consensus-based goal, such as Healthy People 2010, set by a consensus of experts; such goals may be set more or less stringently than other benchmarks.
• A national average over all States, which shows the norm of practice nationwide but, being an average estimate, will represent a weaker goal than the best-in-class estimate.
• A regional average, which a State can use to compare itself to other States that are more likely to face similar environments; but, as a goal, it may be less aggressive than the best-in-class goal.
• An individual State rate, which itself can be used as a baseline against which to evaluate State-level interventions and progress over time within the State or to offer as a norm for local provider comparisons.

Some of these benchmarks can be found in the NHQR—the national and regional averages. The best-in-class estimate, not reported in the NHQR, can be derived from data in it. Go to Appendix G for details on the best-in-class estimate and other benchmarks that can be derived from the NHQR. Appendix H describes how to conduct statistical significance testing to determine whether or not comparisons of estimates show significant differences.

Asthma Benchmarks for States

A focused and limited set of measures for tracking quality nationally on an ongoing basis has not yet been specified for asthma. Thus, the NHQR has not yet settled on a complete set of consensus-based measures for asthma. As noted above, the National Asthma Survey is expected to inform that process in the future.

For this Resource Guide, benchmarks were calculated for asthma measures that were chosen based on availability of BRFSS data at the State level, current clinical guidelines, advice from an expert steering committee, and measures that will have a direct link to State budgets. They include:

• Process measures—services important for controlling asthma and preventing complications:
  • Routine checkups for patients with asthma (two or more planned doctor visits in the past 12 months).
  • Medications (use of medication to control asthma).
  • Advice to quit smoking (for asthma patients who smoke).
  • Flu shots (recommended for patients with asthma).
• Outcome measures—avoidable health care use:
  • Urgent care visits for asthma.
  • Emergency room visits for asthma.

Table 4.2 includes benchmarks—the national and best-in-class averages—for these measures. Figure 4.1 (53 KB) shows regional variations and the extent of the spread between States for each measure. The State analyses which follow illustrate four of these measures in more detail.

BRFSS has limitations for establishing benchmarks for State performance, including limited questions on asthma and other technical issues. These are discussed further in Appendix E.

Studying Individual States Against Benchmarks

This section compares four States, as examples, to the key benchmarks for the asthma measures. The four States were chosen because they show variation across the measures in how States ranked against the benchmarks in 2004. In Figures 4.2 (11 KB), 4.3 (11 KB), 4.4 (11 KB), and 4.5 (11 KB), States are compared to a national average and a best-in-class State average for each measure.

Though the theoretic limit may be difficult to achieve for many valid reasons, some States have already achieved the best-in-class estimate. Although the average over all States is often used to assess a State's performance, aiming for it means the State aims to be average rather than the best. Also, in some cases, a quality improvement team may set goals higher than the best-performing States because they may view all States as poor performers.

Maryland

Figure 4.2 reveals the following about asthma care in Maryland compared with national benchmarks in 2004. The marked "goal" on the vertical axis indicates the direction of improvement rather than an achievable value.

• In 2004, Maryland was close to the national average benchmark on two measures of asthma care quality—routine checkups and urgent care visits. Maryland has room for improvement on these dimensions to become a best-performing State.
• Maryland was below the national average for percentage of asthma patients who take medications for asthma. Given the importance of medication use to control asthma symptoms and prevent asthma attacks and their position on this measure, Maryland may want to investigate asthma drug therapy within the State, determine the locales or subpopulations for which such therapy is lacking, and develop a targeted program to improve the use of prescription drugs in the State for residents with asthma.
• Maryland appears to be statistically no different from the national average and the best-in-class average for emergency room visits. Small samples interfere with the ability of the data to distinguish between average and best-in-class in this case. Because of the weakness of this test and because the percent of people with emergency visits is higher in Maryland than nationally, Maryland might want to determine this rate more precisely. Maryland statewide hospital emergency department data system may want to address this issue.

Maryland can improve the treatment of asthma in the community and reduce the number of expensive emergency services in the State. Also Maryland has an opportunity to reduce its hospitalizations of patients with asthma for all age groups (go to Table 1.2).

Michigan

Figure 4.3 reveals that:

• In 2004, Michigan was at the national average benchmark for two measures—routine checkups and urgent care visits. Thus, Michigan has room for improvement on these dimensions to become a best-performing State.
• Michigan appears to be statistically no different from the national and best-in-class averages for the other two measures—use of asthma medications and emergency room visits. Small samples impede a robust test between average and best-in-class for these important measures, which points out the need for better assessment methods. Larger samples for BRFSS may be a relatively inexpensive solution for better statistics.
• Michigan was not among the below-average States for any of these four asthma measures in 2004. This suggests that Michigan's efforts toward disease prevention and control may have contributed to this positive result. Michigan also is helped by its average sociodemographic characteristics, especially an average poverty rate.

Thus, Michigan may want to improve its strategy for measuring asthma care quality and could justify focusing attention on improving the frequency of checkups for people with asthma in order to become a best-performing State. If that strategy is done well, it could reduce the cost of expensive urgent/emergency care. Also, by improving outpatient care Michigan has the opportunity to in turn reduce costly hospitalizations related to asthma (go to Table 1.2).

New Jersey

Figure 4.4 shows that:

• In 2003, New Jersey was among the best-in-class States for routine checkups for people with asthma. Although this is excellent performance among all States, the best performers only reach the 38-percent mark for the percent of people with asthma who have planned care visits two or more times per year. Thus, New Jersey may want to aim for higher checkup rates for its population with asthma.
• New Jersey's estimate for asthma medication use is statistically no different from the national average or best-in-class average. Again, the small samples blur the ability to make the distinction, but the value of the estimate is closer to the all-States average than the best-in-class States estimate. Another factor with this measure is that the spread of the values across the States is very narrow, suggesting that medication use in asthma care is relatively uniform across the States.
• New Jersey was worse than or at the national average on the use of urgent and emergency care, respectively.

Despite New Jersey's excellent performance on checkups and reasonable performance on medication therapy, its poorer performance on use of expensive urgent care services raises a question. How effective are community-based checkups for people with asthma if they use a high level of urgent care services? New Jersey may want to explore the nature of checkups for people with asthma and determine whether health care providers are using the best asthma management practices (Module 1) with their patients.

Vermont

Figure 4.5 (11 KB) shows that:

• In 2004, Vermont was among the poorest-performing States for routine checkups for people with asthma.
• Vermont appears to be statistically no different than the national average and best-in-class average for use of asthma medications. Again this distinction cannot be made definitely due to small sample size.
• Vermont was among the best-in-class States for the two outcome measures related to expensive emergency medical care. Vermont has low rates of urgent care visits and emergency department visits for asthma.

This result—reasonable outcomes for emergency services use, but poor processes for checkups and medication, which appear to be inconsistent with each other—suggests that these measures by themselves are not the strongest determinants of patient outcomes and that other underlying factors are at work. The next section discusses some of the external factors that can affect the quality of care in communities.

Return to Contents
Proceed to Next Section