Appendix E: BRFSS Measures, Data and Benchmarks

In 2003, asthma data were collected under the Behavioral Risk Factor Surveillance System (BRFSS) for 5 process measures, 7 outcome measures, and 3 prevalence measures. Those measures for adults with asthma include asthma history, routine check ups, doctor visits for asthma, limited activity due to asthma, medications for asthma, asthma symptoms, asthma episodes, emergency department visits, urgent care visits, and sleep difficulty due to asthma. The number of entities reporting varied from 15 to 54 depending on the measure. All 50 States, DC, and 3 U.S. Territories collected data on receipt of influenza vaccination in the past year. In our analysis, adult smokers with asthma were studied to determine the prevalence of smoking and asthma.

The BRFSS data are based on telephone surveys developed by the Centers for Disease Control and Prevention (CDC) but administered by each State independently. The survey consists of a core set of questions developed by CDC, additional questions developed by the States, and separate, optional modules for States to use. The asthma module, which contains the quality-of-care questions, is optional for State use. More information about the BRFSS data and methods as well as interactive databases with some State and local level asthma data are available at: http://www.cdc.gov/brfss.

Limitations of BRFSS Data

Every data source has limitations that can relate to the population represented, methods used to collect the data, definitions, and analytic approaches. These factors affect the estimates generated from a data set. When similar measures from two data sets differ, the cause can usually be traced to the limitations of the data sets. By understanding the limitation of a data set, the strengths and weakness of estimates from the data set can be assessed and the estimates can be used more responsibly.

Limitations of BRFSS data include the following:

• BRFSS samples are kept small to minimize survey costs for States. The State BRFSS samples for the year 2001 range from 1,888 to 8,628 respondents (see: http://www.cdc.gov/brfss/technical_infodata/surveydata/2003.htm). Small samples increase the variance of estimates and decrease the size of the difference between two subpopulations that can be detected through the survey responses. In fact, among the asthma measures, the small sample sizes impeded statistical tests of differences, as discussed below.
• The BRFSS survey excludes people without a residential phone and people who are institutionalized. This means that the total population of interest—all people with asthma—will not be represented in the estimates that come from the survey.^a This weakness can be dealt with by carefully discussing BRFSS results in relation to the population it represents.
• BRFSS data are self-reported and reflect the perceptions of respondents. An advantage of self-reports is that they can reveal information that cannot be obtained from other sources; for example, the receipt of flu vaccinations for people who do not see a doctor during the year. A disadvantage of self-report data is that respondents may have difficulty recalling events, understanding or interpreting questions, or responding truthfully to questions such as about compliance with advice. Furthermore, cultural and language barriers and limited health knowledge can affect the quality of self-reported data.^a These problems may occur with different propensity for different subgroups.

BRFSS data, like most surveys, are limited by budget constraints. Because BRFSS is funded by States which vary considerably in resources allocated to health surveys, these fiscal disparities may affect the quality of the data across States. Such data quality shortcomings can include bias from differential response rates, varying followup periods, and variations in interviewer protocols or skills (for example, extent of probing for answers).

Small Sample Size in BRFSS

Table E.1 shows that small sample sizes in the BRFSS supplemental asthma survey result in tests that are unreliable. For example for smoking cessation counseling, 15 of 15 reporting States could not be distinguished from the average of the top 2 States (or top10 percent of States). This is partly because smoking cessation counseling is commonly provided across all States (the distribution of percent counseled is narrow), in combination with the small numbers of individuals interviewed in BRFSS. The smaller the difference to be detected, the greater the sample needed. The same issues are apparent for the measure "average number of symptom-free days in the past 2 weeks." Fourteen of 19 estimates are indistinguishable from the top decile, again a problem of small sample size.

By contrast, "flu shots in the past 12 months" is a measure collected from the core BRFSS survey and thus more reliable estimates result. Eleven of 54 entities represent States comparable to the best-in-class average of 5 States in the top 10 percent.

The issue of sample size is the main reason that the National Healthcare Quality Report (NHQR), which produces annual estimates, did not include State-level BRFSS data. For State estimates, multiple years of BRFSS should be used.

Estimates for individual BRFSS measures by State (including the District of Columbia and U.S. Territories) are presented in Tables E.2-E.16.

Tables, Appendix E:

Table E.1: Selected quality measures for asthma by State, District of Columbia, and U.S. Territory, 2003.

Table E.2: Lifetime asthma prevalence: Percent of people who were ever told by a health professional that they have asthma by State, District of Columbia, and U. S. Territory, 2003.

Table E.3: Current asthma prevalence: Percent of people who were ever told they have asthma who still have asthma by State, District of Columbia, and U.S. Territory, 2003.

Table E.4: Age at asthma diagnosis: Percent of adults with asthma who were diagnosed before age 10 by State, District of Columbia, and U.S. Territory, 2003.

Table E.5: Urgent care visits: Percent of adults currently with asthma who had at least one urgent care visit for asthma with their provider in the past 12 months by State, District of Columbia and U.S. Territory, 2003.

Table E.6: Emergency room visits: Percent of adults with asthma who have had at least one visit to the emergency room for asthma in the past 12 months by states, District of Columbia and U.S. Territory, 2003.

Table E.7: Asthma attacks/episodes: Percent of adults with asthma who had an asthma episode in the past 12 months by State, District of Columbia and U.S. Territory, 2003.

Table E.8: Limited activity due to asthma: Average number of days adults with asthma were unable to work or carry out usual activities in the past 12 months by State, District of Columbia and U.S. Territory, 2003.

Table E.9: No sleep difficulty due to asthma: Percent of adults with asthma who had no difficulty sleeping due to asthma during the past month by State, District of Columbia and U.S. Territory, 2003.

Table E.10: Routine care for asthma: Percent of adults with asthma who had 2 or more planned care visits for asthma during the past 12 months by State, District of Columbia and U.S. Territory, 2003.

Table E.11: Doctors visits for asthma: Percent of adults with asthma who had a physician visit for asthma in the past 12 months by State, District of Columbia and U.S. Territory, 2003.

Table E.12: Medications for asthma: Percent of adults with asthma who took asthma medication in the past month by State, District of Columbia and U.S. Territory, 2003.

Table E.13: Asthma symptom-free days: Average number of days adults with asthma were free of asthma symptoms in past 2 weeks by State, District of Columbia and U.S. Territory, 2003.

Table E.14: Asthma symptoms: Percent of adults with asthma who experienced asthma symptoms every day in past 2 weeks by State, District of Columbia, and U.S. Territory, 2003.

Table E.15: Smoking cessation counseling: Percent of adults with asthma who were advised to quit smoking by a health professional by State, District of Columbia, and U.S. Territory, 2003.

Table E.16: Percent of all adults who received flu shots and percent of adults with asthma who received flu shots by State, District of Columbia, and U.S. Territory, 2003.

---

^a Nelson D, Holtzman D, Bolen J, Stanwyck C, Mack K. Reliability and validity of measures from the behavioral risk factor surveillance system (BRFSS). Sozial un Praventivmedizin 2001;46(Supp 1):S3-42.

---

Return to Contents
Proceed to Next Section