Structure of Dietary Measurement Error

Summary: An analysis of data from the Observing Protein and Energy Nutrition (OPEN) Study finds that measurement error in a food frequency questionnaire seriously reduces its ability to detect potential associations of absolute protein or energy intake with disease relative risk. The impact of this measurement error is somewhat less after adjusting for energy. Measurement error in the 24-hour recall also casts doubt on its use as a reference instrument for validating/calibrating of food frequency questionnaires for nutritional epidemiology studies.

Reference: Kipnis V, Subar AF, Midthune D, Freedman SL, Ballard-Barbash R, Troiano RP, Bingham S, Schoeller DA, Schatzkin A, Carroll RJ. Structure of dietary measurement error: results of the OPEN biomarker study. Am J Epidemiol 2003;158(1):14-21.

Food frequency questionnaires (FFQs) are commonly used in analytic epidemiologic studies that examine the relationship between diet and cancer. The fact that recent large cohort studies have failed to find a consistent relationship may be due to a true lack of diet-cancer associations or to methodologic limitations of the studies, especially measurement error in the FFQ. FFQ measurement error is important because it often leads to underestimates of disease relative risks, therefore reducing the statistical power of the study to detect their significance. This underestimation is called attenuation. To compensate for this problem, investigators have integrated calibration substudies that include a presumably more accurate reference instrument, such as food records or a 24-hour dietary recall (24HR) to estimate attenuation factors and adjust for them. To apply this approach correctly requires that any inherent error in the reference instrument be independent of true intake and of error in the FFQ.

Substantial evidence indicates, however, that commonly used dietary reference instruments are unlikely to meet these requirements. In this study, which analyzed data from the Observing Protein and Energy Nutrition (OPEN) Study, investigators evaluated absolute protein intake as well as total energy and energy-adjusted protein intakes among a sample of 484 people aged 40-69 living in the Washington DC suburb of Montgomery County, MD. The use of an FFQ, 24-HR, and reference biomarkers for protein (urinary nitrogen) and energy (doubly labeled water), allowed the authors to evaluate the structure of dietary measurement error in the FFQ and 24HR and to examine whether adjusting for energy substantially reduced measurement error in reported intake, permitting remaining error to be reliably corrected for by the commonly used approach.

Results of this analysis showed that the impact of FFQ measurement error for total energy and absolute protein intake was severe. The biomarker-based attenuation factors were close to 0 (in nutritional studies, the attenuation factor is generally between 1 and 0; values close to 0 lead to more serious underestimations of relative risk). Correlations between the FFQ and true intake also were very low. Even thought this impact seemed to be less severe after adjusting for energy, the results suggest that measurement error is still an important problem. Further, the investigators found that the 24HR was seriously flawed in that it had both intake-related bias and person-specific bias. This latter bias was correlated with person-specific bias in the FFQ. As a result, it did not meet either requirement for a valid reference instrument and misrepresented the impact of measurement error in the FFQ.

These results have several important implications for nutritional epidemiology. First, FFQs are unable to provide sufficiently accurate reports of absolute protein, nonprotein, and energy intakes, which reduces their ability to detect moderate associations with disease. Second, FFQ-based energy-adjusted nutrient intakes may be just accurate enough to use in large cohort studies to detect moderate diet-disease associations. However, because this study was restricted to energy, protein and nonprotein intakes, it is not clear whether these results would also apply to other dietary components, especially non-energy-contributing nutrients. Third, these results throw doubt on the use of 24-hour recalls as a reference instrument for validation or calibration studies of absolute protein and nonprotein intakes as well as energy-adjusted intakes.

Learn more about the Risk Factor Monitoring and Methods Branch's dietary assessment activities.

<< Previous Summary