| Federal
Committee on Statistical
Methodology Office of Management and Budget |
FCSM
Home ^ Methodology Reports ^ |
Statistical Policy Working Paper 13 - Federal Longitudinal Surveys
Click HERE for graphic.MEMBERS OF THE FEDERAL COMMITTEE ON STATISTICAL METHODOLOGY Maria Elena Gonzalez (Chair) Daniel Kasprzyk Office of Information and Bureau of the Census Regulatory Affairs (OMB) (Commerce) Barbara A. Bailar William E. Kibler Bureau of the Census Statistical Reporting Service (Commerce) (Agriculture) Yvonne M. Bishop David Pierce Energy Information Federal Reserve Board Administration (Energy) Edwin J. Coleman Thomas Plewes Bureau of Economic Analysis Bureau of Labor Statistics (Commerce) (Labor) John E. Cremeans Jane Ross Business Analysis Social Security Administration (Commerce) (Health and Human Services) Zahava D. Doering Fritz Scheuren Defense Manpower Data Center Internal Revenue Service (Defense) (Treasury) Daniel H. Carnick Monroe G. Sirken Bureau of Economic Analysis National Center for Health (Commerce) Statistics (Health and Human Services) Terry Ireland Thomas G. Staple National Security Agency Social Security Administration (Defense) (Health and Human Services) Charles D. Jones Robert D. Tortora Bureau of the Census Statistical Reporting Service (Commerce) (Agriculture) PREFACE The Federal Committee on Statistical Methodology was organized by OMB in 1975 to investigate methodological issues in Federal statistics. Members of the committee, selected by OMB on the basis of their individual expertise and interest in statistical methods, serve in their personal capacity rather than as agency representative. The committee carries out its work through subcommittees that are organized to study particular issues and that are open to any federal employees who wish to participate in the studies. Working papers are prepared by the subcommittee members and reflect only their individual and collective views. This working paper of the Subcommittee on Federal Longitudinal Surveys discusses the goals, management, operations, sample designs, estimation methods, and analysis of longitudinal surveys. Conclusions are drawn about where to use longitudinal surveys, and the need to have an evaluation component in these surveys. The Appendices contain twelve case studies of recent longitudinal surveys. The report is intended primarily to be useful to Federal agencies in choosing to do, and then in designing, carrying out, and analyzing data from longitudinal surveys. The Federal Committee on Statistical Methodology intends to organize seminars to discuss the report with interested Federal agency staff members. The Subcommittee on Federal Longitudinal Surveys was co-chaired by Barbara A. Bailar and Daniel Kasprzyk, Bureau of Census, Department of Commerce. MEMBERS OF THE SUBCOMMITTEE ON FEDERAL LONGITUDINAL SURVEYS Barbara A. Bailar* (Co-chair) Lawrence Ernst Bureau of the Census (Commerce) Bureau of the Census (Commerce) Daniel Kasprzyk* (Co-chair) Marie E. Gonzalez* (ex officio) Bureau of the Census (Commerce) Office of the Information and Regulatory Affairs (OMB) Barry Bye Catherine Hines Social Security Administration Bureau of the Census (Commerce) (Health and Human Services) Dennis Carroll Curtis Jacobs Center for Statistics Bureau of Labor Statistics (Education) (Labor) Robert Casady Inderjit Kundra National Center for Health Energy Information Statistics Administration (Health and Human Services) (Energy) Steven B. Cohen Bruce Taylor National Center for Health Bureau of Justice Statistics Services Research (Health (Justice) and Human Services) ADDITIONAL CONTRIBUTOR TO THE REPORT Lawrence Corder Research Triangle Institute (Previously National Center for Health Statistics) *Member, Federal Committee on Statistical Methodology ACKNOWLEDGEMENTS This report is the result of collective work and many meetings of the Subcommittee on Federal Longitudinal Surveys. Each chapter had a principal author (or authors), as noted below, but the final report, particularly the introduction and summary sections, reflects contributions from all of the Subcommittee Many useful suggestions on content and organization were made by Maria Gonzales, chairperson of the Federal Committee on Methodology (FCSM). Barbara Bailar, Co-Chair of the Subcommittee, prepared the Introduction and the concluding Chapter, which embody the discussions held by the whole Subcommittee. All of the FCSM members reviewed several drafts and made many important suggestions. The Subcommittee in particular wishes to recognize the valuable contributions made by the primary reviewers: Zahava Doering, Fritz Scheuren and especially Monroe Sirken, who read and commented on two drafts of the complete report. The principal authors of each chapter of the report are: Chapter One Catherine Hines Chapter Two Lawrence Corder Chapter Three Bruce Taylor Chapter Four Daniel Kasprzyk and Lawrence Ernst Chapter Five Barry V. Bye The Subcommittee thanks also the following persons who were responsible for preparing the Case Studies that appear in the Appendix: Edith McArthur (SIPP), Curtis Jacobs (CPI), Steve Kaufman (ECI), Dennis Carroll (NLS-72, HS&B;), Catherine Hines (NLS), Barry V. Bye (RHS, WIE), Stephen B. Cohen (NMCES), Robert Casady (NMCUES), James L. Monahan (LED), John DiPaolo, Robert Wilson, and Peter J. Sailer (SOI). Catherine Hines edited the report. Joanne Watson (Bureau of the Census) prepared each of the drafts, and the Subcommittee thanks her for her patience and accuracy. iii GLOSSARY OF ABBREVIATIONS AHS American Housing Survey (Formerly Annual Housing Survey) CPI Consumer Price Index CPS Current Population Survey ECI Employment Cost Index HCFA Health Care Financing Administration HS&B; Longitudinal Survey of High School and Beyond ISDP Income Survey Development Program ISR Institute for Social Research (University of Michigan) NCES National Center for Education Statistics NCHS National Center for Health Statistics NCS National Crime Survey NLS National Longitudinal Surveys of Labor Market Experience NLS-72 National Longitudinal Study of the High School Class of 1972 NMCES National Medical Care Expenditure Survey NMCUES National Medical Care Utilization and Expenditure Survey OSIRIS Statistical Analysis software, Survey Research Center, U. Michigan PSID Panel Survey on Income Dynamics RAMIS Data base management system, Mathematical Research Inc., Princeton, N.J. RAPID Data base management system, Statistics Canada, Ottawa RHS Retirement History Study SAS Data base management system, SAS Institute, Cary, N.C. SSA Social Security Administration SIPP Survey of Income and Program Participation SIR Data base management system, SIR, Inc., Evanston, IL SOL Statistics of Income Program, IRS WIE Work Incentive Experiment, SSA iv TABLE OF CONTENTS Page GLOSSARY OF ABBREVIATIONS vi INTRODUCTION 1 Chapter I: The Goals of Longitudinal Research 5 Chapter II: Managing Longitudinal Surveys 11 Chapter III: Longitudinal Survey, Operations 19 Chapter IV: Sample Design and Estimation 35 Chapter V: Longitudinal Data Analysis 49 Chapter VI: Summary and Conclusions 63 APPENDIX: Case Study 1 Survey of Income and Program Participation 67 Case Study 2 Consumer Price Index 75 Case Study 3 Employment Cost Index 89 Case Study 4 National Longitudinal Study of the High School 97 Class of 1972 Case Study 5 High School and Beyond 101 Case Study 6 National Longitudinal Surveys of Labor Market 105 Experience Case Study 7 Social Security Administration's Retirement 111 History Study Case Study 8 Social Security Administration's Disability 115 Program Work Incentive Experiments Case Study 9 National Medical Care Expenditures Survey 123 Case Study 10 National Medical Care Utilization and Expendi 127 tures Survey Case Study 11 Longitudinal Establishment Data File 137 Case Study 12 Statistics of Income Data Program 147 REFERENCES 153 INTRODUCTION Since the 1960's, the Federal government has sponsored an increasing number of longitudinal surveys as vehicles for research on administrative and policy issues. The goal of the Federal Committee on Statistical Methodology's subcommittee on Federal Longitudinal Surveys is to identify the strengths and limitations of longitudinal surveys, and to propose some guidelines for using them most effectively. Beginning its work, the subcommittee found that there were multiple definitions of a longitudinal survey, so our first task was to define what this report would mean by the term. The difficulty arises because there are two facets to the definition, design and analysis. To be absolutely clear, one must distinguish between a longitudinally designed survey and a survey with longitudinal analysis. We have elected to put these components together in our definition. The distinguishing features of a longitudinal survey are: - repeated data collection for a sample of observational units over time; - the linkage of data records for different time periods to create a longitudinal record for each observational unit; and - the analysis is based on the longitudinal microdata and refers to data collected over time. The essential feature is that, from the beginning, there is a plan to elicit data from the future for each observational unit. This definition excludes some surveys with longitudinal elements, such as the Current Population Survey (CPS). The Survey of Income and Program Participation (SIPP) is included here as a longitudinal survey, although there are as yet no longitudinal analyses of SIPP. Federal agencies also conduct surveys of establishments that have longitudinal elements but these are not yet true longitudinal surveys either. There is an effort to create a longitudinal file for manufacturing firms at the Bureau of the Census. We included this program as a case study in this report because, although it does not meet our definition, it may be of interest to readers. Similarly, Federal agencies maintain longitudinal files of administrative records that do not meet our definition. Yet they may be used in ways that are similar to the analysis of longitudinal surveys, so we have included an example, the Statistics of Income Data Program, as a case study. 1 Rotating panel surveys* are often described as longitudinal surveys. They are not, but they may share many sampling, estimation, and analysis characteristics with longitudinal surveys. In addition, there is a tendency for ongoing rotating panel surveys to be changed to make longitudinal analysis possible. The National Crime Survey (NCS) is currently considering such a transition, and one possible result of the current redesign activities will be to create a longitudinal NCS data file if the cost is not prohibitive. There is interest in moving in the same direction with both CPS and the American Housing Survey (AHS, formerly the Annual Housing Survey). We should anticipate that eventually more rotating panel surveys will be modified, or designed from the beginning, to make longitudinal analysis possible. At this time, however, many rotating panels lack longitudinal data files, and many longitudinal surveys are designed without rotating panels. The subcommittee members examined in detail 12 recent longitudinal surveys sponsored by the Federal Government, as examples and illustrations. These are: (1) the Survey of Income and Program Participation (SIPP); (2) the Consumer Price Index (CPI); (3) the Employment Cost Index Survey (ECI); (4) the National Longitudinal Study of the High School Class of 1972 (NLS-72); (5) High School and Beyond (HS-B); (6) The National Longitudinal Surveys of Labor Market Experience (NLS); (7) the Social Security Administration's Retirement History Survey (RHS); (8) The Social Security Administration's Disability Program Work Incentive Experiments (WIE); (9) The National Medical Care Expenditure Survey (NMCES); (10) the National Medical Care Utilization and Expenditure Survey (NMCUES); (11) the Longitudinal Establishment Data File; and (12) the Statistics of Income Data Program (SOI). The surveys chosen for case study treatment were selected to represent a variety of sponsors, research questions and kinds of respondents. Each of the 12 case studies is described in the Appendix,,and they are frequently cited to illustrate important points throughout the text. We hope that the chapters of the text and the case studies in the Appendix will convince readers of four points that emerged from the subcommittee's review of longitudinal surveys. First, longitudinal survey designs are appropriate, and even required, for certain kinds of research. These include, but are not limited to, such topics as gross change, the causes of change, or the role of attitudes in change. However, many longitudinal surveys have not made full use of their longitudinal design in the analysis. Second, longitudinal survey design, operation, and analysis techniques are still evolving. There are a number of important design issues that are not yet explored or understood. An example is the optimal length of time between interviews, and the number of interviews to conduct to achieve research objectives. To some extent the variations in survey design ___________________________ * A panel is a sample of persons selected to participate at a particular point in the longitudinal sequence. In a rotating panel survey the sample units have a fixed duration. As they leave the sample, they are replaced by new units which are introduced at specific points in time. 2 reflect the wide and legitimate differences between the research goals that each survey was designed to accomplish. This does not explain, however, all the existing variation in methods . Decisions about sample design and attrition, about selecting the best respondent or analytical units, about the best estimation, imputation or weighting schemes, or about the impact of varying personal, mail or telephone interviews over the course of a longitudinal survey, have not always been consistent. Third, the important question of the costs of longitudinal surveys compared to cross-sectional surveys has yet to be answered. There are conflicting reports about the relative costs of the two types of survey. Costs are usually cited as higher for longitudinal surveys, but the costs being reported are confined to data collection costs and processing costs. This does not compare the full range of survey costs including quality costs, costs of analysis, and other such elements which could, in the long run, change the picture of the relative costs. The fourth and final point that emerged from the subcommittee's review was that the surest method for learning answers to design, operational, and analysis issues is to build an evaluation component into a longitudinal survey. By this means a record of comparative performance is created which benefits others. The case studies presented in this report, in particular, show how progress occurs when evaluation is built into survey operations, and how forethought and planning, far more than additional expense, are needed to increase our knowledge about longitudinal survey design. This report is presented in 6 chapters. The first chapter is a review of the kind of research question for which a longitudinal approach is appropriate, illustrated with examples. The second and third chapters describe some of the problems encountered in planning and managing longitudinal surveys. Chapter four discusses problems related to sample design and analytical units in longitudinal surveys, and special problems of estimation and weighting. Chapter five describes and evaluates major approaches to the analysis of longitudinal surveys. The final chapter, number six, summarizes some issues the subcommittee members recognized as important, and outlines the need for building an evaluation component into prospective longitudinal surveys; both to answer questions about the quality of data derived from each survey and to answer questions about optimal design for future longitudinal surveys. 3 CHAPTER 1 THE GOALS OF LONGITUDINAL RESEARCH There are at least five distinctive advantages to using a longitudinal survey rather than a cross-sectional survey some of these advantages are shared by rotating panel surveys. 1. A longitudinal sample reduces sampling variability in estimates of change. This is an advantage shared with rotating panel surveys such as CPS and NCS. 2. A matched longitudinal file provides a measure of individual gross change for each sample unit. This is an advantage shared to some extent by rotating panels, which can provide a measure of gross change, but not usually on an individual basis. 3. Longitudinal survey interviews usually have a shorter, bounded reference period that reduces recall bias in comparison to a retrospective interview with a long reference period. Rotating panels such as CPS and NCS also share this advantage. Longitudinal surveys with long intervals between interviews may lose this advantage. 4. Longitudinal data are collected in a time sequence that clarifies the direction as well as the magnitude of change among variables. 5. Longitudinal interviews reduce the respondent burden involved in creating a record that contains many variables. A single interview could not collect comparable detail without excessive respondent burden and fatigue. In addition, the quantity of data collected in a longitudinal survey is usually greater than that from several cross-sectional surveys because of the correlational structure of longitudinal data. There are also some distinct disadvantages to longitudinal surveys. Some of these are:. 1. The analysis of longitudinal surveys is dependent on the assembly of the microrecord data. The full advantage of compiling a detailed longitudinal record with many variables may not be available until years after the start of data collection. 2. Beginning refusal rates may be comparable to those of cross-sectional surveys, but the attrition suffered over time may create serious biases in the analysis. Principal Author: Catherine Hines 5 3. A longitudinal survey, including several data collections, is more costly than a single retrospective cross-sectional survey. A longitudinal survey may be less costly than a series of cross-sectional surveys. It is speculative whether a longitudinal survey is more costly than a rotating panel survey. 4. The estimates of gross change derived from longitudinal surveys tend to be inflated over time by simple response variance, The combined or net effect of such influences as simple response variance, response bias and time-in- sample bias effect on longitudinal estimates of gross change are still poorly measured. 5. Longitudinal surveys are often improperly analyzed, not taking into account longitudinal characteristics or attrition. For some research goals, the advantages clearly outweigh the disadvantages. For other research goals this may not be the case. Research goals that demand longitudinal surveys are described in this chapter. A. Measuring Change Both cross-sectional and longitudinal surveys can be used to measure change. The National monthly estimate of unemployment based on the CPS is always compared to the estimate for the previous month or the same month a year ago. Estimates of such things as crime victimizations, retail sales, housing starts, or health conditions are all compared to estimates from a previous time period. None of these data are currently based on longitudinal surveys. Which measures of change need a longitudinal file structure? One example is the components of individual change. These are measures of gross change for the observational units between points in time.* Longitudinal data are frequently displayed in a time- referenced table, showing the characteristics, attitudes, or beliefs of the sample at time 1; cross-tabulated by the same characteristics, attitudes, or beliefs at time 2. Another example is the average change for an observational unit. As pointed out by Duncan and Kalton (1985), if data are available for several time points for each observational unit, then a measure of average change or trend can be estimated. Finally, a longitudinal design permits the measurement of stability or lack of stability for each observational unit. Measures of gross change are of interest in several of the case studies described in this report. Respondents are followed through employment and unemployment (NLS), training and the labor force (NLS-72, HS&B;), into and out, of poverty (SIPP), or between health, treatment, and disability (NMCES, NMCUES, RHS, WIE). The focus is sometimes on movement across an arbitrary threshold (such as poverty, defined by household composition and income), and sometimes on a continuous measure. The observation periods in a longitudinal survey are commonly called waves. A wave describes one complete cycle of interviewing, from sampling to data collection, regardless of its duration. 6 In independent (i.e., cross-sectional) samples, sub- populations with very different gross-change patterns are indistinguishable if the sum of the changes is similar. This has been important to studies of employment. The NLS, for example, can distinguish a hypothetical population where 15% of the people are never employed, from a population where at each interview a different 15 % respondents report unemployment. A cross-sectional survey could not make the same distinction, which is vital to the development of intervention policies. Another example can be cited from the field of social indicators research. A series of variables, measured longitudinally, can be used to construct models for estimation to examine change over time with great elegance. (See Land, 1971, 1975.) Young adults in the years after full-time school are frequent longitudinal survey subjects (NLS Youth Cohorts, NLS-72, HS&B;) because individuals in these years are known to pass between statuses (employment and unemployment, school and training programs, in and out-of the armed services, between households) rapidly and irregularly. Cross-sectional studies would miss all the individual reversals and repetitive change. To develop detailed models of the causes of change in these fluid populations, longitudinal measures are needed to capture the record of individual and gross change. For example, cross-sectional studies of college enrollments have generally found relatively high stability over a number of years, whereas analysis of NLS-72 data identified frequent individual change occurring at a stable rate. A substantial percentage of the college students surveyed exhibited erratic enrollment patterns characterized by dropping out or transferring between 4-year and 2-year colleges. In light of these findings, student financial assistance (grants and loans) have changed. Legislation has shifted aid to channel the funds directly to the students, who choose the college they wish to attend -- rather than channelling the funds to college officials, who decide how the funds are doled out to enrolled students. Studying the relationship between attitudes and behavioral change poses particularly difficult problems in research design. The problems inherent in determining which variable in a pair changes first are present, and they are exacerbated by the problems encountered in surveys of subjective phenomena, such as attitudes. Using retrospective questions to ask respondents to reconstruct thoughts or feelings as they existed in the past has proved unreliable. Prospective longitudinal surveys provide the most reliable data on change in knowledge or attitudes, because longitudinal measures are collected while the subjective states actually exist. This appears to reduce the bias frequently caused by suppression or distortion of respondent recall. In addition, unlike retrospective measures of attitudes, contemporary measures can sometimes be probed or even verified. The longitudinal surveys of high school students (NLS-72 and HS&B;) demonstrate the method's power to collect data on changing subjective states, and to study causation. These surveys have measured attitudes and expectations about employment, and subsequent employment experiences and behavior. The data, which could not have been collected cross-sectionally, can be analyzed to understand the formation of attitudes, as well as to evaluate the effects that attitudes have on subsequent behavior. 7 When the research goal is to measure a component of individual change, longitudinal surveys have strong advantages. They are the only method available to collect data on a recent occurrence basis over a long period of time. Although a retrospective cross- sectional survey could be used to attempt the same thing, the recall bias may be a strong force against this decision. The bias from the attrition in a longitudinal survey has to be balanced against the bias or lack of information in a retrospective cross- sectional survey. The bias from attrition is usually preferred. Price and wage changes are measured in longitudinal surveys (i.e., the CPI and ECI) because the longitudinal sample design holds other variables constant. The assumption can be made that whatever unknown sampling bias exists in later waves was also present in earlier waves, and can be dismissed as a possible source of the changes being measured. B. Assembling Detailed Individual Records Longitudinal surveys generally provide researchers with more detailed records for each individual than is practicable through a cross-sectional design. In a longitudinal design, an extremely detailed record can be accumulated for each subject without making any single observation period (i.e., interview or wave) excessively burdensome. By 1982, for example, records for the original respondents in the NLS contained up to 1,000 data items for each sample case. To create a record of comparable detail complexity would have required a one-time questionnaire of extraordinary length. In addition, responses referring to earlier time periods would have been reconstructed from memory, reducing their reliability. In many instances, researchers are looking for cause- and-effect relationships that are more likely to be accurate if the data are compiled on a current rather than retrospective basis. C. Collecting Data That is Hard to Recall Some surveys ask questions that respondents have difficulty in answering precisely or objectively after much time has passed. These include questions that call for the kind of detail that people seldom recall clearly (such as complete records of expenditures, or health treatments), and questions that refer to events that respondents tend to telescope, embellish or suppress in their memories after time has passed (such as crime victimization, health problems, or visits to the doctor). Questions such as these have been used successfully in longitudinal surveys, in which the previous interview provides a clear marker to bound respondent recall, and which are constructed with short reference periods between interviews. For example, the Consumer Expenditure Survey, conducted as part of the CPI program, collects detailed records of household spending patterns through longitudinal interviews. (See Case Study no. 2 in the appendix.) A longitudinal survey with relatively short reference periods is one of the best methods for producing aggregated data for a longer time period, such as a year. For example, the primary goal of the NMCES and,NMCUES programs 8 was to develop estimates of medical expenditures for a calendar year. This was accomplished by obtaining medical expenditure data every 3 months and Compiling an annual total. A similar example is the new continuing Consumer Expenditure Survey, which covers all consumer expenditures. The SIPP program employs a similar design, using interviews at 4 month intervals to produce annual aggregates. The relatively short, bounded reference periods for these longitudinal surveys improve reporting by eliciting events closer to the time they occur. This increases the completeness of aggregated estimates and reduces error. D. Modelling Studies and Pilot Programs The detailed case histories built up in longitudinal surveys are important in analyzing the impact of alternative policies or intervention strategies. The complex individual case records accumulated in a longitudinal panel survey provide a microcosm in which the impact of changes can be simulated. Questions can be answered about the probable impact of changing a program's eligibility criteria, for example, or about the benefits which specified classes of respondents might anticipate under,various program changes. Intervention programs can be evaluated through longitudinal surveys to Study their effect on respondents with known characteristics. A sufficiently detailed record makes it possible to simulate alternative interventions, and predict a range of effects. (See Case Study 9 on the WIE, for example.) In some cases longitudinal surveys, pilot intervention programs and Federal policy experiments evolved together in the 1960's. Several longitudinal surveys authorized as components of pilot or experimental intervention programs to measure program effects and ensure that decision-making information would be available when it was needed. Longitudinal data collection components were built into pilot income maintenance programs, for example, administered temporarily in cities in New Jersey, Indiana, Colorado and Washington State. In conclusion, tho points about the periodicity of longitudinal research should be stressed. First, longitudinal data are never available immediately; any data that are based on the sequence of measures over time cannot be fully extracted until the final measures are collected. If information is needed at once, another research design has to be used which incorporates some alternative to a true longitudinal approach; such as retrospective measures, or the use of administrative records. Even if the quality of data from a longitudinal survey would be clearly superior, that would be irrelevant if the schedule outweighs these other considerations. Second, longitudinal data can be used cross-sectionally to provide immediate data as long as the research focus is not specifically on changing measures over time. Each wave of a longitudinal survey can also be analyzed as a cross-sectional survey. Thus some data can always be made available immediately. Record data from non-going longitudinal surveys can be analyzed quickly from a cross-sectional perspective to serve certain analytical purposes without delay. It is also possible to add questions to the current waves of a longitudinal survey to meet immediate data needs, using an existing longitudinal sample and base-line demographic data for maximum efficiency. In these ways a longitudinal design adds analytical strengths without sacrificing the potential for cross-sectional research. 9 CHAPTER 2 MANAGING LONGITUDINAL SURVEYS As described in the previous chapter, prospective longitudinal surveys have proved to be an important research approach, but certain limitations have also emerged that must be considered when these surveys are planned. The problems related to staff and management of longitudinal research differ in kind as well as degree from those encountered in cross-sectional research. The core of the problem in managing a longitudinal survey is a conflict between the need for long-term and for short-term resources. Plans and funding must be stable over many years, but the need for staff rises and falls over the course of a longitudinal survey. Most organizations sponsoring longitudinal surveys have solved the dilemma through some combination of permanent and temporary staff. Fluctuations in resources are less pronounced in longitudinal surveys that employ non-going rotating panels (such as SIPP or, to some extent, the CPI) than they are in fixed panel surveys in which interviews are conducted at longer intervals (such as NLS, NLS-72, or HS&B;). The major difficulty faced in planning and managing a longitudinal survey is in maintaining a core group dedicated to the project, and maintaining consensus between this group and senior agency staff. These groups tend to view long-term commitment of Staff and resources in different ways. The schedule, funding, and staff needs of a longitudinal survey are viewed differently by survey designers, by agency directors, and by those responsible for operations. It is a constant challenge to generate commitment to a long-term goal such as analysis of data, when senior staff with direct authority over the project often changes before the survey is completed. A. The Need for Long-Range Planning The need for long-range planning and organization for a longitudinal survey should be brought to the attention of senior staff very early with a planning document that outlines the workload, survey tasks, and anticipated products over time. The planning document should be prepared in conjunction with an analysis plan, and the design of the instruments and procedures will then follow once all groups are in agreement with the planning document. Long range planning is vitally important to a longitudinal survey, because it promotes enduring support at a senior agency level, it widens the pool of sponsors and supporters; and it begins the process of documentation that ensure continuity of operations. Principal Author: Lawrence Corder 11 A large-scale longitudinal Federal survey generally has at least nine principal management phases which may be briefly described as follows: 1. Budget Planning. Up to five years before data collection is to begin, a general plan must be conceived and provisions made to obtain continuing staff and funding resources throughout the longitudinal project. 2. Development of Position Papers. These are draft planning documents which discuss options, costs, and yields associated with various sampling plans, data collection designs, or questionnaires. These ensure widespread and enduring support for the longitudinal research. 3. Procuring outside assistance. If a contract is to be awarded, requests for proposals must be prepared, cleared and advertised, and responses must be evaluated before a contract is signed. This is a common approach to levelling out resource needs. 4. Final Research Plans. This stage includes final OMB clearance, conduct of field tests, revisions as necessary, and detailed agreements with any other cooperating agencies. 5. Data Collection. This refers to the full-scale field data collection. Longitudinal surveys (such as NLS) which have been extended beyond the original research period have repeated these 5 stages independently several times. 6 . File Preparation. Development of the system for data entry, data base design, processing, etc., may also require systems for optical scanning of questionnaires, machine/or manual edit steps, preparation of code books, the construction of composite variables, plans to preserve privacy in public data files, and numerous other activities. Each operation must be fully documented, to ensure comparability between waves. 7. Planning the Analysis. While the overall goals oft he analysis must be planned in the early stages, some details cannot be finalized until the data are available on computer files and code books are completed. Also, as policies shift, new analytical priorities must be met. In all cases, this process requires plans which may include in-house analyses and contracts for analyses. Contracts require a repetition of the procurement process described in phase 3. 8. Conduct of Analyses. These may go on for several years. Cross-sectional, analyses can be conducted as soon as one wave of interviews has taken place. Longitudinal analyses take place after some or all other waves are completed. 9. Publications. With in-house and professional peer reviews, these may continue for several years. 12 Each phase requires substantial time to complete, contains specific activities and results in the preparation of key documents. The final products of any longitudinal surveys are usually public-use data files and reports.* Ideally, these should be supplemented by rapid preparation of in-house documents as part of the policy-making process. Schedule milestones and due dates are part of any longitudinal survey, and the ultimate success of the project and even the usefulness of the analytical results may be judged against their timeliness. It is not unusual for a longitudinal survey to consume a decade or more from inception to completion of the publication plan. The NMCES and NMCUES Studies, for example, both took 8 to 10 years to complete. While field operations and the period for analysis vary with each survey's objectives and resources, the successful pre-field period is probably very similar in each case. The planning period should be dedicated to achieving consensus internally, then to producing instruments and obtaining clearances and approvals (for contracts as well as for questionnaires). A typical schedule for completing pre-field activities alone (excluding budget planning) would frequently require 12 to 18 months. Some of the most severe criticisms of longitudinal surveys have resulted from insufficient planning. It is not uncommon, for example, to omit thorough planning of the analysis. Then, at a production stage, it is discovered that people have different ideas on the tables and data to be produced and analyzed. It is also necessary to plan the linked files carefully so that the data needed for longitudinal analyses are readily available. Unfortunately, the planning of budgets and field work often takes precedence over the planning of processing and analysis, sometimes leading to delays, acrimony, and sometimes shifts in support. B. Funding Longitudinal Research The actual unit costs of doing longitudinal surveys may be no higher than for a series of cross-sectional surveys of comparable size and complexity (Wall & Williams:30). There is conflicting evidence on comparable costs, probably reflecting non-standard cost reporting on survey operations. Funds, however, must be committed over a number of fiscal years and budget plans are not easily altered. There is a trade-off to be made when errors are discovered or improvements can be implemented. Additional costs must be carefully considered, as well as the effect of changes in methodology on the longitudinal analysis. Errors, of course, should be corrected or, if too costly, an indication of their effects provided. Changes in methodology are different from changes necessitated by errors and must be thoroughly explored. Provision should be made to share information with analysts and data users on real change vs. methodologically-induced change. (The change to computer assisted telephone interviewing is one such change that needs careful exploration.) If errors or methodological changes result in higher costs, alternative methods of meeting those costs should be considered: higher funding, smaller sample size, more time between interviews, delayed processing, and so forth. Surveys of business or industrial establishments are often an exception to this rule, to protect the identity of large firms that dominate certain samples. 13 Inter-agency cooperation can help meet long-term funding needs. The Health Care Financing Agency (HCFA) and the National Center for Health Statistics (NCHS) chose this approach in conducting NMCUES. Inter-agency agreements frequently involve the Census Bureau for data collection and analysis, but they may also be used between other agencies with related research goals. Inter- agency Cooperation in longitudinal surveys could take the form of joint sponsorship of a new longitudinal survey, or it could be in the form of using an existing longitudinal sample as a vehicle for research to save the cost of starting a new longitudinal survey. The NLS-72 provides an example of a consortium approach: For the fifth follow-up interview in NLS-72, the National Science Foundation appended questions on math and science teachers, and the National Institute on Child Health and Human Development joined with the National Center for Education Statistics (NCES) to fund questions on child care and early childhood education issues. Longitudinal surveys are generally long term projects with significant start-up costs. If a survey can he constructed to serve more than one agency through an inter-agency agreement, start-up costs may be shared and several agencies will be bound to multiple-year funding commitments. When agencies select outside contractors to conduct longitudinal research, competitive procurement is required. The decision to use a contractor to conduct a survey increases the time needed to start a project, because approval of contracting plans must be added to other planning tasks. One advantage of contracting out the survey work is that it gives an agency access to additional staff support in cases where the agency has no authority to add permanent staff. Contracting for data collection by an outside agency may or may not be more expensive than employing a government organization for this purpose. In comparing costs, NCES found that the first NLS-72 follow-up, conducted by the Census Bureau, cost slightly more than the second follow-up, conducted by Research Triangle Institute (RTI), despite inflation. Other longitudinal surveys, including NMCES and NMCUES, have had just the opposite experience. The most cost-effective mode of operation appears to depend on the kind of survey, not on the agency conducting it. The duration of longitudinal surveys often requires periodic recompetition once a competitive award has been made. As a result, agencies have found themselves switching contractors part way through the data collection phase of a longitudinal survey. The competitive award of each data collection wave can, however, help control overall survey costs, because it provides contractors with an incentive to hold down their costs. The possibility of changing contractors over the life of a longitudinal survey requires a detailed documentation of methods that goes far beyond what is needed for any one-time survey. This level of documentation was not anticipated when the original contract to collect data for NLS-72 passed from the Educational Testing Service to RTI, and the change in contractors caused difficulties. Based on this experience, NCES now 14 builds a sub-contract to the previous contractor into any subsequent data collection awards. As a result, a later transfer of the NLS-72 contract from RTI to NORC was accomplished without problems. C. Staff Needs Staffing requirements for a longitudinal survey typically vary substantially, both by number and by type of staff throughout the history of the project. Staffing is much more controlled in rotating sample surveys, whether they are longitudinal or cross- sectional. Funding and staff needs for a longitudinal survey are much greater during the data collection period than during any other phase. However, some of the types of people needed for data collection, such as interviewers, are not needed in later phases. Staff monitors for field work and data processing are in high demand at early stages as well as intermediate stages. Because of sporadic needs, the use of a core group of survey professionals in combination with temporary staff, or interagency agreements or outside contracts, can be the best method to ensure adequate staffing for the entire effort. To distribute the costs of a contract more evenly over a longitudinal survey, NCES and NCHSR have used incrementally-funded contracts. During the longitudinal survey, separate contracts are awarded for each phase or wave. Each contract extends over two or more years. At any point, some survey tasks are being advertised for competition while others are being completed under contract. Looked at from the standpoint of each fiscal year, the total costs and level of effort remain more nearly constant. NCES has also found that giving agency survey analysts the responsibility for monitoring contract performance will help control variations in staffing patterns. By employing temporary peripheral groups in addition to permanent staff groups, two problems are solved: Research staff needs are met without adding permanent personnel to an agency; and peak workload needs are met without jeopardizing tight survey schedules. Inter-agency agreements or contracts not only bind parties to a specified set of research goals, but they also permit the level of staff effort to rise and fall as needed. D. Maintaining Core Staff The duration of longitudinal research projects creates another management problem (which has been called a Methuselah effect by Herbert Parnes). Each phase of a longitudinal study, such as planning, data collection, or analysis, is frequently carried out by different individuals, who may not even be part of the same organization. The relative inflexibility of a longitudinal study plan is an analytical necessity, but it could also prevent interim analysis or refinements in the design. For these reasons, it has been suggested that non-going longitudinal surveys may hold little interest for the calibre of professional staff that is needed for management or analysis (Wall & Williams: 35). NCES, however, has successfully attracted talented analysts to manage the agency's longitudinal surveys. To some extent this may be because NCES ensures that the Agency's staff have challenging responsibilities for program 15 analysis. Agencies which see only data collection as their primary mission may be more apt to encounter the staff problems recognized by Wall and Williams. in order to allow mid-course corrections and modifications of the survey plan, NCES uses a multi-phase sampling design (as in HS+B). This, too, contributes to the flexibility of the NCES longitudinal survey program. E. Data Collection and Processing Schedules Longitudinal surveys have become notorious for developing serious backlogs because data collection takes precedence over all other tasks. The schedule for observations is usually the least flexible aspect of the design, because each subject must have an identical record structure. As data collection continues, it creates an ever-growing backlog of other procedures, such as analysis. Uncompleted tasks tend to accumulate, becoming increasingly difficult to finish. To prevent backlogs and delays, a longitudinal survey must be well-organized and planned so that analysis and data release keep pace with data collection. Data collection schedules are not the only factor in backlogs. Another factor is data processing, including file linkage. Survey organizations that are more accustomed to doing cross-sectional surveys or other non-longitudinal surveys often have difficulty recognizing the special processing needs of longitudinal surveys. Databases need specification, key variables,need identification, and a policy on imputation needs to be thought through. Ideally, all this needs to be done when the survey questionnaire is designed, but this ideal is seldom, if ever, met. F. Data Analysis Data analysis is often looked on as the rewarding part of the job after the difficulties of data collection and data processing. Analytical interests often go beyond the agency conducting the study. Some agencies include analysis contracts in their contracting for services. Usually some analysis is done by agency personnel. One possibility to counter some,of the delay caused by the time it takes to complete a longitudinal survey is to analyze each wave as if it were from a cross-sectional survey. This not only provides timely data, but raises questions to be answered at later stages, and generally whets the appetite for more data and more analysis. Recent data from non-going longitudinal programs can be analyzed relatively quickly to serve some analytical purposes without delay. It is also possible to add questions to the current data collections of a longitudinal survey to meet immediate data needs. G. Release of Data A principal goal of any longitudinal survey should be to produce public use data tapes and analytical reports rapidly, both for policy-makers and the interested public. If public use files are to be created, then procedures to 16 protect confidentiality must be worked out in advance, File structure and documentation need to be readily available. Variance estimation must be provided for those using the file. The permanent survey staff should maintain a role in the preparation of files and reports, so that their expertise and interest are not lost. In conclusion, longitudinal surveys, sometimes taking 5 years or more to complete, inevitably encounter staff changes. Two management approaches can minimize the loss of institutional memory. First, it is vital that every survey activity be documented. Interview instructions, edit specifications, variable definitions, file layouts, sampling, weighting and imputation methodologies, all instruments and procedures should be recorded and readily available. This task is very labor-intensive and, unfortunately, apt to be slighted when staff time is short. Second, inter-agency agreements or contracts may clearly lay out both the procedures to be used and the final products. It is also wise to specify key contractor staff persons who cannot be replaced without sponsor approval. These actions are important to minimize the effect of staff changes and to prevent errors and delays. 17 18 CHAPTER 3 LONGITUDINAL SURVEY OPERATIONS The principal differences between field and processing operations in one-time surveys and in longitudinal surveys are created by the use of time as a significant factor in research. Longitudinal surveys typically encounter changing conditions, and survey designers have developed and evaluated a variety of methods for controlling the problems that can be caused by change in the sample or changes in the design or administration of the survey. A. Sample change over time The composition of the sample may be expected to change across waves for a variety of reasons. Respondents may refuse to participate, they may die, they may move and cannot be found, or they may leave the sampling frame (e.g., by entering an institutional population or by moving abroad). The danger is that the sample becomes increasingly less representative of the target population as time passes. To minimize the effects of these problems, new observational units are routinely introduced into the samples of some continuing surveys as time passes. 1. Selection of new units into sample For some longitudinal surveys, they are a number of concerns related to the length of time respondents are kept in sample. Respondent burden across several interviews may produce a decline in the quality of data gathered or may result in increasing refusal rates. Respondents may also leave the sampling frame, move and cannot be tracked, or die, thereby affecting the representativeness of the sample. for these reasons, it may be desirable to institute a rotating panel design, which regularly moves new respondents into the sample and retires other respondents after a fixed number of interviews or period of time. The Survey of Income and Program Participation (SIPP), the National Crime Survey (NCS), the new Consumer Expenditure Survey (CE), and the Consumer Price Index (CPI) have all adopted rotating panels. SIPP introduces new respondents annually and retains them for 2-« years (7 or 8 interviews) before rotating them out; NCS introduces new respondents monthly and interviews them for 3-« years (7 interviews). The CE Survey introduces respondents monthly and interviews them five times on a quarterly basis, while the CPI introduces new respondents once every five years and interviews monthly or bimonthly. Fienberg and Tanur (1983) note that rotating panel designs may create some problems of inference, according to conventional sample survey theory, in that random selections of respondents occur at different times for different respondents. The argue, however, that this is only important when date of selection is related to temporal changes in the phenomena the survey was designed to measure. The inferential Principal Author: Bruce Taylor 19 difficulties which might result from a rotating panel design must be balanced against the reduction of attrition-related bias, which is the alternative. 2. Movers Some respondents may be expected to move from originally sampled housing locations (or telephone numbers) during their time in sample. Depending on the purpose of the survey and procedures adopted to track movers, respondent mobility has varying implications for the representativeness of the sample over time. a number of factors may enter into decisions regarding whether, or how, to follow movers. A crucial consideration is to determine the most important unit of observation for the survey. A longitudinal survey of persons may be designed to follow sample individuals or households, if the substantive goals of the survey would be served by retaining as many of the originally sampled respondents as possible. A number of surveys, such as SIPP and NLS, focus on individual and household economic data, which continue to be relevant to the purposes of the survey regardless of respondent mobility. Consequently, following movers is an appropriate means to maintain data quality over time for such surveys. Following movers may create other problems, however. For instance, if there are ecological correlates for the phenomena of interest, such as crime or quality of housing, then following mobile respondents may result in deterioration of the geographic representativeness of the original sample, with a consequent potential for bias in some measures for later waves. A rotating panel design may minimize this problem, because newer respondents are more likely to reside in the originally sampled housing location. Another reason for following movers is that respondents may move for reasons related to the substantive goals of the survey. This makes it important to know why they move. If this is the only reason for following movers, then collecting data for only one wave after a move may be enough. In NCS, for example, some respondents may move from a high-crime area to a safer neighborhood, and it would be important to determine the proportion of moves which were related to crime victimization can be measured, but not the future consequences of victimizations for such movers. The SIPP is attempting to follow all individual movers. Because living arrangements vary according to economic circumstance --and affect eligibility for social welfare programs -- a change in residence can be related to changes in income and program participation. Thus, for SIPP it is crucial not to lose data on movers. The CPI, on the other hand, follows only those movers who provide services, such as doctors or lawyers, since their expertise is the item being purchased. When a commodity outlet changes location, this move is considered a unit "death" and the CPI record is terminated. The actual procedures developed for following movers are likely to reflect the field procedures of the organization conducting the survey, the collection mode used, the distance involved, and the costs associated with tracking movers. If the organization conducting the survey uses decentralized collection procedures, a respondent moving from the jurisdiction of one regional office to another may be more difficult and more expensive to track. Also, the costs of following movers may be greater if a face-to-face collection mode is used, rather than a telephone design, where tracking procedures may 20 be limited to obtaining a new telephone number. Depending on the cost, administrative difficulty, and proportion of respondents who move far enough to create problems, it may not be desirable to follow all movers or to rely on standard collection modes. SIPP field procedures, for instance, indicate that personal interviews need not be administered if the respondent has moved beyond 100 miles from any sample PSU, and rules also differ for respondents younger than fifteen years of age. If survey procedures allow telephone interviews in lieu of face-to-face interviews, a phone contact may be a desirable alternative for movers who are difficult to reach. The type of sample involved may also affect the ease with which movers may be located. For instance, it is usually easier to find a mover through neighbors or subsequent occupants of a sample housing unit if an area sample has been adopted rather than with a random digit dial sample. Asking respondents to notify the field office with pre-printed cards when they move can be a partial solution, but this option relies heavily on the respondent's cooperation. 3. Attrition When projected across waves of a longitudinal survey, manageable levels of non-response in a cross-sectional survey can become significant sample attrition. The potential for attrition in a longitudinal survey sometimes limits sample definition. Tracing mobile respondents generally accounts for a large proportion of field problems as well as costs, and refusal rates are likely to grow over the life of the survey. Incomplete records and missing interviews create analytical complexities that are unparalleled in cross-sectional research. Attrition is most dangerous when it is correlated with the objectives of the survey. For example, there is evidence that sample attrition may be related to victim status in the NCS. To the extent that the sample loses victims at a faster rate than non-victims, estimates from later waves will be biased. Also, Fienberg and Tanur(p.17) note than in social experiments disproportionate loss of respondents for different treatments may be a problem, because treatments often vary in their attractiveness to participants. Sample attrition between observation periods may create the illusion of change when means are compared between waves, without adjusting for non-response. In study focused on identifying change, there is a risk that changes are spurious, due to sample attrition. In addition, respondent participation that varies from panel to panel could produce the appearance of change even when aggregate non-response is stable. The estimates of central tendency (Cook & Alexander: 191). Mean test results from longitudinal panels of students taking ETS exams were compared to mean test results derived from a cross-sectional survey of the same population. The means were significantly different, which the analysts attributed to selective attrition in the longitudinal sample. Effects of attrition in demographic surveys have been harder to predict. Attrition does not necessarily created unmanageable bias in a longitudinal survey: The NLS was still contacting 92 percent of living respondents 3 years after the original contact, and still contacting 80 percent of eligible respondents 12 years after the study began (U.S. Department of Commerce:321). In the ISDP panels of 1978 and 1979, attrition did not climb steadily over the five or six interviews administered to respondents. Instead, it leveled off and then declined slightly over all waves (Ycas:150). Nonetheless, a combination of attrition and varying participation from wave to wave can create serious 21 problems in creating complete records. In the 1979 ISDP panel, for instance, only two thirds of the original sample persons had complete interview records (Ycas:150). Calculating the response rate in longitudinal surveys is itself difficult. The measures used in cross-sectional research are often not adequate for measuring non-response in complex records, as they do not reflect cumulative non-response across waves and do not take into account changes in the size of the eligible sample due to births, deaths, and the addition of new household members. The illustrate, non-response for entire housing units in the NCS is sometimes reported at 4 percent. However, when records for housing locations are linked to form a longitudinal file, it has been found that over half of the originally sampled housing units are missing at least one interview. This discrepancy is due to the fact that the former figure is a cross-sectional measure of unit non-response in a particular wave and does not account for the approximately 10% of sample housing units unoccupied at the time of interview (Fienberg & Tanur:14). This figure also dies not cumulate non-response over time. While the lower figure is an appropriate measure for many cross-sectional uses of NCS data, it clearly is inadequate for reflecting the completeness of linked housing unit records. The methods that have been developed for tracing respondents in longitudinal surveys have been successful, but they have also proven to be expensive. The Census Bureau has estimated that the cost of contacting each wave of an ISDP research panel increase by 8 percent over the previous wave, due to the costs of following movers and interviewing additional households (Fienberg & Tanur:11- 12, White & Huang). However, NCES also found that per-unit tracing costs for the High School and Beyond (HS&B;) Survey were approximately 20% less than the cost of base year sampling, which illustrates the economies which can be realized by mounting a longitudinal study, rather than separate cross-sectional studies. To control costs, as well as potential bias, each longitudinal survey must investigate the characteristics of respondents who move. Depending on empirical evidence about how atypical non- respondents are, a judgment can be made about the proper balance between the costs of tracing respondents and an acceptable level of non-response. Sample definition offers another approach to limiting unscheduled attrition. The probability of becoming a non- respondent is not randomly distributed among the population. In longitudinal samples such factors as rural resident, interval since contact, and region of the U.S. affect the probability of maintaining contact (Artzrouni:21-24). Some longitudinal designs have therefore sought to minimize attrition by avoiding the respondent classes that are most susceptible to attrition. Setting aside respondent classes to control attrition can conflict with attaining a sample that truly represents the reference population. However, a sample chosen without regard to eventual tracing difficulties may also gradually lose its representative power through attrition. Only empirical evidence can indicate the extent to which characteristics that predict attrition co-vary with the characteristics that the study is designed to investigate. A sampling design which sets aside respondent classes with potential attrition problems should be undertaken only after careful consideration of the relative magnitude of bias which could be introduced by such a strategy and other alternatives, such as imputation for missing data or performing analysis on the remaining sample cases of an initially representative sample. In cohort or panel studies, which require measurement to begin and end at the same time for all respondents, implementation of a rotating panel design, which reduces the impact of attrition by replacing respondents over time, will clearly not serve the goals of the survey. One possible strategy for dealing with attrition in such studies is to impute. 22 missing data, based either on statistical models or on complete data from prior waves or from respondents with similar characteristics. Another possibility is to reweight the sample for each wave to reflect non-response for various demographic groups in the sample. (See Chapter 4.) Duncan, Juster, and Morgan (1982) model such a procedure for the Panel Study of Income Dynamics (PSID), conducted by the Institute for Social Research (ISR) at the University of Michigan. They compare results for data gathered with persistent efforts to pursue respondents and for the data set which would have resulted if less intensive respondent contact strategies had been adopted. When the latter is reweighted to adjust for missing cases and compared with the first data set, there are minimal differences in outcome measures. While this procedure has promise for minimizing bias resulting from non-response across waves, it may also allow some relaxation in pursuing respondents, allowing cost reductions in survey administration. The authors do note, however, that reweighting entails some risk of covariation-related bias in multivariate estimates, especially for models that are not well specified, and that maintaining an adequate number of respondents in some key subsamples may remain a problem. A reasonable precaution to minimize the deleterious effects of sample attrition is to minimize respondent burden, which has been variously described as the amount of time which an interview entails or as the complexity of the task required of respondents for successful completion of an interview. Under the Paperwork Reduction Act of 1980, each Federal statistical program is restricted to a limited number of hours available for data collection in a fiscal year, thereby encouraging reduction of the burden placed on respondents. In addition to the statutory reasons for limiting the length of Federally sponsored surveys, controlling respondent burden may also improve data quality for longitudinal surveys in a number of ways. An important aspect of this data quality enhancement is that, respondent participation may be encouraged by reducing interview tedium, thereby reducing refusal rates and enhancing the representativeness of the sample over time. Respondent burden hours may be reduced by a careful evaluation of the utility of collecting information in every wave. The SIPP, for example, minimizes respondent burden by dividing the survey into a core questionnaire ad ministered at each interview, plus "topical modules" to collect data not required as regularly. Sometimes only a subsample of respondents should answer certain topics. Finally, lengthening and/or varying the intervals between waves should also be considered as a means for reducing respondent burden. The CPS, while not a longitudinal survey, adopts this strategy of varying tim e between interviews. Respondents are interviewed for four months in succession, not contacted for the following eight months, and then interviewed for a final four months. 4. Changes in Units of Observation A slightly different sample of respondents participates in each wave of a longitudinal survey. Such changes in sample may result from scheduled introduction or retirement of sample units in a rotating panel design, from attrition, or from introducing new respondents when household composition changes. This variation causes difficulties related to defining the correct reference population, in weighting for item non-response, and in weighting respondents who enter and leave the sample. In addition, the changing sample of respondents and aggregate units creates unique difficulties in analyzing data above the person level A variety of approaches has been used to define units of analysis in longitudinal research, and each has specific problems and strengths. These are discussed in detail in Chapter 4. 23 It should be noted here, however, that all weighting adjustments should be planned simultaneously. The problem of adjusting for non-response is the converse of problems created by persons entering the sample, and the adjustments for entrants and non-coverage, once selected, can be accomplished in a single operation. Split and merged households present particular problems for sample comparability across waves. Such recomposition of households creates obvious difficulties for longitudinal matching, which will be discussed below. However, changes in household membership also raise questions about how to treat new members of split households who were not members of the originally sampled household but who came into sample because of their associations with original sample persons. Rules developed by the ISDP offer one method which seems generally applicable to a number of surveys: New household members were added to the sample, but if they left the household, or if this household subsequently split, only those members who were selected for the original sample were followed. This procedure avoids excessive growth of the panel, thus minimizing artifactual changes in aggregate panel statistics, but still collects relevant household data which correspond to data from "stable" households. Whether a change in a household constitutes the birth or death of the sample unit depends on the goals of the survey. If the survey samples households and does not follow movers, then a complete turnover in the household occupants would indicate the birth of a new unit. If housing locations are sampled, then such a turnover would not constitute a death as long as the hosing unit remains occupied. The death of a member of the household, or event he head, does not constitute death of the unit for a household- based sample, but a divorce or separation often will be defined as termination of the unit. If an individual respondent leaves the sample, the reason for the departure should be determined. If the respondent has died, then the individual record should be terminated. However, if the respondent leaves the sampling frame for other reasons (e.g., entering the military or moving abroad), it is possible that he or she may return during the life of the panel, and the record should be retained. Often the death of a unit can be determined by observation. For instance, when a housing unit is vacant or destroyed and the sample is location-based, termination of the record may be indicated. However, in other cases respondents must be queried regarding the status of the unit. If the unit of measurement is the household, occupants of the sample location must be asked whether they lived at the current address when the previous interview took place to determine whether they should be considered part of the sample. (Rules for this decision will vary between surveys.) If only part of the household has moved since the previous visit, it may be necessary to determine the reason for the departure to ascertain whether the movers remain in the sampling frame. In designs which do follow movers and which allow the formation of new households during the life of the sample, permanent departure of individuals to form new households will indicate the need to establish new household records. (See Chapter 4 for a fuller discussion of these issues.) B. Changes Related to Respondents' Time in Sample Varying sample participation is not the only change over time which complicates inference from longitudinal data. A number of factors related to the time respondents remain in sample may produce changes in survey measures which are independent of any substantive changes in the phenomena under investigation. These factors include variation over time in the rules for interviewing particular respondents and changes in 24 respondents' approach to the interview based on increased experience with the survey instrument as the sample matures. 1. Response Variability Due to Changes in Respondent The manner in which a survey is administered may vary from respondent to respondent. "Proxy" interviews may be administered, in which adult household members complete interviews on behalf of younger respondents, or in which available household members supply data for other individuals in the household. (In some cases such proxies are restricted to household members who are not present, but, in other instances, one household member will supply personal data for all individuals in the household.) Respondent rules are also frequently needed for collecting household information if there is more than one respondent per household. A number of possibilities exist for respondent rules. For example, one respondent in a household may be selected to provide household data, while personal data is requested from each respondent individually. Alternatively, all respondents may be asked for household data. In the latter case, inconsistencies might be reconciled in the field, for instance, when respondents report conflicting details regarding a household crime incident. A computer edit, or a postweighting algorithm might also adjust for differences in reporting, when household measures are simply the sum of individual measures. Respondent rules can affect longitudinal data over tim e. For instance, during a longitudinal survey, younger respondents may become eligible to complete an interview without proxy, and may begin to report information of which previous proxies are unaware. There is also evidence that household-respondent status may affect the manner in which personal data are reported, particularly if the two types of information requested are related. Biderman, Cantor, and Reiss (1982), for example, find that respondents who report household data also report higher levels of personal crime victimization than do respondents who do not report household data. They also find that, if the household respondent changed between interviews, levels of personal victimization for the affected persons would also change. The authors hypothesize that the initial battery of household victimization items serves as a warm- up for personal items and aids recall for household respondents. If the household respondent is allowed to change across waves, then two effects should be anticipated. First, the quality of personal data reported by a given respondent is likely to change over time, depending on whether he or she serves as the household respondent. Second, different household members will vary in their knowledge of the relevant data, so the quality of household data may also be expected to change over time and thereby bias transition estimates. There are some obvious remedies for these problems. First, proxy interviews should be minimized, recognizing that obtaining certain information directly from younger respondents may be inappropriate or that there maybe no other way to collect data for some respondents. Surveys vary in their reliance on data collected by proxy (eg., about 60% for NCS, 40% for SIPP), and such a policy is likely to produce an improvement in data quality proportionate to the fraction of data currently collected in this manner. Second, care should be ta ken in assigning responsibility for answering questions about the household over time, either by consistently assigning this responsibility to the same respondent or by requesting these data of all respondents. The latter procedure minimizes the effect of an unavoidable change in household respondent and makes any respondent effect consistent across all waves however, due to mandated 25 ceilings on response burden for federally sponsored data collections, the additional precision realized may not justify the substantial number of redundant questions which are required. It should also be noted that the reconciliation procedures or post- weighting that would be required may make such a strategy very difficult to use. 2. Panel Bias A number of factors associated with respondents' time in sample may produce changes in survey measures over time and thereby complicate explanation. The impact of these factors has been described as a history effect, secular effect, maturation effect, rotation group bias, time-in-sample bias, or Heisenberg effect. These factors include the reactivity of respondents to survey measures, changes in the performance of the respondent role, the "conditioning" effect of multiple administrations of the survey instrument, the aging of the panel, interaction between interviewers and respondents, interviewers' perceptions of their role, and the correlation between variables of interest and the probability of response. Changes in survey measures due to such effects present a danger for bias in longitudinal estimation. Consequently it is important to consider the influence of such factors when designing a longitudinal survey and to minimize the potential for such changes. This is a difficult task, because the reasons for the phenomenon are not clearly understood. Ideally, the process of measurement should itself produce no change in the phenomenon under investigation. Research methodology in experimental psychology, for example, often involves disguising the purposes of research, so that the subject will produce the behavior under investigation with minimal "contamination" by the research procedure. In survey research, however, the respondent must not only understand the measures being collected but also must be led to appreciate the purposes and value of the research if response rates are to remain high. This is particularly important for longitudinal surveys, where retaining sample is a crucial goal Consequently the danger of reactivity between survey interviewing and the phenomena under investigation is a particular problem. Researchers studying labor market experience, for example, have speculated that repeated interviews asking about job mobility might cause some of the mobility reported (Parnes:15). Questions about mobility may in fact cause subjects to consider the possibility and act upon it. National Crime Survey data also indicate that proportionately fewer crime incidents are reported in successive waves. This finding may stem from respondents' heightened awareness of vulnerability to crime, caused by participation in the NCS, which results in increased precautions taken against crime victimization. It has been suggested that respondents in a longitudinal sample might exhibit non-typical behavior Simply because repeated questioning regarding a topic may alter respondents' perceptions of the subject under investigation and change their behavior or attitudes accordingly. For respondents no remain in sample, their responses can change over tim e solely as a function of longevity in the panel These temporal variations in response have implications for the quality of longitudinal data which are often unpredictable. In some cases, the quality of data may improve over time. Respondents may understand the respondent role better with repeated interviewing or pay greater attention on a day-today basis to the experiences being measured, with a consequent improvement in the richness or accuracy of the data gathered. Alternatively, if respondents or interviewers find the interview tedious or burdensome, they may become less enthusiastic about the 26 task over successive waves and avoid or give incomplete responses to survey items. One aspect of such a decline in data quality is the possibility that respondents may be "conditioned" by their participation over several waves to provide answers which produce artifactual changes over time. For instance, respondents may learn that a particular response will trigger a long battery of questions, which they may prefer to avoid in the future. This is one alternative explanation for the decline in the rate of crime victimization reported in the NCS over successive waves. Respondents may learn that reporting a crime incident leads to an additional series of items for each incident reported, which results in a substantially longer interview. The Census Bureau's Current Population Survey (CPS), which is not strictly a longitudinal panel survey but which has many of the attributes of a longitudinal survey, exhibits a similar trend. Reporting unemployment triggers a battery of questions dealing with reasons for unemployment and activities directed towards looking for work. Reported unemployment invariably falls between the first and second waves of interviews in the CPS. This phenomenon in CPS could be related to several factors. One has to do with repeated interviewing and attrition. Williams and Mallows showed that, if the probability of response in a given save of interviewing was correlated with variables of interest, then, even with no change in the variables, a spurious change would occur. The passage of time can also produce unintended change between observations because of gradual shifts in the meaning of questions and answers. Even when questionnaires are not changed, there may be evolution In the way respondents perceive or answer questions, which produces the appearance of movement (Parnes:14). This might be caused by events (including the survey itself), by maturation in the sample, or by non-response. It is very difficult to determine whether a change across waves is real change or spurious change. Continuing validation research is necessary to identify panel bias in longitudinal data. Panel bias may be studied by comparing data collected in subsequent waves of a longitudinal survey to data collected in cross-sectional surveys (as in Cook & Alexander). Although some conditioning or panel effects may be inevitable, several tactics can be used to minimize their impact. One option is to implement a rotating panel design to replace respondents after a predetermined number of interviews. This procedure affords two primary benefits. First, those respondents who have been in sample the longest are replaced with more "inexperienced" respondents. Second, the temporal overlap of old and new sample facilitates studies of time in sample effects. All respondents are administered the same instrument under the same conditions at the same time, which serves to test alternative hypotheses about panel effects. Another possible means to attenuate or postpone the effects of panel bias is to minimize the respondent burden imposed by the interview. Careful construction of the instrument to minimize tedium and encourage respondent rapport should be central concerns in planning any survey but take on added importance in longitudinal data Collections, because of the need to sustain the active participation of respondents overepeated interviews. The overall length of the instrument may play a role in the respondents willingness to participate fully in successive contacts. However, design of the instrument to minimize tasks which the respondent is likely to find either tedious or particularly difficult is also an important consideration. Use of long follow-up batteries should also be minimized, to attenuate the effects of respondent conditioning. 27 C. Operations Change Over Time Changes in the administration of a continuing survey are almost inevitable. Revisions to the instrument, redesign of the sample, introduction of new collection modes, and transfer of data collection responsibilities to another organization can all introduce changes in the data and compromise the validity of longitudinal comparisons. While a consistent time series may be difficult to maintain under such circumstances, means exist which allow the analyst to deal with the effects of such changes. Eventually in most longitudinal research there is a pressure to change the survey measures in response to changing hypotheses. In addition, later findings frequently indicate a need for measures of new variables. Particularly when longitudinal research is exploratory and designed to identify significant correlates of change, researchers may be inclined to correct large a mounts of data to minimize future requirements for change in the questionnaire design. This aspect of longitudinal research may be costly, but it is an understandable precaution given the tendency for research hypotheses and/or policy-aims to change over time. To accommodate changing methods, a survey may be run under old and new procedures simultaneously for a period of time, to allow comparisons between data collected before and after the change. Ideally, both old and new designs should be implemented at full sample, in effect twice the usual sample size, but budget constraints will often make this impractical The CPS has adopted this double-sample strategy to phase in new samples based on the 1980 Census. The CPI also used both old and new sample designs simultaneously for a six- month period in 1978, when the survey was revised. Another strategy to consider when a questionnaire item is rewritten or a derived variable in a file is altered is to make changes in such a way that analysts may record the revised variable to correspond to the original variable (and vice versa), or to retain old questionnaire items in the revised instrument for some time. NCES adopted the latter strategy for the HS&B; survey when it adopted an "event history" approach to gathering employment and education data. In addition to the new items, the previous "Point in time" activity item was continued, allowing calibration of new items to the old and providing a degree of comparability between versions. To reduce field costs, many sponsor agencies have approved designs which permit data collection by telephone after the first visit. NMCES and MNCUES, for example, used phone contacts for follow-up interviews. The available evidence suggests that such changes in mode may not produce uncontrollable fluctuations in the measures obtained. Benus (1975) notes that data collected by telephone and by personal visit for the Panel Survey of Income Dynamics (PSID) are quite similar. Groves and Kahn (1979) found overall that univariate distributions and bivariate relationships were not significantly different for 200 questions ad ministered by telephone and in person. However, they note that telephone interviews elicited more rounded financial figures, less detailed responses to open-ended questions and narrower distributions on some attitude items. They also indicate that respondents tend to perceive telephone interviews as longer than personal interviews of the same length. Findings that telephone respondents tend to give more "don't know" answers to filter questions triggering other questions may be related to this difference in perception of length. Telephone respondents may be more eager to bring the interview to a close. Consequently minimizing respondent burden seem s particularly crucial for interviews conducted by telephone. 28 While the research literature on the effects of interviewing mode on survey response is generally encouraging, there are enough examples of differences in respondent behavior to indicate that a mixed mode design should not be implemented without adequate pretesting and analysis of the effects. One danger is that a particular questionnaire design or questions about a certain subject area might trigger mode-related differences in respondent behavior. To facilitate measurement of such mode-related response variability, it is desirable to design shifts in mode of data collection so that the changes across waves are systematic, making the effects measurable. It is also important in surveys which do not require interviews with all household members to ensure that interviews are obtained from the same household members when the interviewing mode varies across waves, as respondent availability may vary by mode. In conclusion, prospective longitudinal surveys require administrative and operational features that are different in kind as well as degree from those in cross sectional research. The long-term analytical goals of the survey must be considered in planning every aspect of sample definition and weighting. Provisions should be made for validation studies to evaluate such factors as attrition and panel bias. Finally, changes in format, operations and staff must be anticipated and managed in ways that ensure the comparability of measures from wave to wave. In practice it is worth noting that there are only a limited number of organizations which handle nearly all large-scale longitudinal surveys. Due to their experience, these organizations have a high level of expertise, and the continuity of experience contributes to successful planning and implementation. However, the concentration of longitudinal research in such a small number of organizations increases the impact that any errors, such as limitations in the sampling frames most commonly used, would have on the representativeness of longitudinal research. D. Processing While the measures collected in longitudinal research may be similar, to those collected in cross-sectional studies, there are special problems in controlling and interpreting them. The sheer size of the data files created in national longitudinal surveys creates special problems in processing and analysis. The massive files can be difficult, expensive, and slow to process, which has often limited their use to organizations with the staff, equipment, and often complex software capable of handling complex data sets. As a result, data analysis has typically lagged behind the accumulation of data (Kalachek:17). Fortunately, this situation is changing with the advent of public use files for multivariate analysis and with the dissemination of m ore user-friendly "statistical data base" packages to facilitate data management and analysis. In processing data from longitudinal surveys, difficulties are encountered related to cross-wave case matching, cross-wave data revisions, and preparation of data files for analysis. Often there is no single "best" procedure for processing, because ease of processing and analytical requirements are not always compatible goals. Errors in individual record files can cause multiple problems. Often items which should remain consistent across waves (e.g., race and sex) or which should change only in predictable ways (like age and marital status) will exhibit changes due to respondent confusion, transcription error by interviewers, or keypunching errors by processing staff. Detecting these errors is important, not only because such items often define key 29 demographic variables for analysis, but because such items are frequently needed to match cases. Errors are also inevitably introduced when imputations are made for missing data. Several procedures are possible to minimize errors. For SIPP, the field office staff immediately checks completed interviews to reconcile discrepancies, avoiding more costly correction of data after they have been keyed. Another possible procedure is to build computer edits into the processing system to detect inconsistencies between current and prior interviews. NLS-72 and HS&B; use machine edits to identify and resolve inconsistencies for about thirty critical items. Another option, utilized by CPI, is to create a machine-generated control card, which avoids errors in transcription and which provides interviewers with prior-wave data necessary to reconcile discrepancies in the field. This latter procedure, however, can also lead to reduced reporting of actual change. 1. Cross-Wave Matching In order to link data across waves, variables must be created to match records at the desired unit of analysis. A number of data management issues must be addressed, including the consistency of linking variables across waves, providing for longitudinal matching at multiple levels of analysis, and rules for matching merged and split households. If longitudinal records are not matched correctly between waves, the effects can be similar to sample attrition or non- response. The records of one or more observations will be missing from a respondent's longitudinal file, giving the appearance of missing interviews. One possible consequence of matching errors is error in analysis, either because incomplete records are deleted, or because missing data are imputed. If records are linked incorrectly, longitudinal data are also likely to produce flawed results by showing false changes in status. Even cross-sectional analyses may be in error, if control card information or data from previous interviews are carried over onto the improperly matched record by the processing system. A number of procedures are possible for linking units accurately from wave to wave, including matching of household and individual line numbers, or matching independent person and/or household identification numbers. Economy in the number of variables used for a match is generally a virtue, because the opportunity for mismatches due to transcription or coding errors increases with the number of variables used. So does the likelihood of missing data, which often results in the computer assigning a missing data code, which hampers matching. Limited redundancy in linking variables can, however, provide some protection against false matches, in that such cases are more likely to be flagged in the matching process. Validation procedures to detect longitudinal mismatches should be incorporated into the processing system and can often rely on demographic variables which either should not change over time (e.g., race, sex, or date of birth) or which can be expected to change in predictable fashion (e.g., marital status or age). Such methods are particularly useful when person-level matching is performed using the assigned line number of respondents within household. It is also useful to imbed check digits in key linkage numbers, to detect miskeying. In addition to careful design of validation variables, immediate error checking by the field office of items important for matching and validation is likely to reduce the number of mismatches significantly. 30 Often, person records are linked across waves by matching on household ID and on the line number of an individual within the household record. This is usually cumbersome, and it makes linking individual data across waves extremely difficult if an individual moves out of the sampled household, if the household dissolves, or if the household merges with another household, all of which render the previously assigned household ID obsolete. Consequently, for surveys which are intended to follow individuals, regardless of the duration of their association with a sampled household or household location, assignment of an independent person ID is highly desirable. This is not to argue that ID is at other levels of observation are not useful, as longitudinal analysis at household, person, or event level is often needed. The important consideration is that linking variables be designed so that changes in sample composition do not prevent record matches. SIPP has implemented an ID which, while complex, illustrates the sort of linkage which is often desirable. (Cf Jean & McArthur, 1984). The ID consists of: PSU number - 3 digits Segment number - 4 digits Serial number - 2 digits Address ID - 2 digits Entry address ID - 2 digits Person number - 2 digits Household ID consists of address ID, PSU, segment, and serial numbers. The latter three numbers are fixed once assigned. The entry address ID also does not change. The first digit of the address ID indicates the wave at which the household was interviewed at that address. The second digit sequentially numbers, by address, households resulting from a split into two or more households by original sample persons. The first digit of the person number indicates the wave at which the respondent entered the sample, and the second two digits sequentially number persons within the household. This ID also remains fixed. Linking households or individuals with the SIPP system is fairly straightforward. Households whose composition does not change require the household ID, and individuals require the household ID and person number to provide a match. The inclusion of a fixed entry address ID also facilitates matching records for individuals or households who move, and for split households. Combining the person number and the entry address ID provides a person number which remains constant regardless of changes in address and household composition. This provides a link to data collected for an individual across all waves, allows a match to the initial household, and permits the analyst to filter data for only the original survey respondents, if desired. This system remains adequate for multiple movers or for households which split a number of times. In 1979 two waves of interviews from an ISDP panel were merged into a single longitudinal file using personal identification variables. Mismatching between records proved to be a significant problem, and there was evidence that additional matching errors were undetected (Kalton & Lepkowski:26). A second file was created using ID numbers rather than personal characteristics. This file had significantly fewer discrepancies during edit checks for such items as sex and age, indicating that fewer matching errors occurred with the use of the ID number for linking. 31 Sometimes the potential of longitudinal data has not been exploited because of the complexities involved in updating data with information collected in subsequent waves. For instance, a respondent may report a crime victimization or a health problem, but information on insurance coverage will remain incomplete, because the claim had not been settled at the time of the interview. It is frequently desirable to revise or add data during a later interview and to create an automated control system which would allow revision of the original record. One possibility is to provide a check item on the instrument for information which is frequently incomplete. The control system could then flag incomplete data during processing and direct the interviewer to follow up on this question in a later wave. Similar procedures were used in N M C E S and N M C U E S, which allowed validation of data collected on health care payments and insurance coverage during later interviews. Revising files obviously creates some complications, and there are trade-offs between ease of processing and ease of analyzing the revised records. One of the simplest procedures for processing is to reserve a field for follow-up data in the interview along with an incident or event ID which allows a match to the original record. This procedure unfortunately would make the analyst's task considerably more difficult, in that several files would have to be scanned to locate all updated material. The required matching and file restructuring routines would also be rather cumbersome and expensive to run, unless the data were released in a form compatible with a statistical data base which performed the matching. These complexities create potential for data management errors, particularly for inexperienced users accessing public use files. The alternative is to correct the original records based on followup data and to release the updated files. A disadvantage of this procedure is that several versions of the same, file would be in circulation.* Nonetheless this procedure appears to have greater potential for facilitating straightforward analysis and management of the data, particularly if early versions of a file are labeled as "preliminary." 2. Data Structures to Facilitate Analysis A number of strategies may be used to create longitudinal data files. One is to create, a separate fixed length record for each case at the smallest unit of analysis, with separate fields devoted to repeated measures of the same variable. Often this is not feasible, because this procedure entails a thorough revision of the file every time a new wave is completed. It is often preferable to produce a separate file for each completed wave or even more frequently if data collection extends over a lengthy period and to include in the files a number of linking variables which remain constant for each case across waves. Other than the size of the files produced, the main difference between these two approaches then is in the processing system adopted: The former produces Integrated longitudinal files, while the latter produces files resembling crow-sectional data sets which allow the analyst to link the records later. Producing a file which uses the smallest unit of observation as the basis for a record is often not the most efficient structure for a data set. A number of surveys ________________________________ *This is not as serious a problem for longitudinal files, the latest version of which can more easily be identified, as it is for cross-sectional files created from a particular wave. 32 collect data on households, individuals within households, and discrete events experienced by the household in aggregate or by individual members. Given the implicit "nesting" of such data, creating a file based on the smallest unit will result in much redundant information for higher level units. The number of events recorded and the number of household members may also be expected to vary between households, and variable length records will result, necessitating extensive "padding" to create a rectangular file. A more efficient strategy in such cases is to produce hierarchical files with the data pertaining to each level of observation appearing in separate records and with variables appearing in more than one type of record to allow for linkage across levels. A number of software packages such as SAS and OSIRIS now exist which can process and analyze such files. In addition, a number of "statistical data base" packages are available, such as SIR, Canada's RAPID, and Mathematical Policy Research's R A MIS, which provide sophisticated capabilities for matching across waves and levels, and which thereby simplify the analyst's data management tasks in working with longitudinal files. Decisions regarding the optimum structure for a longitudinal file also need to take into account the expected size of files. Limits on the number of records many soft ware packages can process may be exceeded by the size of large federal data collections. Consequently, file structure options for facilitating analysis of longitudinal data may be constrained. Sponsors may find it necessary either to forego compatibility with some otherwise useful software packages or to release subsets of their data to provide compatibility with a wider range of software packages. 3. Confidentiality Processing operations and data structures for analysis cannot be designed solely to reduce costs, complexity, or bias. They must also protect respondent privacy as far as possible. This is sometimes not compatible with maximum efficiency. Procedures for protecting confidentiality of paper records and of tape records must be thought through carefully. The problem of maintaining respondent confidentiality is more difficult in longitudinal surveys than in cross-sectional surveys. In cross-sectional research, the confidentiality of a response can be protected by stripping responses of identifiers at an early stage in processing. In longitudinal surveys, response records must be linked to personal identifiers, sometimes for decades, until data collection and analysis are complete. Longitudinal records commonly contain multiple identifiers in order to facilitate tracing and to ensure that records can be matched after each wave, regardless of missing data. Name, address and Social Security number are often augmented with the name and address of family, neighbors, or friends who are to be contacted in tracing respondents who have moved. The large number of identifiers, plus their dispersion across records and across time, makes protecting confidentiality in a longitudinal survey far more difficult than in cross-sectional research. However, most research organizations have learned over the years how to protect paper records. An illustration of one solution to problem is that adopted by N C ES for the NLS-72 and HS & B: Identifiers are stripped from the tape prepared by the contractor before it is turned over to the sponsor agency. These data are maintained by the contractor but may only be used with the explicit approval of the sponsor. The procedure provides a complicated, layered procedure which inhibits any unauthorized access by sponsor, contractor, or public users and provides protection similar to that of a cross- 33 sectional study. This example illustrates a number of the basic safeguards which should be integrated into any longitudinal data collection effort. First, identifiers should be used only to maintain the quality of the data, e.g., for tracing respondents or for matching purposes. Second, only staff performing these functions should be allowed access. Hardcopy media containing identifiable data should be stored in a secured area to limit access. Electronic files should be similarly secured and, when in use, access should be restricted by the operating system to authorized processing personnel only. Third, all privacy- relevant data should be stripped from public use tapes before release. Ideally, the collection agency should separate identifiers during processing and store them on a file separate from the substantive data. Finally, when data Section is complete, all copies of identifiers should be destroyed. Even when such measures are taken, agencies and research organizations must consider the possibility of confidentiality breaks. The quantity of information available about respondents creates the possibility that a series of rare responses can identify respondents. Current research in confidentiality is addressing this problem and should provide useful guidelines for enhanced security measures in the near future. 34 CHAPTER 4 SAMPLE DESIGN AND ESTIMATION There are many issues in the design and estimation strategies for longitudinal surveys that are identical to those for cross- sectional surveys. Some issues, however, such as weighting and compensating for nonresponse become more complicated with a longitudinal survey. Usually the complications arise because of the changing nature of the population, as discussed in Chapter 3. In this chapter, we discuss some of the major design and estimation problems, many of which need more research. A. Defining a Longitudinal Universe Defining the initial study universe for a longitudinal survey is no more complicated than defining the universe for a cross- sectional study, The initial universe is fixed at a specific point in time and is explicitly d fined. Sample units can be selected and the only difficulties are related to the sampling frame itself. Time, however, gradually complicates the problem of defining a longitudinal universe. The study universe usually does not remain constant over the period of the longitudinal survey, as was discussed earlier., The universe of individuals, households, families, or establishments changes over time. If a universe changes slowly along the critical dimensions of the survey, the problem of a longitudinal universe definition may be ignored. However, if changes in the universe over time are not trivial, a static universe definition may not be sufficient. The choice of definition for the longitudinal universe will have a direct effect on data collection and analysis. Judkins et al (1984) describe three methods for defining a longitudinal universe. These ideas are generalizable to any longitudinal study of persons or other units. One method for defining a longitudinal universe is to select a specific time during the course of the study as the point that defines the universe. If the universe is defined at the time of sample selection, it is called a cohort study. Units in the sample are defined at the time of the first interview. At later waves of interviewing, data need be collected only from these units. All inferences and estimates refer only to the universe in existence at the time of the first interview. For example, for the CPI commodities and service sector, the universe is a set of cohort samples with attrition due to deaths. Births are introduced only when an entire cohort is replaced with a new sample. Principal Authors: Daniel Kasprzyk and Lawrence R. Ernst 35 The longitudinal universe may also be defined at a time other than the time of sample selection. Under both scenarios, statistical, operational and methodological problems may arise because the sample was selected at one point in time and the analyses of the study universe reflect a different point in time. It is possible that elements of the study universe at the time of sample selection are no longer part of the longitudinal universe; it is also probable that elements of the longitudinal universe which exist at the time of definition were not in existence at the time the sample was drawn. This creates an operational problem -- whether to collect data from these "entrants" to the longitudinal universe -- and it creates a statistical issue, the development of estimation methods for this universe. For example, in the SIPP universe (the non-institutional population, and members of the military not living in barracks) individuals may leave the universe by moving outside the United States, to an institution, to military barracks, or by dying. At any time during the study period persons may enter the SIPP universe by returning from overseas, institutions, or military barracks, or through birth. A second method of defining a longitudinal universe extends the first method by looking at more than one time point. Several time points are selected, each one defining a universe at that time. Then the entire set of units -defined by these different cross-sectional universes is included in the longitudinal universe. Thus, if a person entered a sample household by being born or returning from overseas sometime after the initial interview, that person would be included in the longitudinal universe. People can be added to the universe, and anyone who is in the universe for any of the time periods should be included in the estimation. For analysis of aggregations of persons, such as households and families, some identification of aggregations at each time point is necessary. Since these aggregations can and do change over time, conceptual, operational and statistical difficulties occur. See, for further discussion of this subject, the section on units of analysis in this chapter. This approach, however laden with difficulties, is the approach which best captures the dynamics of the longitudinal universe. The third method for defining the longitudinal universe is also an extension of the first method, but instead of including all units that enter, leave or stay, this approach includes only those that are common to all the selected time periods. In this approach, one includes in the definition of the longitudinal universe only those elements which were members of all cross- sectional universes. This definition leads to a static universe containing only those elements which do not enter and exit the universe. For example, for households, families, and establishments the universe contains only those units in existence throughout the entire survey period. As discussed above, defining the longitudinal universe can be a problem when it contains units which enter and leave the cross- sectional universe. When the units are establishments or a group of individuals, some decision concerning "rules of continuity" is necessary. The next section briefly reviews models for longitudinal household (family) units of analysis. 36 Units of Analysis Aggregations of persons, such as households and families, present difficult conceptual and practical problems in longitudinal surveys. Over time individuals enter and leave households, and set up new households. It is no longer obvious how a household or family should be defined when time becomes an integral part of the definition. McMillen and Herriot (1985) attempt to reduce the possible definitions to a reasonable number, in order to conduct an empirical evaluation of alternative concepts. They also provide a brief review of the historical basis for a longitudinal definition of households. Much of the discussion below is based on the McMillen and Herriot (1985) paper and one by Kasprzyk and Kalton (1983). Three models have been used to describe household and/or families over time: 1) a static model; 2) an attribute model; and 3) a dynamic _model. The static model of households (or families) classifies households at one point in time, and reflects a cross-sectional perspective. Households and their members are defined at one point and individual characteristics are aggregated over the survey period to provide summary statistics for aggregated analysis units. A critical, but false, assumption has to be made that the household composition remains fixed during the survey period. This definition is not truly longitudinal, because it ignores any changes that each unit may undergo. In this approach weighting the so-called longitudinal sample corresponds to weighting the cross- sectional sample. Note, however, that for CPI or any Laspeyres type index the assumption of fixed composition is what is desired, since the change in composition of sales is being held constant so that price change is the only thing measured. The second model for defining households or families over time is the attribute model. In this model, the individual is the unit of analysis, and household and family characteristics are treated as individual attributes. As a result, the problem of changing units over time is avoided. Results under this approach are expressed as "X% of persons live in households with attribute "Y", rather than "X% of households have attribute Y." Household characteristics are, therefore, attributes of the individual. The attribute model has been used extensively by the Survey Research Center of the University of Michigan for the analysis of data from the University of Michigan's Panel Study of Income Dynamics. Dynamic models, the third type, represent the most difficult conceptual and operational problems. In these models, households (or other groups of individuals) are defined over time, not at one point in time, by a set of rules. These rules, often referred to as continuity rules, identify the initiation, continuation, and termination of the analytic unit. Three examples of continuity rules which have been proposed as dynamic definitions of households are presented in McMillen and Herriott (1985). It is not obvious that one set of rules is better than others; in fact, one concept may be more useful for certain kinds of analyses, but not for others. Little empirical work using alternative dynamic concepts has been published, although Citro (1985) has recent begun an investigation using data from the SIPP development program it remains to be seen whether the dynamic concepts can be properly interpreted and employed to provide useful results for policy application. 37 C. Sample Design For a longitudinal study with a static population, that is, one in which there are no additions over time, the need for longitudinal estimates presents no special difficulties in sample selection. It is only necessary to choose a single sample at the selected point in time, as if a one-time survey were being conducted, and then follow the sample units initially chosen. For such a study there is, in general, no ambiguity about the analytic units, and no additions are permitted to the population. The longitudinal studies of the National Center for Education Statistics (NCES) are examples of this approach. The populations for all the other longitudinal surveys described in this report are dynamic in nature. For these surveys initial sample, selection presents no particular difficulties. It is only necessary that each unit in the population at the time the initial sample is chosen have a known probability of selection. Complications arise, however, because of the additions to the universe, and the care that must be taken in order to follow the sample units of analysis over time. Ideally, provision should be made at the design stage to give additions to the universe a chance of entering the sample, or, failing that, to make adjustments for their absence at the estimation stage. For SIPP, Employment Cost Index (ECI) and items in the CPI for which the Point of Purchase Survey (POPS) is the source, the problem of new units is partially alleviated by employing a rotating panel design. Thus, all additions to the universe will eventually be given a chance of selection, with the length of time between panels as the maximum lag. For the ECI and the CPI, because of the difficulty of identifying births quickly, this is the only provision made for additions at either the design or estimation stage. In general, additions to the universe in these surveys have no chance of affecting the estimates until the selection of the next sample or panel. This again is consistent with the Lespayres concept of a fixed set of items and outlets for Measuring price Change only. In contrast to the ECI and the CPI, the designs of NMCES, NMCUES and SIPP give individuals, families and households that are additions a chance of selection as soon as they enter the universe. At each round of interviewing in these surveys not only is the initial sample interviewed, but so are all individuals currently residing in a household with the original sample people. Individuals joining the universe and moving into a household containing at least one person who was in the universe when the initial sample (or most recent sample) was chosen have a chance of entering the sample. So does any family or household joining the universe that contains at least one individual who was in the universe when the initial sample was chosen. Other individuals, families and households that join the universe have no chance of selection. To cite another example, the CPI rent survey samples building permits in order to identify new units quickly. Care must be taken in the design of longitudinal surveys to assure that the analytic units used in the estimation process for a specific time interval are followed throughout that time interval. In general, this is not a serious problem with surveys such as the ECI and CPI, since the definitions of analytic 38 units for these surveys generally include a fixed location such as an item at a specific outlet. Furthermore, in cohort studies such as the High School Class of 1972 which only makes estimates for individuals selected in the initial sample, there are no difficulties other than the operational problems associated with following people. However, for NMCES, NMCUES, and SIPP there are difficulties associated with following certain sample analytic units. A key reason for these difficulties is that a household or family may continue to exist under most longitudinal definitions even though it no longer contains any individuals who were initially in the sample. Under the procedures established for each of these surveys, the household or family will no longer be followed. Ernst, Hubble, and Judkins (1984) discuss this problem in detail. Any individuals who are additions to the universe and who are to be used in the estimation process should also be followed. Provisions were made to do this in T#ICES and NMCUES but not in SIPP. Tn fact, it has not been decided whether additions will be used at all in SIPP for longitudinal person estimation. Judkins et al (1984) discuss this question. D. Weighting There may be several stages of weighting a sample. One is to reflect the original universe; another is to adjust for nonresponse; a third may be to adjust for sample coverage. Longitudinal surveys have the usual weighting problems of cross- sectional surveys and then at least one additional problem. That is to provide a longitudinal weight to be used during analysis. In this section, we discuss the simple unbiased weighting and adjustment to independent estimates. Nonresponse, since it can be handled either by weighting or imputation, is deferred to the next section. I. Unbiased Weights Typically, the unbiased or base weight for a sample unit is the reciprocal of its probability of selection. In longitudinal surveys, this has generally been the weight assigned to sample units which were in the universe at the time the sample was selected. The development of base weights becomes more complicated-in surveys such as NTICES, WCUES, and SIPP which incorporate additions to the universe in the estimation process, since it is often not practical to compute selection probabilities for such analytic units. For example, NMCES and NMCUES families which are additions to the universe will generally be used in the estimation process if, and only if, at least one member of the new family had been a member of a sample family during the first round of interviews. It would be extremely difficult to determine the first round families for all the members in the new family, and then compute the probability that at least one of the first round families could have been selected. Fortunately, it is not necessary to know the probability of selection in order to obtain base weights which yield unbiased estimators. See Ernst, Hubble and Judkins (1984) for a description of this methodology. 39 Several longitudinal weighting procedures will now be described. Since most of them will be defined in terms of cross- sectional weights, it is useful to define what is meant by the cross-sectional weight. The first round cross sectional weight for a sample household is taken here to be the reciprocal of the probability of selection. For all nonsample households in the universe this weight is zero. For any time period after the first interview it is defined to be the mean of the first round cross- sectional household weights for all persons in the household who were in the universe during the first interview. This type of weighting procedure is currently used in SIPP to produce cross- sectional household and family estimates. There appear to be only two precedents for the weighting of longitudinal households and families -- NMCES and NMCUES. For these surveys each family was assigned its cross-sectional weight at the date the family was first formed (See Whitmore, Cox, and Folsom (1982)). The only other survey where serious consideration is being given to the longitudinal household estimation issue is SIPP. Five alternative methods for obtaining unbiased longitudinal weights are discussed in Ernst, Hubble, and Judkins (1984): 1. The NMCES/NMCUES procedure, assigning each longitudinal household (family) its cross-sectional weight at the date the household (family) was first formed. 2. For any time interval, assigning each longitudinal household (family) its cross-sectional weight at the beginning of the time interval. 3. For any time interval, assigning each longitudinal household weight the average of the first round weights for all persons who remain members of the household throughout the time interval. If there are no such people, the longitudinal household weight is zero. This procedure generally has a slight bias. 4. For any time interval, assigning each longitudinal household the average of its monthly cross-sectional weights. 5. If a longitudinal household is defined as an attribute of a specific individual, such as the householder or principal person, then assigning the longitudinal household the first round weight for that specific individual. The procedures listed apply to the restricted universe of all households in existence throughout the time interval of interest. Some modifications are necessary to apply these procedures to the unrestricted universe of all households in existence for a portion of the time interval of interest. There are advantages and disadvantages to each procedure. They differ, for example, in their need for data from longitudinal households which no longer contain any first round sample persons, or their need to ask retrospective questions in order to determine the appropriate weights. Finally, we briefly discuss longitudinal person estimation. NMCES and NMCES employ longitudinal person estimation that incorporates additions to the universe. Each additional person is associated with a first round family 40 and then assigned the first round weight of that family. For SIPP, it has not been decided whether individuals who are additions to the universe will be used in the person estimation process or, if so, how they would be weighted. One procedure being considered is to assign to persons who join the universe the cross-sectional weight of the household that they are a member of at the time they join the universe. 2. Adjustments to Independent Estimates As a final step in the weighting process for several longitudinal demographic surveys, the population is partitioned into demographic groups and individual weights are adjusted so that the sample estimates of the demographic subpopulations agree with independently derived estimates. In general, this estimation step reduces sampling variability and biases resulting from undercoverage. In the National Longitudinal Surveys (NLS) this adjustment was done for age-race-sex groups for the time of initial sample selection. The adjusted estimates of totals for each group were made to agree with independently derived Bureau of the Census estimates. The Census estimates are obtained by carrying forward the most recent census data to take account of subsequent aging of the population, mortality and migration between the United States and other countries. Since the same sample cases are followed throughout the life of the survey, no subsequent adjustments to independent estimates were made with the following exception: an annual adjustment was made for the cohort of young men (ages 14-29 in 1966) to maintain agreement with the independent estimates. This adjustment corrects population underestimates for men who were not represented in the original sample because they were in the Armed Forces at the time the sample was selected and who subsequently returned to the civilian population. For annual data files from NMCES and NMCUES, family weights were adjusted so that the estimated number of families existing as of March 15 of the interview year agreed with counts from the March Current Population Survey. For each demographic group the adjustment factor used for sample families in existence on March 15 was also applied to families that did not exist on this date. This was done with the assumption that the rate of undercoverage and nonresponse was the sane for all families in a demographic group, irrespective of whether or not the families existed on March 15. Details of this procedure are given in Whitmore, Cox and Folsom (1982). For person estimation in the NMCES' and NMCUES' annual data files, the adjusted family weights for each sample individual's first round family were further adjusted separately for each individual to produce agreement with independently derived age- race-sex estimates. The adjustment factor applied to each sample individual in a group was such that the average of the adjusted- sample estimates of numbers of individuals in each group at four times during the year agreed with the average of the independent estimates at the same four times. Details are provided by Jones (1982). No decision has been made yet on how longitudinal weights for SIPP will be adjusted to agree with independent estimates. One possibility is to use procedures similar to the NMCES and NMCUES procedures. A potential drawback to that approach is that survey estimates will agree with the independent 41 estimates at only one point in time. If agreement is required at other points in a time interval, then adjustment procedures could be modified so that the adjustment factor is not the same for each sample unit of analysis within a demographic group, but instead is also a function of the starting and ending date of that sample unit. This modified approach to adjustment has several disadvantages, such as possibly requiring some weighting factors to be very large. E. Nonresponse In A Panel Survey Nonresponse in longitudinal surveys can be treated from either the cross-sectional or longitudinal perspective. References concerning the treatment of nonresponse in panel surveys are in Kalton, Kasprzyk and Santos (1980), Kalton, Lepkowski and Santos (1981), Kalton and Lepkowski (1983), Marini, Olsen and Rubin (1980), David, Little, and McMillen (1983), Little (1984, 1985). Assuming the data, requirements for the survey mandate a longitudinal analysis, then the longitudinal perspective is clearly the more desirable, since it reflects the survey design. If nonresponse in a longitudinal survey is treated from a cross-sectional perspective, each wave is treated as a separate survey. This has practical advantages in that the release of wave data may occur more quickly than if the separate waves were first linked, and linkage problems resolved. A disadvantage is that records with imputed data will be inconsistent from wave to wave because data processing and estimation procedures are implemented independently from one time to the next. Despite the inconsistencies at the micro-record level, changes in aggregates from the wave to another can be investigated. From a longitudinal perspective, nonresponse in a longitudinal survey is viewed not as nonresponse in a set of unrelated observations but as nonresponse in a set of variables with some logical dependency between two or more points in time. For example, in the CPI missing prices at time t are imputed based on prices obtained at time t-1, and on current average price movement for the item. This view adds considerable information to the data set for the treatment of nonresponse. However, it raises issues concerning the treatment of nonresponse which have not been addressed from the cross-sectional perspective. Longitudinal surveys can be treated as cross-sectional to generate point-in-time estimates. Because of the repeated interviews, however, indicator variables can measure status over time, thus providing better information on patterns of behavior, transitions from one state to another, and the length of time in a particular status. The importance of obtaining this kind of information justifies linking the waves as quickly as possible and treating nonresponse from a longitudinal perspective. The treatment of nonresponse in longitudinal surveys is in many ways no different then in cross-sectional surveys. The above discussion attempts to provide some indication of the similarities and differences in the two approaches. The time dimension adds a level of complexity for all decisions related to the treatment of nonresponse. First is the problem of longitudinal data base construction; efforts need to be made to construct longitudinal files which allow analysts to use the panel aspect of the survey. This includes, at a minimum, ensuring that sample units in one wave are linked to sample units in other waves and that critical data items remain consistent from 42 one interview to the next. Second is the problem of selecting imputation or weighting to handle nonresponse on one or more waves. Third is the problem of timing for release of data. Cross- sectional imputation offers the practical convenience of releasing data as soon as each wave's data are available. However, not all data useful for good imputation are available this way. Imputed values are likely to be better when a combined data set is used. Fourth, in spite of the fact that longitudinal imputation is frequently more effective than cross-sectional imputation, a back- up system is necessary to handle cases where values needed for longitudinal imputation are missing.* 1. Types of Nonresponse Three types of nonresponse occur in surveys: noncoverage, unit nonresponse, and item nonresponse. Noncoverage is the failure to include some units of the survey population in the sampling frame, which means they h ave no chance of appearing in the sample. This may occur, for example, because of incomplete listings at the final stage of selection. Unit nonresponse occurs when no information is collected from the designated sample unit. It can occur because of a refusal, because of a failure to contact the unit (no one at home), or because the unit is unable to cooperate (language difficulties). Item nonresponse occurs when a unit participates in a survey, but does not provide answers to all the questions. It may occur because: 1. the respondent does not know the answer to the questions; 2. the respondent refuses to answer the questions; 3. the interviewer fails to ask or record the answer to the question; 4. the response is rejected during an edit check (e.g because it is inconsistent with another response. The distinction between noncoverage and total and item nonresponse is important because it affects the type of compensation procedure adopted. With noncoverage, the survey can provide no information other than that ___________________________ * The following sections describe imputation and reweighting to handle item and unit nonresponse in connection with improving finite population estimates. Imputation and reweighting strategies are not used, however, when estimating mathematical models of an underlying random mechanism or process. Since such analyses focus on estimation of model parameters, neither assigning values to individual cases nor adjusting to independent estimates is appro- priate. Instead, methods of model estimation are used to account for the missing data under the assumption that the same model applies to all sample cases, even though some cases provide more complete histories than others. Model estimation by the method of maximum likelihood is the most common approach (Tuma and Hannan (1984), chapter 5). The contribution of each sample case to the likelihood function is derived; and if the observations are statistically independent, then the likelihood function is, in most cases, the product of the individual contributions. 43 available on the sample frame. Compensating for noncoverage is usually carried out by using sources external to the survey to produce some form of weighting adjustment, as described in the last session. Noncoverage in a longitudinal survey can be problematic depending on the population which is to be measured. If the population is approximately static, (that is, the amount of change in the population over the life of the panel is not substantial), then the treatment of noncoverage from the longitudinal perspective is not any different than from the cross-sectional perspective. To be precise, however, changes in the survey population should be reflected in later waves of the panel. Often this does not occur because of operational reasons or because such a small proportion of the population is involved. For example, in SIPP the person population does not change greatly over the life of a panel. The principal changes are children who reach adulthood during the life of the panel, deaths, immigrants, emigrants, and persons returning from military barracks and institutions. The survey design captures information about new adults, deaths, and emigrants; however the design does not cover new entrants to the population who live in households which do not include adults eligible for initial sample selection, such as households in which all members are from the following sectors: 1. U.S. citizens returning from abroad; 2. immigrants who move into the U.S. after the first wave of interviewing; and 3. persons who return from military barracks or institutions. The different approaches suggested for treating total and item nonresponse illustrate a concern for the kind and amount of data available for use in compensation procedures. Total nonresponse is typically treated by some form of weighting adjustment, using data available from the sample frame in addition to observations obtained by the interviewer. With item nonresponse, the responses to other survey questions may provide information. To use other responses effectively, item nonresponse is usually treated with some form of imputation (that is, by assigning values for missing responses based on responses from respondents with similar characteristics) rather than with weighting procedures. From the longitudinal perspective, the issue of unit and item nonresponse is not very well defined. From this perspective, a unit's record consists of all information collected on the unit over the life of the panel. This suggests, however, that data missing for one or more waves of a panel can, in fact, be treated as item nonresponse. Nonresponse on one or more waves of the panel may logically,be treated as item nonresponse for all variables that should have been recorded for that wave(s). The distinction between unit and item nonresponse is not obvious, and, often, in the interest of simplicity, a judgment must be made identifying the appropriate level of response necessary to treat a case for item nonresponse rather than unit nonresponse. Ultimately, these issues are best resolved after empirical research on the nature, extent, and patterns of the missing information. This, along with knowledge of the uses of the data, will help determine a strategy for handling nonresponse in a panel survey. 44 2. Total Nonresponse Total nonresponse in a cross-sectional survey means that no one at the household responded for one reason or another. It is often called unit nonresponse in cross-sectional surveys. It is generally handled by weighting adjustments, using data available on the sample frame such as region, city, block, type of area; or available from interviewer observation, such as race of householder. Usually the data available for weighting adjustment is quite limited. In a longitudinal survey the concept of total nonresponse can take on a different meaning, including units which provided information for some, but not all, of the waves of the panel. Thus, viewing the entire longitudinal record as complete response, and responses at one or more waves as partial responses, the definition of total nonresponse can be reconstructed to include units which participate in the survey some part of the time. These units, despite having provided more data than "true" total nonrespondents, can be treated as total nonrespondents. In NMCUES, for example, total nonresponse is defined to include units (individuals), responding in fewer than one-third of the waves they were eligible for interview. (See Cox and Bonham, 1983,, and Cox and Cohen, 1985). 3. Unit Nonresponse For the purpose of this discussion, unit nonresponse will refer to individual or person nonresponse to one or more interviews in a longitudinal survey. The length of a longitudinal survey increases a) the amount of data available for nonresponse adjustments and b) the complexity of nonresponse compensation procedures. Each individual's microdata record does not consist of unrelated, independent observations taken at different points in time, even though the data may be collected in that manner. Many variables reflect the same measure at different points in time. The status of a variable, such as income, at one point is frequently related to its status at a previous point. In a cross- sectional survey only two response categories exist, response and no response. In a longitudinal survey of n-waves there exist 2n possible patterns of response. For example, in a 3 wave study there are eight possible response patterns illustrated as follows (where NR refers to nonresponse and R refers to response): 1. R R R 2. R R NR 3. R NR R 4. NR R R 5. R NR NR 6. NR NR R 7. NR R NR 8. NR NR NR Response patterns are usually classified as forming a "nested" pattern of nonresponses (i.e., variables from early waves of the survey are observed more often than variables from later waves), or as "non-nested". Attrition is a form of nested nonresponse, and estimators for dealing with nested nonresponse have been discussed in the incomplete data literature. (See Anderson (1957).: Rubin (1974), or Marini, Olsen and Rubin (1980).) 45 The three wave study example illustrates the kind of difficulty which can occur when one or more waves of data are missing. Case 1 is an example of total response -- an interview is obtained in each wave of the panel. Cases 2 and 5 illustrate attrition and nested nonresponse. Cases 3 and 4 illustrate non-nested patterns of response (two out of three interviews obtained) and cases 6 and 7 illustrate different non-nested patterns of nonresponse with only one of three interviews obtained. Case 8 is an example of total nonresponse. The difficult decisions about nonresponse which must be made for a three wave study are indicative of problems with surveys of more than three waves. One way of treating unit nonresponse in a panel survey is to define the level of response necessary for a unit to be considered a "responding" unit. All units which exceed this response level would be treated as if they were present in all waves of the panel and their missing interview data regarded as a form of item nonresponse; units with a response level less than the standard would be treated like total nonresponse. Underlying these alternative strategies for handling wave nonresponse is the issue of whether it is better to use imputation or weighting to adjust for wave nonresponse. The weighting procedure simultaneously compensates for all data items of a nonrespondent, but reduces the sample size available for analysis. Weighting adjustment procedures also typically incorporate many fewer control variables than an imputation procedure, although David and Little (1983) suggest a model based approach which increases the number of variables used in the adjustment. Imputation, whether it be cross-sectional or longitudinal, fabricates data. The uninitiated user may not understand this and may attribute greater precision to the estimates than is warranted. Imputation techniques by their nature may fail to retain a covariance structure of the data. However, by identifying critical data items in advance, an imputation procedure can be developed to control key covariances. In practice, a two fold strategy of using both weighting and imputation procedures may often be the best solution (David and Little (1983)). A more detailed discussion of the weighting versus imputation issue for wave nonresponse can be found in Kalton (1985) and in Kalton, Lepkowski and Lin (1985). 4. Item Nonresponse In the previous discussion it was noted that one way of treating unit nonresponse was to consider it a "form of item nonresponse" in a longitudinal record and use imputation techniques. That is, in a longitudinal survey, unit nonresponse can be treated conceptually as item nonresponse. Item nonresponse, because it typically refers to missing data item(s) in an otherwise completed interview, provides a good illustration of the fact that there is nothing theoretically special about longitudinal imputation. As Kalton, Lepkowski, and Santos (1981) have stated, longitudinal imputation for item nonresponse is simply imputation for item nonresponse using auxiliary data from a larger data base, including using longitudinal data elements as well as cross-sectional ones. The principal distinction is the availability of data which are highly correlated with the missing data, usually the same variable measured at different points in time. For example, the imputation in CPI is done from this perspective. 46 Theoretically, a decision concerning cross-sectional versus longitudinal imputation in a longitudinal survey is obvious. The longitudinal approach can certainly do no worse than the cross- section approach. The longitudinal approach can use any of the variables measured on a wave, but in addition it can use variables from other waves. As Kalton and Lepkowski (1983) point out,,if response on an item is highly correlated over time, then the value from a previous interview will be a good predictor of the missing value at the current interview. Two exceptions to this statement should be noted: (1) the predictor variable must be reported at more than one point in time; and (2) the variables used in a cross-sectional imputation system are known to be poor predictors of the missing value and thus would likely be poor predictors in a longitudinal system. The two limitations are important because they suggest that empirical analysis of cross-wave data is necessary before developing a cross- wave imputation system. They also point out that in addition to an imputation system using two or more waves of data a fallback cross- sectional method is often needed to compensate for items which are missing in every wave of the panel. Using cross-wave measures as auxiliary variables in an imputation scheme has special significance when individual changes will be analyzed. Obviously, if imputed values are assigned without conditioning on the previous wave's value, measures of change are very likely to be distorted. In this case, modeling state-to-state transitions becomes extremely important in developing an imputation system. Some methods for longitudinal imputation Are discussed by Kalton and Lepkowski (1983). These methods make use of the stability a variable may have between successive waves of a panel, and they include: 1. direct substitution 2. cross-wave hot deck imputation 3. cross-wave hot deck imputations of change 4. deterministic imputations of change A simulation to compare results using these 4 approaches is also described in the same source. 47 CHAPTER 5 LONGITUDINAL DATA ANALYSIS INTRODUCTION In the past, much longitudinal analysis has been done cross- sectionally, with each wave of a survey analyzed independently. The linked records were often difficult to use and discouraging to analysts. With improved data bases and the use of statistical techniques to analyzes transitions, trends, and change, longitudinal surveys are now showing their distinct analytical advantages. A. Determinants of Longitudinal Analysis Methods Longitudinal analyses study,the change in some unit -- a person, a family, a business and so on -- over time. The focus is not on a description of the current status of the unit. Rather, interest is usually directed at the underlying process that determines any observed change. The methods employed in the analysis of longitudinal surveys depend on four factors: (1) the nature of the process being studied, (2) the type of variables being measured, (3) the analytic objectives, and (4) the method of data collection. These factors taken together determine the kind of mathematical models of the process that are appropriate and estimable. 1. The Nature of the Process Being Studied Many processes can be represented as the flow of a unit between some set of categories (states), such as the change in a person's employment status from employed to unemployed. Such a representation requires an enumeration of the possible categories and a probabilistic description of how movement takes place from one category to another. The flow of the process may be discrete or continuous in time. In a discrete time process, change of state occurs only at a fixed set of points. For example, eligibility for many government benefit programs is a discrete time process. Social Security Administration old age and disability programs, AFDC and many State welfare programs all pay monthly benefits. Eligibility for benefits changes only at discrete points in time, spaced one month apart. Other processes, such as change in health status, changes in price level, death, change in attitudes, or employment, can change state at any point in time and are therefore continuous in time. The process under study may be time stationary or time nonstationary. A process is time stationary if its probabilistic structure and its governing parameters are not themselves changing over time. Processes which are not stationary in time are the most common. The payment of benefits under government programs often undergoes structural changes as the result of legislative and administrative actions. Morbidity is Principal Author: Barry V. Bye 49 continuously affected by advances in medical science, and individual labor force decisions are in part determined by changes in the national economy. 2. The Type of Variables A process may be described by variables which are discrete or continuous; and variables may be either observable or unobservable. Labor force status -- employed, not employed, out of the labor force -- is a discrete observable variable with three mutually exclusive and exhaustive states. Variables such as well being and satisfaction, on the other hand, are often taken to be continuous variables that are usually measured only imperfectly by a set of indicator variables. 3. The Analytic Objectives The analysis of longitudinal data may have several objectives. Descriptive analyses are concerned with the regularities of the process under study. Such analyses often use cross tabulations at two or more points to show gross and net change of the units. There are other descriptive statistics: the number of times that a certain state has been entered since the last measurement, the average length of time spent in a given state, the distribution of probabilities for the next transition and the derivation of calendar period estimates not based on retrospective reports. Hypotheses tests often deal with differences in these statistics among several subpopulations. Researchers interested in causal analyses tend to focus on the underlying structure which governs the process. Mathematical models of the transition from one state to the next become prominent in causal analyses, and the estimation of the parameters becomes the, primary statistical goal. The signs and statistical significance of the estimated parameters are usually interpreted in the context of some higher level generalization or theory. Sometimes longitudinal analyses are designed to project a process into the future. Projection is of primary concern in evaluating changes in government programs or the results of field experiments, particularly When the full effects of the changes have not yet been realized. Projection usually requires a mathematical model of the process. The parameters are then estimated from longitudinal data. 4. The Method of Data Collection Two major strategies are used in gathering longitudinal data. In the first approach a complete history of the process is obtained. This approach is the event history method. Measures include the sequences of states occupied by the individual units, and the times when changes in state occur. The second approach is the multi-wave method. Tn this approach the current status of the units is obtained at two or more points in time, but information is often lost on the duration and sequence of events, and on the possibility of multiple changes between measurements. Information on the duration of events may not even be collected in the multi- wave method. For example, at the initial interview, there may be no data concerning the initial status of the process. At the final interview, there are no data concerning the next state of the process. 50 To summarize, the appropriate data collection strategy for a longitudinal survey is chosen by assessing the nature of the process, the variables, and the research objectives. For example, structural analyses of discrete, observable processes will require event histories (see Tuma & Hannan, 1984). On the other hand unobservable variables such as attitudes can only be measured in a multi-wave panel context, because the best one can do is measure the current status at any fixed point. Multi-wave methods have been used in most large scale surveys even when the focus is on observable processes. The resulting logs of information often severely restricts the analyst's ability to recover the underlying parameters and to discriminate between competing mathematical models. (See Coleman, 1981, and Singer & Spilerman, 1976). B. Analysis Strategies for Longitudinal Data Many of the approaches that are used for the analysis of cross sectional data are applied to longitudinal data as well (see Dunteman and Peng, 1977). There are two ways to use longitudinal data in these analyses. In some cases, variables are measured repeatedly over time. In other cases, longitudinal data are used to establish the temporal sequence of a set of variables. Establishing the correct temporal sequence of a set of variables is important for assessing causal linkages within the set. Categorical data are collected in longitudinal surveys as well as cross sectional surveys. These data can be arrayed in cross tabulations showing the relationship between antecedent and outcome variables. When the status of a particular variable is measured at more than one point in time, cross tabulations can be constructed that describe the change in status of the sample units over time. When longitudinal data are placed in cross tabular form, the statistical techniques used to analyze cross-sectional data may be applied. These contingency table analysis techniques include the general testing of hypotheses about the structure of the table (Landis & Koch), the use of log-linear modeling (Bishop et al, 1975, Dunteman & Peng, 1977, and Hauser, 1978), and the development of certain classes of latent structure models (Clogg, 1979). In longitudinal studies where the outcome variable is continuous, a number of cross-sectional analysis models have been applied. These models fall within the realm of regression analysis and the analyses of variance and covariance. One of these methods, path analysis (see Blaylock, 1970), involves estimating a sequence of regression equations where all endogenous variables, ordered in time, are regressed upon all preceding variables. Path analysis methods have been extended by J™reskog and Sarbom (1979) to cases where the outcome and predictor variables are in principle unobservable (latent) and can only be measured imperfectly by a set of indicator variables. When such variables are measured at several points, J™reskog's methods can be used to determine whether the nature of the construct is changing over time and which predictor variables account for the changes. While cross-sectional analysis is often adequate for describing changes in status and identifying determinants, these methods are usually unsuitable for the analysis of the underlying process that generated the data. Social processes are often better represented by discrete-state, 51 continuous-time stochastic models. The first step in constructing this kind of model is to specify rates of transition between states. A number of researchers (see-Coleman, 1981, Ginsberg, 1972a and 1072b, and Tuma, 1976) have shown that regression analysis -- usually specified in terms of linear or logistic equations with the outcome as the dependent variable -- can supply information about the rates of transition only for a severely limited class of models. In those cases where regression is useful, the process must have run a sufficiently long time that the observed proportions in the outcome categories are not themselves changing over time. Even when cross tabulations show status change between two (or more) points, model identification can be problematic. The data are often equally compatible with more than one model. Because of the problems encountered when applying cross- sectional analysis methods to longitudinal data, current analysis strategies focus directly on the rates of transition from one state to the next. In the biological sciences these investigations fall under the rubric of survival analysis (see Elandt-Johnson & Johnson, 1980). In the social sciences, general theories of stochastic processes are applied (see Bartholemew, 1973 and Tuma & Hannan, 1984). While these new methods permit a richness of analysis not possible with cross-sectional methods, they can have a significant impact on sample design and data collection issues. Many of the techniques require event history data rather than multiwave panel data. In those cases where only longitudinal data are obtainable, observations at unequally spaced survey dates are often required. Many of the new approaches utilize non-parametric methods or rely on maximum likelihood techniques for the estimation of model parameters. Applying these techniques properly to the complex sample designs found in longitudinal surveys remains a largely unexplored area in statistical research. C. Examples of Longitudinal Analysis Because there is such a wide variety of methods, the flavor of longitudinal analysis is best captured through examples. Two Social Security Administration projects will be discussed; the first is the Social Security Administration Retirement History Study (RHS). In this project some examples of the more familiar cross-sectional approaches are presented. The second is the Social Security Disability Program Work Incentive Experiments (WIE) which provide examples of some current analytic strategies. 1. Social Security Administration Retirement History Study The Social Security Administration's Retirement History Study (RHS) is a multiwave survey designed to address a number of policy questions relating to the causes and consequences of retirement. Among these questions are: Why do individuals retire before age 65? How well does income in retirement replace preretirement earnings? What happens to the standard of living after retirement? The original sample of 12,549 persons was a multi-stage area probability sample selected from members of households in 19 retired rotation groups from the Current Population Survey. The sample was nationally representative of persons 58 through 63 years old in 1969. Initial interviews were conducted in the spring of 1969 and then in alternate years through 1979. Data collected during this period provide 52 detailed information on work history, sources of income, expenditures, health, and attitudes toward and expectations for retirement. Results from the RHS have been reported in a number of Social Security Administration research reports (listed in SSA publication #73-11700). The data have also been analyzed by researchers outside the government via public use tapes. An interesting variety of cross-sectional analytic methods suitable for multi-wave data have been used with the RHS data. One example is a two-wave descriptive analysis of the change in income between 1968 and 1972 using simple turnover tables (Fox, 1976). The second example is a three-wave structural equation model of income satisfaction (Campbell and Mutran, 1982). a. Analysis of income change Fox examined income level and change between 1968 and 1972 by constructing simple turnover tables. One of these tables (table 1 on page 59 and 60) classified respondents or couples by their income position in 1968 and 1972. The table shows the marginal distributions each year and the joint probability of change separately for married couples, unmarried men, and unmarried women, crossed by work status in 1968 and 1972. The table indicates some increase in income over time for persons either employed or not employed in both years, and, as expected, a substantial decrease in income for persons employed in 1968 but not employed in 1972. Among this latter group, Fox (1976) noted that income loss for unmarried men appeared greater than for unmarried women. Fox's findings are examples of general questions that can be answered by the analysis of turnover tables. 1. Are income changes between the two points different for different subpopulations? 2. Are there differences in marginal income distributions between sub-populations at a given time? A number of authors (Bishop et al, 1975, Hauser, 1978, Landis & Koch, n.d., and Singer, 1983) have shown that hypotheses involving marginal distributions and attribute-by-time interactions can be specified and tested using existing methods for the analysis of categorical data. For example, testing whether income changes vary by subpopulation is the same as testing for a 3 (or higher) way interaction between income level at time one, income level at time two, and subpopulation characteristics. The weighted least squares approach (Landis et al., 1976) would be an appropriate methodological approach for testing this kind of hypothesis, especially for complex sample designs. Given a consistent estimate of the sampling covariance matrix for the table cells, appropriate test statistics for a wide variety of hypotheses can be computed. Fox's analysis also illustrates two additional methodological issues. We are informed in the technical note to his report that only 63 percent of the sample respondents had usable income data in both 1968 and 1972 due to the 'very conservative editing" of income response. In both years, respondents had to give usable answers to about 20 different income components (twice that, if married). An inadequate response to 53 any one of these components was enough to cause a nonresponse for the entire set. Three questions immediately arise. What is the effect of response error for individual income items on the analysis of the turnover tables? Would imputing missing income items affect the analysis? How did analyzing only the partial data set affect the analysis? Response errors are likely to result in-an overestimate of change in income class, because some of the observed change is due to reporting error rather than to real change over time. Generally, in order to separate real change from classification error, an observation at a third point is required. This third observation could be a reinterview, taken soon after one of the regular waves, designed to measure reporting error directly. However, under certain modeling assumptions, three widely spaced observations can also provide estimates of real change and classification error (see Bye & Schecter 1980 and 1983). A second problem resulting from classification error arises when attempting to measure differences among various subpopulations. There may not be real change at all; the analyses may simply reflect differences in the propensity for response error among the subpopulations, leading to incorrect interpretations. The effect of imputation on the analysis of turnover tables will depend on the specific imputation scheme. If, for an individual, responses from other waves are used to impute missing values for a particular wave, real change may be understated. If, on the other hand, the amputations are carried out separately for each wave, real change will most likely be overstated. Particular care must also be given to substantive interpretations, when the same attributes are used both for imputation and for substantive analysis. Analyzing partial data sets requires an assumption that the nonresponents are like the respondents. Usually no studies have been carried out to support that. To the extent that nonrespondents are different, as they frequently are in health and income studies, the data set is biased and the interpretation is inadequate. b. Stability of income satisfaction Campbell and Mutran (1982) present an analysis of the stability of income satisfaction over time using data from three waves of the RHS -- 1969, 1971 and 1973. They assume that income satisfaction is an unobserved continuous variable measured imperfectly by two indicator variables. The two indicators are "satisfaction with the way one is living" (SAT), and "ability to get along on income" (GET). Figure C (page 61) presents a path diagram for one of the models estimated by Campbell & Mutran (1982). (The estimated covariance matrix of the observed variables is shown in Table 2, page 62.) Campbell and Mutran posit that income satisfaction is in turn a function of health status, (an unobserved variable with three indicators), of actual income level in 1969, and of the number of times in the hospital in 1970. The authors note that this path model is significantly underspecified but provides an interesting example of the use of LISREL methodology (J™reskog & S™rbom (1978) and (1979)). 54 LISREL unites factor analysis and structural equation modeling for a wide variety of recursive and nonrecursive models with and without measurement errors. (see J™reskog & S™rbom, 1976). The LISREL approach assumes that both measurement and structured equations are linear in the unknown parameters and that all variables are normally distributed. 2. Social Security Administration Disability Program Work Incentive Experiments Under the provisions of the Disability Insurance Amendments of 1980, the Secretary of Health and Human Services was directed to develop and carry out experiments and demonstration projects designed to encourage disabled beneficiaries to return to work and leave the benefit rolls. The primary objective of the experiments is to save trust fund monies. The bill itself contains several examples of the kind of change in the post entitlement program that Congress had-in mind. These include changing entitlement provisions for Medicare benefits, lengthening the trial work period, and modifying treatment of post entitlement earnings, such as the application of a benefit offset based on earnings. Congress imposed important constraints on the experiments: they must be of sufficient scope and size that results are generalizable to the future operation of the disability program, and no beneficiary may be disadvantaged by the experiments as compared to the existing law. Eight treatment groups and a control group have been proposed (see (SSA, 1982, for details). Each treatment group represents an alternative to the current post entitlement program representing either some change in the law or administrative practice (or both). A two stage stratified cluster sample of 31,000 newly awarded beneficiaries was planned for the experiments. The sample would be representative of all beneficiaries under age 60 at the time of award. The sample beneficiaries would be assigned at random to one of the nine experimental groups in such a way that the full experimental design is replicated in each geographic cluster. The total sample size in each treatment group would be 3,000, and there are to be 7,000 in the control group. Under the current disability program, a beneficiary who returns to work despite continuing severe impairment is granted a 24 month period in which to make a work attempt while remaining on the benefit rolls (the first 12 months with full benefits, the second 12 months with benefits in suspense.) workers are expected to need 1 or 2 years to return to work and 2 or 3 years to complete the trial work period and be terminated from the rolls. Thus an observation period of 4 to 5 years is required to track beneficiaries through the shortest of the post-entitlement out- comes. Observed short-run labor force response will provide some information about the effects of the treatments, but trust fund savings will be significant only if employment is sustained in some groups. Thus, sustained work is the key labor force parameter in the evaluation of the work incentive experiments. 55 At the same time, the analysis of short run labor force outcomes, commencing about 2 years after the experiments begin, is a necessary first step in gauging trust fund effects. The data available for the short run analysis will consist of a voluntary baseline questionnaire (face-to-face interview) covering socioeconomic and demographic background items plus a series of mandatory quarterly reports (mail with telephone followup) showing the beginning and end of work attempts and monthly earnings for each month of the quarter. The response to the quarterly reports is mandatory because work reports and monthly earnings are required for administrative purposes. a. Short run longitudinal analysis of return to work The first step in the analysis of return to work will compare the proportion of beneficiaries who have made a work attempt among treatment and control groups. However, short run differences could be misleading if the full effect of the treatment has not been realized. Consider the hypothetical outcome in figures A and B below. Click HERE for graphic. 56 In Figure A the difference between treatment and control is small for the first two years, but becomes large afterwards. In Figure B, short run difference appears large at first but then becomes smaller. Clearly, change over time in the proportion of beneficiaries who return to work is most important in determining the experimental effect. The rate of change of this proportion over time for beneficiaries who have not yet returned to work is called the hazard rate function (or hazard function). A short run evaluation of return to work will focus on differences in rates of return to work among treatment and control groups. Using individual observations of the time of return to work, the first analysis of return to work will be to estimate and graph the cumulative hazards of return to work for treatment and control groups and test the difference between the hazards. If there are differences between treatment and control groups, the graphical displays of the cumulative hazards should provide a useful guide. These can then be used to project long run differences in the probability of return to work among the experimental groups. Introduction of covariates from the baseline questionnaire might also improve the accuracy of these predictions (see Hennessey, 1982). b. Structural Models of Duration -- Testing a Sociological Theory It has been suggested that the longer a beneficiary remains on the disability rolls, the less likely he or she is to return to work. The reason given is that the beneficiary makes the necessary social and psychological adjustments to continue in the role of a disabled person. The fact that population rates of return to work for disabled beneficiaries decline over time is often taken as evidence supporting this theory. However, one can show that population heterogeneity can account for an apparent decline in population transition rates over time, even if the individual rates are constant or increasing. (See Heckman & Singer, 1982, for example.) Therefore any assessment of the apparent negative duration dependence must account for population heterogeneity. One way to examine this issue is to specify and estimate a structural model for the hazard function for return to work. The parameters are usually estimated by maximum likelihood methods, incorporating the likelihoods for sample cases moving from nonwork to work at time t, and for sample cases which haven't yet moved by time t (which, in this case, is the end of the observation period). c. Estimating long run trust fund effects The Disability Amendments mandate that the primary evaluation of the experiments be in terms of trust fund effects. In general, the cost to the trust funds of an individual beneficiary is the sum of the expected costs to the Disability and Medicare funds between initial entitlement and the termination of benefits or the attainment Of age 65. The cost to the disability trust fund can be further broken down into the sum of the cash benefit payments plus the cost of vocational rehabilitation (if applicable) 57 minus the payback of FICA contributions (if the beneficiary returns to work) during this period. The estimation of long run effects requires the projection over time of the probability of receiving cash benefits for disability, the expected amount of those benefits, the probability of working, and the expected earnings level. An analysis plan for the WIE is being developed which is based on a continuous-time stochastic model. The state space for the process admits four possibilities: E.1 : Recovered E.2 : Deceased E.3 : Nonworking Beneficiary E.4 : Working Beneficiary At the time benefits are awarded the beneficiary is assumed to be in state E.3. The beneficiary can switch between states E.3 and E.4 until he or she reaches state E.1 or E.2 (which are taken to be absorbing states) or reaches age 65 (and is automatically converted to the old age program.) A semi-Markov model is proposed to link the various work and non-work episodes over time. This model assumes that each work and non-work period is independent of prior work history (but might depend on age and other exogenous factors which can be incorporated into the hazard functions.) Although it is unlikely that this sort of independence does in fact exist, the short observation period effectively precludes the ability to detect the real dependencies. In conclusion, once the hazard functions are estimated separately for each experimental group, future work and benefit status histories will be simulated. These histories together with estimates of earnings and benefit levels will allow the estimation of long run trust fund costs for each experimental group. Using four years of administrative data, Hennessey (1982) found that semi-Markov models of work and benefit status for male beneficiaries can accurately predict the histories three years hence. His,results provide encouragement for this overall analysis strategy. 58 Click HERE for graphic. 59 Click HERE for graphic. 60 Click HERE for graphic. DEFINITIONS OF VARIABLES A. Satisfaction with Income is an unmeasured construct with three indicators: 1. SAT69, SAT71, SAT73 Are you satisfied with the way you Does are living? 4 = More than satisfied 3 = Satisfied 2 = Less than satisfied 1 = Very unsatisfied 2. GET 69, GET71, GET73 Ability to get along on income 4 = Always have money left over 3 = Have enough with a little left over sometimes 2 = Have just enough, no more 1 = Can't make ends meet B. Health is an unmeasured construct with three indicators: 1. LIM69, LIM71, LIM73 Does health limit the kind of work you do? 2 = No 1 = Yes 2. OUT69, OUT71, OUT73 Are you able to leave the house without help? 3 = No limitation 2 = Yes, though health limit work 1 = No C. Number of times in hospital (HOS7 is measured with one indicator D. 1969 household income (INC69) is single indicator of log income from all sources from all sources from Campbell and Mutran, 1992. Reprinted with permission 61 Click HERE for graphic. 62 CHAPTER 6 SUMMARY AND CONCLUSIONS In developing the working paper on longitudinal surveys, the subcommittee found that few of the issues were simple. For each question that was raised there were multiple and sometimes contradictory conclusions encountered in the literature, or in the experience of the subcommittee members. This complicated the task of drawing conclusions about when or how to use longitudinal surveys; what was is clear is that anyone considering a longitudinal survey should remember four general points. These points could apply equally well either to longitudinal or to cross- sectional surveys, but certain aspects are especially important in longitudinal surveys. First, research goals should be clearly stated and alternative kinds of data collection should be evaluated. Cross-sectional research is not automatically less expensive, and certain research goals cannot be attained with one-time surveys. The evidence seems to indicate that longitudinal surveys are not intrinsically more costly than one-time surveys of comparable scope. In many cases, one longitudinal survey will be more efficient than a series of one-time surveys. However, cost considerations may dictate that neither a longitudinal survey nor a series of one-time surveys could be carried out. Compromises are often made on frequency of interview or sample size to permit some longitudinal data collection. - For certain research goals, such as identifying the frequency or duration of change, or the causes of change (as in longitudinal surveys of labor force status), only a longitudinal survey will work. For topics that are difficult for respondents to recall, such as attitudes or detailed behavior (as in longitudinal surveys of retirement, or health treatments, or household income), a prospective longitudinal survey is the best choice. - All other things being equal, a longitudinal survey achieves a given level of precision for measures of change with a somewhat smaller sample than is possible in a series of one- time surveys. In addition, the cost of maintaining contact with a longitudinal sample may be no higher than the cost of selecting and contacting a one-time sample. - Timing of results plays an important part in the decision to select a longitudinal survey. If early results are needed, then a longitudinal survey is not appropriate. If early waves of a longitudinal survey can be analyzed quickly and provide useful information, then some of the timing problem is dissipated. If the research needs can only be met by a longitudinal survey and those waiting for results clearly understand the timing, longitudinal surveys are clearly superior. 63 Second, once the decision has been made to conduct a longitudinal survey, the subcommittee recommends that a greater emphasis be placed on the early formulation of clear and specific analysis objectives as the next step in research planning. The failure to formulate detailed analysis early enough explains some of the disappointments that some organizations have experienced with longitudinal surveys. - As the simplest example, when research objectives are not clearly stated or understood, the longitudinal nature of the data has not always been fully exploited in analysis. - Many of the operational features of longitudinal surveys should only be selected after the development of clear and specific plans for analysis. Even such seemingly unrelated factors at the interval between interviews may be determined by analysis plans. For example, discrim- ination between some simple stochastic models is ruled out if data collection intervals are constant. Other examples are given in Singer and Spilerman's study of longitudinal analysis (1976). - A clear statement of specific research goals, including analysis plans, reduces the likelihood that a project will require unanticipated funding extensions or auxiliary sponsors for completion. Comprehensive planning ensures that a survey will appeal to a wide constituency, and reflect the research goals of an adequate sponsorship base. - Fully developed research objectives make it less likely that a need for different -- or additional -- data will become apparent part way through the survey. Third, longitudinal surveys can easily incorporate features that facilitate the evaluation of internal data quality, and that compare the effectiveness or cost of alternative methods. Repeated data collection makes this possible in ways that are beyond the scope of a one-time survey. - Any longitudinal survey that varies data collection mode while maintaining a constant questionnaire can be a vehicle for studying the impact of mode of interview on response. Evaluations have indicated that the NLS obtained comparable results by using personal or telephone interviews after the first interview, for example. - Data from longitudinal surveys can be used to understand the impact of nonresponse on the representativeness of a sample. The characteristics of nonrespondents in a later wave can be studied through what is known about them from the first interview, or from later follow-ups in which they do respond. In the NLS, each extension of the survey has been preceded by evaluations of the impact of attrition through comparisons with population controls developed in the first wave of interviews. - The effect of continued participation on response can be evaluated each time new persons are brought into the sample or interviewed for the first time. The original HS+B survey program, for example, provided for an additional sample; a group from the original sample to be interviewed only in the later waves, specifically to evaluate panel effects. - Alternative methods for simulating complete response from incomplete data (such as imputing from other cases, or from what was reported in another interview, or by increasing the weight of completed interviews) can be evaluated using a longitudinal file. The final comparisons have to wait until all the waves of a longi- tudinal survey are completed, but preliminary results can be used in earlier waves, and a variety of procedures can be compared at the end of the program in order to select the most effective method. - Data from longitudinal questionnaires can and should be compared to the results from comparable questions asked of similar respondents in one-time surveys. The results of NLS labor force questions were constantly evaluated against cross-sectional labor force surveys. This provides ongoing information on sampling error, and on the impact of questionnaire design on response. - Data from a longitudinal survey, from related administrative records,and from comparable surveys of one-time samples can be compared to estimate the impact of recall periods, or the interval between interviews, or the effect of bounding interviews. The Income Survey Development Program demonstrated the importance of just such an exhaustive testing program which accompanied planning for SIPP. - The costs of alternative data collection strategies should be recorded, along with the operational considerations and the impact on data quality. This information will be invaluable when the most efficient methods must be chosen for other surveys. - The costs and effects of alternative data processing strategies should be recorded to allow comparisons, such as the costs and benefits of matching longitudinal records through characteristics or through unique identification codes for sample persons and households. Early tests such as these led to the development of the case-linking strategy selected for SIPP. These and many other comparisons are possible with longitudinal surveys, because so many materials, respondents and operations vary throughout the course of the survey. With minimal additional efforts toward record-keeping and control, most longitudinal operations can provide important data for evaluating internal data quality and to guide future survey designers. 65 Fourth there are many measurement error problems that exist with any kind of survey, some of which are exacerbated by a longitudinal design. So far, the research on many of these methodological problems has not been definitive, so choices are made based on cost and intuition. There is a rich field for investigations and those seeking to do longitudinal surveys should strive to include some methodological elements. Some of this kind of research has been carried out, as described above, but more is needed. - Time-in-sample bias permeates every survey that requires repeated interviewing. It is not limited to one particular kind of variable or one mode of data collection. As a result there is a systematic bias in the data that shows up when data are compared by the number of interviews a respondent has had. No one knows which set of data are more accurate, those from earlier or those from later interviews. People make judgments based on little or no data, and the topic needs careful investigation. - Response errors have the effect of exaggerating change. People do forget and change their minds, and different household respondents give different answers to the same questions. The length of time between interviews also influences answers. More work needs to be done to separate real from spurious change. - Attrition is a serious problem in longitudinal surveys. Many longitudinal surveys are able to keep 90 to 95 percent of their respondents on each interviewing wave, but even low nonresponse mounts over time. Although compensation strategies look promising, it is troublesome to realize that for some variables, a quarter to one-half of the data are not given by respondents. - There has been little research on the best length of time to allow between interviews. Decisions are based mainly on cost, yet we know that the longer the interval, the less that is reported, and the more that is reported in the wrong time periods. Work needs to continue on this aspect. - It is known that the questions on a survey are not processed one by one by respondents. The presence of questions on other topics affects responses to questions on variables of interest. This happens whether the additional questions precede or follow the main questions. However, the tendency is to keep adding new topics. We may be causing a deterioration of data quality by doing this. Longitudinal surveys are increasingly being used as the basis for policy decisions by the Federal government. In our review, we have become convinced that for some research goals there is no alternative to longitudinal data collection. However, before agencies make the decision to conduct a longitudinal survey, they should carefully consider the important operational, management, and statistical problems associated with them. 66 CASE STUDY 1 SURVEY OF INCOME AND PROGRAM PARTICIPATION I. Purpose of the survey In October 1983, the Bureau of the Census conducted the first interviews of the Survey of Income and Program Participation (SIPP). The SIPP is a nationally representative household survey intended to provide detailed information on all sources of cash and noncash income, eligibility and participation in various government transfer programs, disability, labor force status, assets and liabilities, pension coverage, taxes, and many other items. Data from the survey will provide a multiyear perspective on changes in income, and their relationship to participation in government programs, changes in household composition, and so forth. In general, the SIPP data system is designed to measure elements of the federal tax and transfer system in a comprehensive data base. SIPP began in response to the recognition that the principal source of information on the distribution of household and personal income in the United States -- the March Income Supplement of the Current Population Survey (CPS) had limitations which could only be rectified by making substantial changes in the survey instrument and procedures. For example, the CPS does not provide monthly income, monthly household composition or detailed asset information. These deficiencies became especially serious when the scope of policy analyses was broadened during the 1960's and early 1970's as public assistance programs were expanded and reorganized. Model-builders were forced to make many assumptions and impute intrayear data using CPS data to carry out their activities. In this environment, with analysts requiring more detailed data and improved measures of cash and noncash income, the Income Survey Development Program (ISDP) was established. The purpose of the ISDP, authorized in 1975, was to design and prepare for a major new,survey, the Survey of Income and Program Participation (SIPP). The ISDP developed methods intended to overcome the three principal shortcomings of the CPS for analyses of income: 1) the under reporting of property income and other irregular sources of income; 2) the underreporting and misclassifi- cation of participation in major income security programs and other types of information that people generally find difficult to report accurately (for example, monthly detail on income earned during the year); and 3) the lack of information necessary to analyze program participation and eligibility (annual income estimates were available, but eligibility for most Federal programs is based on a monthly accounting period). Four experimental field tests were conducted to examine different concepts, procedures, questionnaires, and recall periods. Two of the tests were restricted to a small number of geographic sites, the other two were nationwide. The largest test, conducted in 1979, was also the most complex. Although used primarily for methodological purposes, the nationally representative sample of 8,200 households was sufficiently large to provide reliable national estimates of many characteristics. More detailed discussions of the ISDP and its activities are provided in Ycas and Lininger (1981) and David (1983). 67 Because the ISDP was the predecessor to SIPP, it is not surprising that many characteristics of the ISDP are reflected in the SIPP design, including many elements of the survey's design, content, and questionnaire format. II. Sponsors The ISDP development effort was directed by the Office of the Assistant Secretary for Planning and Evaluation in the Department of Health and Human Services and was carried out jointly with the Bureau of the Census, which assisted in the planning and carried out the field work, and the Social Security Administration (SSA), which administers the major cash income security programs. In late 1981 virtually all funding for ISDP research and planning for the ongoing SIPP program was deleted from the budget of the Social Security Administration. The loss of funding for fiscal year 1981 brought all work on the new survey to a halt. Then in fiscal year 1983, money for the initiation of the new survey was allotted in the budget of the Bureau of the Census. In planning the content, procedures, and products of the SIPP, the Census Bureau works closely with a SIPP Interagency Advisory Committee, established and chaired by the Office of Management and Budget (OMB). The committee consists of individuals representing the following departments and agencies: the Departments of Labor, Education, Defense, Commerce, Agriculture, Health and Human Services, Treasury, Housing and Urban Development, and Justice; Energy Information Administration; National Science Foundation; Council of Economic Advisors; Congressional Budget Office; Bureau of Labor Statistics; Bureau of Economic Analysis; Veterans Administration; Internal Revenue Service; and the Office of Management and Budget. III. Sample Design SIPP started in October 1983 as an ongoing survey program of the Bureau of the Census with one sample panel of approximately 21,000 households in 174 primary sample units (PSU's) 1/ selected to represent the noninstitutional population of the United States. The sample design is self-weighting; that is, each unit selected in the sample has the same probability of selection. In February 1985 and every February thereafter, a new, slightly smaller panel of 15,000 households is introduced. This design allows cross-sectional estimates to be produced from the combined sample from both panels. The overlapping panel design enhances the estimates of change, particularly year-to-year change. Since portions of the sample are the same from one year to the next, year-to-year change estimates can be based in part on a direct comparison across 2 years for the same group of households. To facilitate field operations, the sample is divided into four approximately equal subsamples, called rotation groups; one rotation group is interviewed in a given month. Thus, one cycle or "wave" of interviewing takes 4 consecutive months. This design creates manageable interviewing and processing workloads each month instead of one large workload every 4 months; however, it results in each rotation group using a different reference period. 68 Data collection operations are managed through the Census Bureau's 12 permanent regional offices. Interviewers assigned to these offices conduct one personal visit interview with each sampled household every 4 months. At the time of the interviewer's visit, each person 15 years old or older who is present is asked to provide information about himself/herself; a proxy respondent is asked to provide information for those who are not available. The average length of the interview is about 30 minutes. Telephone interviewing is permitted only to obtain missing information or to interview persons who will not or cannot participate otherwise. An important design feature of SIPP is that all persons in a sampled household at the time of the first interview remain in the sample even if they move to a new address. For cost and operational reasons, personal-visit interviews are only conducted at new addresses that are in or within 100 miles of a SIPP primary sampling unit (persons moving outside that limit are contacted by telephone if possible). After the first interview, the SIPP sample is a person-based sample, consisting of all individuals who were living in the sample unit at the time of the first interview -- these people are labelled original sample persons". Individuals aged 15 and over who subsequently share living quarters with the original sample people are also interviewed in order to provide the overall economic context of the original sample persons. Changes in household composition caused by persons who join or leave the household after the first interview are also recorded. These individuals are interviewed as long as they reside with an original sample person. More information about these procedures can be found in Jean and McArthur (1984). IV. Survey Design and Content Each person in the SIPP sample is interviewed once every 4 months for 2 2/3 years to produce sufficient data for longitudinal analyses while providing a relatively short recall period for reporting monthly income. The reference period for the principal survey items is the 4 months preceding the interview. For example, in October, the reference period is June through September; when the household is interviewed again in February, it is October through January. This interviewing plan will result in eight interviews per household. An important design feature of SIPP is the assignment of an individual identification number. Each sample person is assigned a unique fourteen-digit identification (ID) number at the time he/she enters the sample; an additional two-digits code is assigned if the person moves to a new address. A master list of identification numbers is used by the regional offices to monitor the status of interviewing each month after Wave 1. The regional offices keep track of each number on the list representing all the persons assigned for interview in a month; each must be accounted for with a completed questionnaire or a reason for noninterview. The list is updated regularly to account for persons who are added or deleted from the sample. The ID helps to link information about an individual across time; it identifies which household each person is a member of at any point in the panel. Through the ID system, data can be linked from all persons ever associated with a given household throughout the 2 2/3-year duration of a panel. 69 The survey consists of three major components: (1) the control card, (2) the core data, and (3) topical data. The control card is used to obtain and maintain information on the basic Characteristics associated with households and all household members and to record information for operational control purposes. These data include the age, race, ethnic origin, sex, marital status, and educational level of each member of the household, as well as information on the housing unit and the relationship of the householder to other members. A household respondent provides this information, which is updated at each interview. The control card is also used to keep track of when and why persons enter and leave the household, thereby providing enough information to compose monthly household and family groups. There is also space to record information that will improve the interviewer's ability to follow persons who move during the survey. In addition, after each visit, data on employment, income, and other information are transcribed from the core questionnaire to the control card so the data can be used in the next interview as a reference for the interviewer and thus shorten succeeding interviews. A questionnaire is filled for each household member who is 15 years or older. The questionnaire consists of a "core" of labor force and income questions asked during each interview and a set of topical modules which are scheduled during the life of the panel. The core labor force and income questions are designed to measure the economic situation of persons in the United States. These questions expand the data currently available on the distribution of cash and noncash income and are repeated at each interviewing wave. SIPP core data build an income profile of each person aged 15 and over in a sample household. The profile is developed by determining the labor force participation status of each person in the sample and asking specific questions about the types of income received, including transfer payments and noncash benefits from various programs for each month of the reference period. A few questions on private health insurance coverage are also included in the core. Persons employed at anytime during the 4-month reference period are asked to report on jobs held or businesses owned, number of hours and weeks worked, hourly rate of pay, amount of earnings received, and weeks without a job or business in addition to questions about labor force activity and the earnings from a job, self-employment, or farm, the core includes questions related to nearly 50 other types of income as well as the ownership of assets which produce income. The SIPP has been designed to provide a broader context for analysis by adding series of questions on a variety of topics not covered in the core section. These questions are labelled "topical modules" and are assigned to particular interviewing waves of the survey. If more than one observation is needed, a topical module may be repeated in a later wave. The survey design allows for the inclusion of these special modules because less time is required in later waves to update the core information collected in the first interview. The subjects covered do not require repeated measurement at each interview and, therefore, may use a reference period longer than the period used for the core information. Examples of topical modules include health and disability, work history, assets and liabilities, pension plan 70 coverage, tax-related information, marital history, fertility, migration, household relationships, child care arrangements, and pension plan coverage. For more information about the SIPP design refer to Nelson, McMillen, and Kasprzyk (1984). V. Survey Response Rates The first SIPP interviews were conducted in October 1983. At this time, cumulative household noninterview rates are available for the first six waves of SIPP, that is, through August 1985. Sample loss through the sixth wave has been 19 percent, of which 15 percent was due to refusals and other situations in which the interviewer was unable to make contact with the household, and 4 percent was due to movers that the interviewer was not able to contact again. Survey nonresponse rates for persons are discussed in McArthur and Short (1985). In this work they characterize the population that is leaving the sample; comparing these persons' characteristics to those of persons continuing to be interviewed in the survey. At the end of the third wave of interviewing, combining all reasons for noninterview -- including refusals, institutionalization, move s to unknown addresses, persons who were temporarily absent, and so on -- 10.5 percent of all persons who were interviewed during the first wave had left the sample. There is some indication that those noninterviewed persons are different from persons who continue to be interviewed. Noninterviews are more likely to be renters rather than homeowners, to live in large urban areas, and to have reported their marital status to be single or separated. Coder and Feldman (1984) found that imputation for a selected group of items was quite small. In this analysis item nonresponse rates on labor force, income recipiency, and income amounts are examined. They also discussed the impact of self or proxy respondents on nonresponse rates. Lamas and McNeil (1984) discussed the quality of data measuring household wealth in the survey. The nonresponse rate was low for all asset types (1.4 percent) for all persons asked about asset ownership. They found that nonreponse rates varied by type of asset -- lowest for rental property and highest for certificates of deposit -- and by age and education levels of the respondents -- higher nonresponse for older persons and higher nonresponse with greater educational attainment. McMillen and Kasprzyk (1985) used counts of amputations made for each person as the measure of item response rates. The maximum number of amputations that could have been made for an individual was 83. They found that in the first two waves of interviews, 86 percent of the persons had no imputation at all. In Waves 1 and 2, respectively, 87 percent and 92 percent of the cases with some imputation had no more than 3 items imputed. More work planned to study nonresponse is discussed in the research section. VI. Survey Evaluation Work SIPP evaluation work is in an early stage; the Census Bureau and other users of the SIPP data are developing appropriate methods of evaluation. For example, research is being carried on for three types of nonresponse -- unit nonresponse defined as nonresponse to all waves of the survey, wave nonresponse defined as nonresponse to a particular wave interview, and item nonresponse defined as nonresponse to a particular item -- and their patterns of occurrence. 71 Another area of useful evaluation work combines survey data with administrative record data. The SIPP was developed as an integrated data system in order to use combined information sources to validate and supplement information collected in the survey. An internal Census Bureau committee is assessing the potential uses of administrative data linkages and identifying content and availability of administrative record systems for use in demonstration studies. One record linkage project which is currently under development will match SIPP survey data for individuals to their administrative records at the state level. Various federal record systems which may also be brought into this project are also being investigated. At this time both the number of states and the number of records systems involved is limited. A discussion of the quality of the income data collected as of each wave of the SIPP is contained in an appendix to each SIPP quarterly report (U.S. Bureau of the Census). The appendix supplies information on the nonresponse rates for selected income questions, the average amounts of income reported in the survey or assigned in the imputation of missing responses, and the extent to which the survey figures underestimate numbers of income recipients and amounts of income received. For example, in the report for the third quarter of 1983 (P70, no.1) nonresponse rates range from a low of about 3 percent for Aid to Families with Dependent Children (AFDC) and food stamp allotments, to about 13 percent for self- employment income. The report states that survey underestimates of income recipients ranged from about 21 percent for AFDC to about 1 percent for Social Security recipients, and the survey estimate of persons receiving state unemployment compensation payments was about 103 percent of the independent estimate. The underreporting for AFDC is-related to misclassification of this income type as other types of public assistance or welfare. Evaluation of the ISDP is relevant to work in the SIPP. For example because of its design, SIPP has a potential for missing and inconsistent data problems from wave to wave. One area of current research is the phenomenon of significant income changes and program turnover occurring between waves more often than within waves. Some analysis of this phenomenon using data from the 1979 ISDP Panel is presented in Moore and Kasprzyk (1984). Continuing this area of research using data from SIPP, Burkhead and Coder (1985) looked at gross changes in income recipiency from month to month over a period of one year, the first three waves of SIPP. Their examination indicated that change in recipiency statuses was significantly higher for the months that spanned successive interviewing reference periods, that is between the last reference month for one interview and the first from the next interview. Vaughan, Whiteman, and Lininger (1984) also discussed the quality of income and program data in the ISDP. They discuss numbers of income recipients and program participants, and amounts of income and benefits in comparison to independent sources and the CPS. Other relevant studies are: Ferber and Frankel (1981), studying the reliability of the net worth data in the 1979 panel of the ISDP; Feldman, Nelson and Coder (19801, evaluating the quality of wage and salary income reporting in the 1978 ISDP; and U.S. Bureau of the Census (1982). VII. Survey Data Products and Research Activities A number of publications and public-use data files are being generated from the information collected in SIPP. Both publications and data files are 72 identified by whether they are cross-sectional or longitudinal. Two types of cross-sectional reports are planned by the Census Bureau: 1) a set of quarterly reports that focus on core information; and 2) periodic or onetime reports that use the detailed data from the topical modules. The quarterly cross-sectional reports show average monthly labor force activities, income, and program participation statistics. The first quarterly report was issued in fall 1984 (U.S. Bureau of the Census, 1984) and contains data referring to the Third Quarter of 1983. The report covering the Fourth Quarter of 1984 was released in November 1985. The periodic and single-time reports will use the detailed data from the topical modules (for example, disability and earnings, health insurance coverage and household net worth). These reports may also use a combination of the core and topical module data. Plans for longitudinal data reports are under discussion, but they are expected to concentrate on data that can be used to examine trends and changes over time. This may include analyses of the dynamic aspects of the labor force or the effect of changes in household composition on economic status and program participation. Examples of reports under consideration in this series are: economic profile reports, presenting yearly aggregates of monthly data on individuals; comparative profile reports, presenting annual comparisons of the economic activity of individuals; transition reports, providing changes in income and program participation status between two points in time: longitudinal family and unrelated individual reports, presenting the characteristics of longitudinal family units defined in SIPP (see McMillen and Herriot.(1984) for more information on this topic); and special event reports, providing data preceding and/or following a particular event, such as marriage, divorce, separation, the birth of a child, a return to school, a move to a new address, or a job change. SIPP cross-sectional data files are issued on a wave-by-wave basis. 2/ Each file includes person, family, and household information collected in the survey wave. Virtually all data obtained on the core questionnaire are included on the files; certain summary income recodes are also included. Data that might disclose the identity of a person are excluded or recoded in accordance with standard Census Bureau confidentiality restrictions. Wave files are edited, imputed, and weighted in a manner consistent with their use for cross-sectional analysis. A unique identification number is included to allow users to merge two or more SIPP files. However, since the processing of wave files is independent, wave- to-wave data inconsistencies will occur and the user must be prepared to resolve them. Data files containing topical module information will be released together with the core data that were collected at the same time. Identifiers will be included on the file to allow linkage to other topical module files. Plans for producing public-use files designed for longitudinal analysis are now under discussion. The first longitudinal file For SIPP will be a research file containing twelve months of core income data; this is essentially the first three SIPP interviews. 73 A SIPP working paper series has been established as a mechanism to provide timely and widespread access to information developed as part of the SIPP. Papers in the series will cover a broad range of topics including: procedural information on the collection and processing of data; survey methodology research; and preliminary substantive results, such as the measurement of household composition change over time The 1984 and 1985 meetings of the American Statistical Association were used to bring the research community up-to-date on a variety of SIPP-related research issues. A wide range of topics, both methodological and substantive, were covered in sessions organized under the auspices of the Social Statistics and Survey Research Methods Sections. Papers presented in 1984 have been compiled by Kasprzyk and Frankel (1985) and the 1985 papers have been compiled by Frankel (1985). A number of other research projects are underway at the Census Bureau and at independent research centers such as the Survey Research Center/University of Michigan. These projects are vital to the understanding, use, and future development of the SIPP. This work includes studies of longitudinal imputation and weighting strategies; characteristics of persons who become nonrespondents: composite estimation; potential for use of data base management systems; linkage of administrative records and economic data from other census files to SIPP results, see Sater (1985). The American Statistical Association (ASA)-National Science Foundation (NSF)- Census research fellow program has been expanded reidentify explicitly SIPP-related research activities. _____________________________ 1/ A primary sampling unit consists of a county or a group of contiguous counties. 2/ For information about the SIPP public use files, please call the Data Users Services Division at (301) 763-4100 and ask for the "Data Developments" for SIPP. 74 CASE STUDY 2 CONSUMER PRICE INDEX I. Purpose The Consumer Price Index (CPI) is a measure of price change for a fixed quantity and quality of goods and services purchased by consumers. The CPI is used most widely as an index of price change. During periods of price increases, it is an index of inflation and services as an indicator to measure the effectiveness of Government economic policy. The CPI is used also as a deflator of other economic series, that is, to adjust other series for price changes and to translate these series into inflation-free dollars. These series include retail sales, hourly and weekly earnings, and some personal consumption expenditures used to calculate the gross national product (GNP) - all important indicators of economic performance. A third major use of the CPI is to adjust income payments. More than 8.5 million workers are covered by collective bargaining contracts which provide for increases in wage rates based on increases in the CPI. In addition to workers whose wages or pensions are adjusted according to changes in the -- PI, the index now affects the income of more than 50 million persons, largely as a result of statutory action: Almost 31 million social security beneficiaries, about 2« million retired military and Federal Civil Service employees and survivors, and about 20 million food stamp recipients. Changes in the CPI also affect the 25 million children who eat lunch at school. Under the National School Lunch Act and the Child Nutrition Act, national average payments for those lunches and breakfasts are adjusted semi-annually by the Secretary of Agriculture on the basis of the change in the CPI series, "Food away from home". Also, the official poverty threshold estimate, which is the basis of eligibility for many health and welfare programs of Federal, state and local governments, is updated periodically to keep in step with the CPI. Under the Comprehensive Employment and Training Act of 1973, the "low income" criterion for distribution of revenue-sharing funds, is kept current through adjustments based on the index. In addition, the Economic Recovery Tax Act of 1981 provides for adjustments to the income tax structure based on the change in the CPI in order to prevent inflation-induced tax rate increases. These adjustments, designed to offset the phenomenon called "bracket creep", are to be calculated initially in 1984 and reflected in the 1985 tax schedules. II. Sponsors The CPI is collected, analyzed and published monthly by the Bureau of Labor Statistics. The Census Bureau under contract to BLS collects two surveys, the expenditure survey and the Point of Purchase survey which are used to construct sampling frames for selecting the item and outlet sample for the CPI. 75 III. Sample Design - General The most recent major revision of the CPI was completed in 1978. This revision introduced probability sampling procedures at all levels of sampling including within outlet selection of items. It incorporated new expenditure weights from the 1972-73 Consumer Expenditure Survey, new retail outlet samples from the 1974 Point of Purchases Survey, and population data from the 1970 census. It also introduced a second index, the more broadly based CPI for All Urban Consumers (CPI-U) , which took into account the buying patterns of professional and salaried workers, part-time workers, the self-employed, the unemployed, and retired people, in addition to wage earners and clerical workers. The two indexes differ chiefly in the weighting used. In January 1983, the BLS changed the way in which homeownership costs are measured. A rental equivalence method replaced the asset price approach to homeownership costs for the CPI-U. In January 1985 the same change will be made in the more narrowly defined index constructed for the Wage earners and clerical workers (CPI-W). The central purpose of the change was to separate shelter costs and the investment component of homeownership so that the index would reflect only the cost of shelter services provided by owner-occupied homes. Several key concepts indicate the nature of the Consumer Price Index and guide the way in which it is calculated. 1. Prices and Living Costs. The CPI is based on the prices of food, clothing, shelter and fuels, transportation fares, medical services, and the other goods and services that people buy for day- to-day living. It is constructed in accord with statistical methods that make it representative of the prices of all goods and services purchased by consumers in urban areas of the United States. Price change is measured by repricing essentially the same market basket of goods and services on monthly or bimonthly time intervals and comparing aggregate costs with the costs of the same market basket in a selected base period. The longitudinal aspect of the survey is the month to month linkage of the sample of item/outlet specifications (quotes) and their price, size and quantity for the given quote. 2. Weights and relative importance. The weight of an item in the index is derived from a survey of consumers which provides data about the dollar amount spent for consumer items during the survey year. In a fixed weight index, such as the CPI, the implicit quantity of any item used in calculating the index remains the same f rom month to month (for example, the number of gallons of gasoline) . This should not be taken to mean that the relative importance, of gasoline in the average consumer's budget remains the same. Relative importances change over time because they reflect the effect of price change on expenditures. Items whose prices rise faster than the average become relatively more important. 3. Sampling. Since it is impossible to obtain prices for all expenditures by all consumers, the CPI is constructed from a set of samples not all of which are longitudinal in nature: a. A sample of areas selected from all U.S. urban areas. 76 b. A sample of families within each sample area for expenditures of consumers, this sample need not be longitudinal, but linkage of records from a series of interviews was used. c. A sample of outlets from which these families purchase goods and services. A household survey which is used to identify and construct the sampling frame of outlets is not longitudinal, however, the sample of outlets selected from this frame is longitudinal., d. A sample of items for the goods and services purchased by these families. This is the primary longitudinal component of the CPI. It is from these samples that weights are developed and data are obtained for the monthly calculation of the index. Specifics for each sample or sampling stage are described as follows: A. CPI Area Design Pricing for the CPI is conducted in 87 sample geographic areas. Eighty five strata were defined by combining similar PSU's according to the following 1970 Census characteristics: 1. region, population size, SMSA versus non-SMSA 2. percent population increase from 1960 to 1970 3. major industry 4. percent nonwhite 5. percent urban This area design resulted in 29 strata with one pricing area per stratum and 58 non-selfrepresenting strata. Twelve publication areas consisting of three city-sizes (non-selfrepresenting SMSA's of over 388,000 population, SMSA's less than 388,000 population, and non-SMSA urban areas) crossed by four Census regions were defined along with the 29 local areas to provide estimated indexes for all urban areas of the country. Each of the twelve region, city size publication areas contained four, six or eight strata. In addition special supplementation was made to support publication for Denver. B. Expenditure Survey Sample Design In 1972-73 two household surveys, a Diary and an Interview Survey were conducted by the Census Bureau for BLS to collect expenditure information for consumer units. The sampling unit for these surveys was a housing unit. The reporting unit was a consumer unit which was defined to be (1) a group of two or more persons, usually living together, who pool their income and draw from a common fund for their major items of expense, or (2) a person living alone or sharing a household with others, or living as a roomer in a private home, lodging house, or hotel, but who is financially independent-that is, income and expenditures not pooled with other residents. Never married children living with parents always were considered members of the consumer unit. The eligible population included the civilian noninstitutional population of the United States as well as that portion of doctors' and nurses' quarters of general hospitals. Armed forces personnel living outside military installations were included in the coverage while armed forces personnel living on post were excluded. Also excluded from eligibility were persons living in college dormitories, fraternity or sorority houses, prisons, monasteries, aboard ships, or in other quarters containing five or more unrelated persons. 77 The first component was a Diary Survey completed by respondents for two consecutive one week periods. The objective of the Diary Survey was to obtain expenditure data on small frequently purchased items which are normally difficult to recall. These items include expenditures for food and beverages, natural gas and electricity, gasoline, housekeeping supplies, non-prescription drugs, medical supplies, and personal care products and services. Consumer units were asked to list all expenses during the survey period. Data on income and family characteristics also were collected. The sample of housing units was balanced across areas and time of year. The records of the two consecutive one week periods for each consumer unit were linked to create two week levels of expenditure. The second component of the CE, called the Interview Survey, was a panel survey in which each consumer unit in the sample was interviewed every three months over a fifteen month period. This survey was designed to collect information on major items of expense as well as on income and family characteristics. Items reported on the interview survey included expenditures for the following: housing, household equipment, house furnishings, vehicles, subscriptions, insurance, educational expenses, clothing, repair and maintenance of property, utilities, fuels, vehicle operating expenses and expenses for out of town trips. The final interview in the fifth quarter provided the regularly recorded expenses plus information on homeownership costs, work experience, changes in assets and liabilities, estimates of consumer unit income and other selected financial information. The quarter records for each consumer unit were linked to form annual records for each consumer unit. Only consumer units responding in at least the fifth interview were used to form these "linked" records of annual expenditures for estimation. The samples of consumer units for the CE were selected as follows. For both the diary and interview survey the nation was stratified into 216 geographic strata using stratification variables defined for the Current Population Survey of the Census Bureau. Thirty of these areas were designated as selfrepresenting. Half of the housing units in each self-representing area were covered in the first survey year and half in the second survey year. The 186 equal sized non-self- representing areas were divided into two 93-area groups. One sample area from each of the 93 groups was in sample in each of the two survey years. Each sampling area was randomly selected proportional to population from each of the 186 strata. 1. Interview Survey The universe for sample selection was the 1970 Census 20% sample data file. A sample of 12,613 housing units was designated for the 1972 Interview Survey component, and 13,014 housing units for the 1973 Interview Survey. For the first year 11.1 percent were vacant, nonexistent or ineligible and the refusal rate was 10.3 percent of the designated sample. Interviews were completed in 9914 units. For the second year 12.9 percent was vacant, nonexis- tent and ineligible with a refusal rate of 9 percent. Interviews were completed in 10158 units. At the time of selection, housing units for the Interview Survey within a PSU were distributed by month within the quarter to allow for data collection throughout each quarter. Each sample unit was visited once each quarter, at 78 approximately the same time in the quarter, and each consumer unit within the household was interviewed. Data from previous quarters were available for the interviewer to use in bounding expenditure reporting. Bounding is an interviewing technique which unduplicates expenditures reported in the previous interview from the current interview. The type of expenditures reported during each interview varied since the recall periods varied from three months to one year. Housing, major equipment, automobiles, subscriptions and insurance were annual recall items. A semi- annual recall period was used for minor equipment, house furnishings, renting and leasing of vehicles, and education. The following sections were covered each quarter: repair, alterations, and maintenance of owned property; utilities, fuel, and household help; clothing and household textiles; equipment repairs; vehicle operating expenses; and out-of-town trips. Interviewing was conducted with any person available in the consumer unit; no attempt was made to interview all persons in the consumer unit, that is proxy responses within a consumer unit were used. Proxy responses for persons away at school was the source for some of the college members of a consumer unit. 2. Diary Survey Again the universe for sample selection was the 1970 Census 20% sample data file. A sample of housing units was selected from this Census file for each year of the diary survey. Approximately 14,590 housing units were designated and 12,661 eligible for the 1972 Diary component, and about 15,210 designated and 12,999 eligible for the 1973 Diary component. These numbers included an augmented sample of households which were to be visited during the four week period preceding the end of the year holidays. Each housing unit was visited twice, once at the end of each week of the two week survey period. For the first year the eligible response rate was 80.1% and 89.9% for the second year. IV. CPI Survey Design and Content The primary longitudinal samples for the CPI is the sample of item/outlet specifications and their respective prices, which are obtained every month. BLS collects prices for the Food, Commodities and Services, Rent, and Property Tax components of the CPI. These prices are collected monthly or bimonthly in all 87 areas. Each one of these components has a separate survey with its own sample design. Data used for the Mortgage Interest and House Prices components of the CPI are not collected by the Bureau but are obtained from outside sources such as FHA and FHLBB. The Point of Purchase Survey (POPS) is the source of the outlet sampling frames for about 60% of the CPI items by expenditure weight. The items not covered by the POPS are grouped together under the heading non-POPS and include rent, property tax, mortgage interest, house prices, utilities, transportation, insurance, and several miscellaneous categories. These sample designs are not described here except rent. 1. Point of Purchase Household Survey - Frame Source In the spring-summer of 1974 a household survey, the Point of Purchase Survey, was conducted by the Census Bureau for BLS to provide the sampling 79 frame of outlets for food and most commodities and services to be priced in the CPI and to provide demographic data for classification of the households reporting an expenditure for an outlet. The survey was conducted in the 85 PSU's defined for the CPI. The commodities and services for which sampling frames were developed in each PSU included food, apparel, drugs, personnel care items, household furnishings and housekeeping supplies, beverages, most medical services, sports equipment, gasoline and automobiles, and automotive parts and services. Expenditures, name, and location of the place of purchase were collected for approximately 100 relatively broad categories of expenditures with reference periods of one week to two years depending on the expected frequency of reporting. To control the expected number of responses received from a household and minimize respondent burden two groups of categories were defined; one set given to 1/4 of the sample households and the second set given to 3/4 of the sample households. The combination of sample size of the households asked a category and the reference period for a given POPS category was designed to generate approximately 6 to 12 not necessarily unique outlets reported for a given PSU/POPS category. For POPS the national sample size was 23,000 designated housing units. Since separate frames of outlets were required for individual CPI pricing area (PSU's), the sample is not self- weighting across PSU's, but within a PSU, the households are selected with a uniform probability. 2. CPI Outlet Sampling Procedures When a sample ELI was selected a specific POPS category was identified for outlet selection. In self-representing areas, sample households were divided into two independent groups by the first stage order of selection. This defined two frames of outlets for outlet selection to support variance estimation. The following approach was used for outlet selection for frames developed from the POPS and CPOPS Survey. A systematic selection of outlets reported for a given POPS category for the W population was made where the measure of size for each outlet was proportional to the average daily expenditure reported for the outlet by all consumer units in the W population. Before January 1982, the outlets for the U population were then selected using a conditional probability technique to maximize the overlap between outlets. The sample outlets for the U population were then selected by a repeat of the systematic selection using the new measures of size. After January 1982 the collection of prices for the W population was discontinued. The sample outlets are now selected systematically with probability proportional to average daily expenditure of the U population. All outlets reported by CPOPS sample families in any sample area are eligible for pricing. However, BLS restricts pricing of outlets to be within a 25 mile radius of a given sample PSU unless 10 or more designated items are identified in some clustered area beyond the mileage limitation. If this is the case, there is no mileage limitation and all items in the clustered area are priced. The non-POPS categories were excluded from the POPS either because existing sampling frames were adequate, or it was felt the POPS would not yield an adequate sampling frame. 80 Each non-POPS commodities and services item has its own sample design. For each item, the frame consisted of all outlets providing the commodity or service in each sample area. A measure of size was associated with each outlet on the sampling frame. Ideally, this measure of size was the amount of revenue generated by the outlet by providing the item to the !CPI U population in the sample area. Whenever revenue was not available, an alternate measure of size, such as, employment, number of customers, or quantity of sales was substituted. Since no measures of size could be determined strictly for the w population, a single sample of outlets and quotes was selected for estimating the index for each population. All samples were selected using the systematic sampling technique with probability proportional to the measure of size available. a. CPI Sample Items The basic CPI item structure is an follows: The seven, major groups (food, housing, apparel, transportation, medical care, entertainment and personal care) are broken into 68 expenditure classes (ECIS) (such as auto repair). Within each EC, expenditures are grouped into one or more item strata (such as body work, power plant repair, component repair, and maintenance and service). There are a total of 265 item strata within each item strata, one or more substrata, called Entry Level Items (ELI's) are defined. There are a total of 382 ELI's. ELI's are the ultimate sampling units for items as selected in the BLS Central Office. They are used in the field by the data collectors as their initial level of item definition within an outlet. An ELI is assigned to one and only one POPS or Non-POPS outlet category. Four regional market basket universes were tabulated into the item strata structure from the Diary and Interview surveys to reflect regional differences within each of the four regions (Northeast, North Central, South, and West) eight independent samples of ELI's were selected for each item stratum. Thus, eight samples of ELI's were selected for each region and for each population-thirty-two sample selections nationally for each population. Each CPI PSU was assigned one or two of the eight item samples from the corresponding region for pricing. Self-representing published areas were assigned two independent item samples and each non- self-representing area was assigned one item sample. These independent item samples were designed to accommodate variance estimation for the CPI. A given item sample for all item strata assigned to a given PSU is called a half-sample. The sample of ELI's and appropriate POPS categories are merged to create specific outlet/item samples. b. Within Outlet Selection for Specific items For each ELI, whether in a POPS or Non-POPS category, the selection of a specific store item by a data collector is performed using multi-stage probability selection techniques with measures of size proportional to percentages of dollar sales usually provided by the respondent for the outlet. To perform this operation, the data collector is provided with a checklist that includes all the descriptive characteristics which are believed to identify the items of the ELI and determine or explain price differences for all items defined within the ELI. In addition, the data collector is given the definition of the ELI, suggested stages of groupings of items to aid in 81 quickly selecting a specific store item and a series of worksheets on which to define the categories of items, post the probabilities and identify the next category within which to select the specific store item by use of the random number table on the worksheet. In developing this procedure, it was necessary to provide the data collector with several alternative methods for defining the categories and obtaining the percentage of dollar sales or approximations to those sales. The procedures developed to obtain the proportion of sales were: a. Obtaining the proportions directly from a respondent. b. Ranking the categories by importance of sales and then obtaining the proportions directly or using preassigned proportions. c. Using shelf space to estimate the proportions where applicable. d. Using equal probability if all else fails. To define the categories, direct responses from the respondent as to what he sells or an inventory technique was used. The procedures make possible an objective probability sampling of items throughout the CPI. They also allow broad definitions of ELI's so that the same tight specification need not be priced everywhere. The wide variety of specific items greatly reduces the within item component of variance, reduces the correlation of price movement between areas, and allows a substantial reduction in the number of quotes required to obtain the same precision as the pre- 1978 index. A second important benefit from the broader ELI's, along with the POPS categories, is a significantly higher probability of finding a priceable item within the definition of the ELI within the sample outlet. Procedure a) was used approximately 60% of the time, procedure b) was used about 30% of the time, procedure c) about 7% and procedure d) the remainder. Once the sample of items in the sample PSU's are identified, the price for the specification which define the items within the sample outlets are priced on a monthly or bimonthly basis. This continues for a minimum of a 5 year period and is the basis for measuring price change for the CPI. This time series for each individual specification is the longitudinal element of the CPI. C. Sample Maintenance Since 1977, the Bureau has sponsored a Continuing Point of Purchase Survey (CPOPS) also conducted by the Census Bureau. This survey is aimed at producing current data on outlets. The CPOPS has been expanded from the original 100 categories of expenditures included in the POPS to 134 categories of which 102 categories are asked from each of two equal size panels. This survey is conducted each year in one fifth of the 87 PSU's on a rotating basis. From the results of this household survey, new samples of outlets and item specifications are rotated into the CPI data collection to replace the old sample of outlets and items priced for the CPI in a given area. d. Response Rates A sample of 24,278 outlets were designated from the original POPS survey for CPI pricing. The out-of-scope response rate was 12.6 percent. There were 82 1,649 with non-responses resulting from no contact, refusals, or temporary agencies. This non-response rate for designated sample units was 6.8 percent. Thus the response rate was 93%. Each year one-fifth of the sample areas have all of the outlets reselected for repricing. Approximately 7300 outlets are selected of which 11.8% are out of scope and the response rate has been 95% from those outlets which have sample items available to price. An annual attrition rate for outlets has been 3.3%. In addition for the outlets which remain in sample, the average annual item substitution rate for items within outlets has been 6.2%. Substitution occurs because an item selected for sample is modified or no longer available and the field representative obtains a description and price for an item most similar to the original item selected from the outlet. V. Rent Survey A. Sample selection The current CPI rent index is based on a sample of approximately 23,000 rental units, allocated among the 87 PSU's. The units were selected from two universes, a stratified multistage, systematic, self-weighting area sample of housing units built before 1970 and a continuously updated sample of newly constructed units. The Bureau of the Census provides the sample of new construction units from building permits. Approximately 2,000 units have been obtained from this source as of 1982. Using an area segment sampling approach, 19,000 rental units were selected from 6,422 area segments. There has been an attrition of about 2,000 units due to conversions to owner housing. This sample has been augmented with approximately 1,500 new segments and 4,000 rental units to minimally support the rental equivalency concept of homeownership. This augmentation followed a process similar to the original area segment sampling approach. B. Data Collection In order to collect the monthly information necessary to calculate the rent index, the sample is divided into six panels of approximately 3,800 units each. The units in each panel are visited twice a year on a six month cycle. The information collected includes the rents paid for the current month and the previous month, information on extra charges and reductions, a description of the unit, and the facilities included in the rent. The latter questions are used to make quality adjustments to the calculated rents in order to assure that the rent change measured is for a set of units of a consistent quality. Data collection is by personal visit or telephone to tenants or property managers. For the CPI Rent sample the response rate for occupied in scope units is 88 percent. VI. Scope and Calculation A. Index and Non-Rent Estimation Prices used in calculating the index are collected in 87 urban areas across the country from about 24,000 retail establishments. 83 Prices of food, fuels, and a few other items are obtained every month in all 87 locations. Prices of most other commodities and services are collected every month in the five largest urban areas and every other month in other areas. Prices of most goods and services are obtained by personal visits. Some repricing for selected easily identified commodities are obtained by telephone and a mail questionnaire is used to obtain electricity rates. In calculating the index, price changes for the various item strata in together with urban area weights which represent each market basket are averaged sent their importance in the spending of the appropriate population group. Local data are then combined to obtain a U.S. average. Separate indexes are also compiled by size of city, by region of the country, for cross-classifications of regions and population-size classes, and for 29 local areas. The estimation for monthly item strata level price relatives (R.t,t-1) is the ratio of two long term relatives for time t and t-1. R.t,O R.t,t-1 = ------ R.t-1,O Each long term relative is calculated as a weighted sum of individual items price relatives m W.i P.ti R.t,O = ä --- ---- iE1 M P.Oi where R.t,o is the long term estimate of price change for a set of items representing the item strata P.ti is the price at time t for item i P.Oi is the price at time 0, the base period, for item i W.i is an estimate of expenditures for the ELI contained in the item strata for which the items are a sample M is the number of eligible sample prices in the ELI The index each month is a weighted average of the price relatives divided by a base expenditure (C.O). The weights (C.t-1,i) of the index are estimates of expenditure for each item stratum which reflect buying patterns of a given reference period and all price change up to the previous month: m I.t,O = C.t-1,i R.t,t-1,i i=1 ------------------ C.O B. Rent Estimation Estimates of the monthly rent price relatives for each market basket are calculated using special cost weights and 1- and 6- month, estimates of rates of change. 84 Let S.1 be the set of units interviewed in time t in a market basket which has rent values for time t and t-1, and S.6 be this set of units interviewed in time t in a market basket which has rent values for times t and t-6. The rents for the ith unit in a market basket for the given time period are represented by r.iT where T-t, t-1, or t-6. The l- and 6-months rates of change, R.t,t-1 and R.t,t-6, are calculated by: ä r.it W.i ä r.it W.i ieS.1 and R.t,t-1 ieS.6 R.t,t-1 = ----------------- ----------------- ä r.it-1 W.i ä r.it-6 W.i ieS.1 ieS.6 where W.i reflects the probability of selection adjusted for nonresponse. Using R.t,t-1 and R.t,t-6' a composite estimate is made of a current month's cost weight CW.t for the market basket: CW.t = P R.T,T-1 CW.t-1 + (1 - P)R.t,t-6 CW.t-6' where P = .65. The value of P was based on simulations of weighted averages of 1- and 6-month rent relatives designed to minimize variances. A final 1-month estimate of rent price change for the particular market basket is CW.t R.t,t-1 = ----- CW.t-1 C. Rental Equivalency In January 1983, BLS will begin measuring the housing component of the CPI-U using the rental equivalency method which assumes the cost of homeownership is the amount which would be paid to rent an equivalent home. Rental equivalency will be measured using a sample of rental units with new weights assigned to each rental unit which reflect the number of homeowner units in the universe for which the rental unit is equivalent. The rent component of the CPI will continue to be measured in the usual way. After 1986 rental equivalency will be measured using a sample of owned units. Rent change will be determined for these units by matching the owned units to equivalent rental units based upon unit and neighborhood characteristics. Using estimated owners rents, monthly change for rental equivalency will be calculated in a fashion similar to that used to calculate the current rent index. VII. Data Products and Analysis The monthly CPI is first published in a news release during the fourth week following the month in which the data are collected. (The index for January is published in late February.) The release includes a narrative summary and analysis of major price changes, short tables showing seasonally adjusted and unadjusted percentage changes in major expenditure categories, 85 and several detailed tables. Summary tables are also published in the Monthly Labor Review the following month; shortly thereafter, a great deal of additional information appears in the monthly CPI Detailed Report. Seasonally adjusted data are presented in addition to unadjusted data because they are preferred for analyzing general price trends in the economy. They eliminate the effect of changes that normally occur at the same time and in about the same magnitude every year, such as price movements resulting from changing climatic conditions, production cycles, model changeovers, holidays, and sales. Seasonal factors used in computing the seasonally adjusted indexes are derived by the X-11 Variant of the Census Method II Seasonal Adjustment Program and are reevaluated annually. The data collected is item descriptive data plus the price, size and quantity of the item being priced. Longitudinal analysis is specifically related to determination of degree of price change and trend for a given commodity sector and explaining the reasons for the change for both the short and long term by examining the micro data and ancillary information for the locale and the nation. In addition, studies are conducted to assess the impact of government policy changes or changing economic conditions on the index. The techniques used are regression, distribution analysis and simulation. VI. Limitations of the Index The CPI is not an exact measure of price change. It is subject to sampling errors which may cause it to deviate somewhat from the results which would be obtained if actual records of all purchases by consumers could be used to compile the index. These estimating or sampling errors are limitations on the precise accuracy of the index rather than mistakes in the index calculation. The accuracy could be increased by using much larger samples, but the cost is prohibitive. Furthermore, the index is believed to be sufficiently accurate for most of the practical uses made of it. Another kind of error occurs because people who give information do not always report accurately. The Bureau makes every effort to keep these errors to a minimum, obtaining prices wherever possible by personal observation, and corrects errors whenever they are discovered subsequently. Precautions are taken to guard against errors in pricing, which would affect the index most seriously. The field representatives who collect the price data and the commodity specialists and clerks who process them are well trained to watch for unusual deviations in prices which might be due to errors in reporting. The CPI represents the average movement of prices for two specified populations but not the change in prices paid by any one family or small group of families. The index is not directly applicable to nonurban workers and others not included in the samples. The index measures only the change in prices and none of the other factors which affect family living expenses, such as changes in the size of the family or changes in buying patterns. Nor does it reflect consumption, such as fringe benefits. Area indexes do not measure differences in the level of prices among cities; they only measure the average change in prices for each area since the base period. 86 Although the CPI has been called a cost-of-living index and used at times as if it were one, there are important conceptual differences between a price index and a cost-of- living index. A true cost-of-living index would take into account not only price changes but also changes in the market basket as consumers adjust their purchases to changes in the relative prices of what they buy. Thus,, during a period of rising prices, a cost-of-living index might rise more slowly than a price index if consumers substitute cheaper items for more expensive ones, or generally reduce expenditures on higher priced items in their budget. However, an index such as the CPI' does not directly reflect such consumer behavior, since the quality and the implicit quantity weights of the items represented in the CPI remain constant. The index indicates what it would cost to maintain the same level of living, not what consumers actually spend on their living costs. What consumers actually spend may reflect a decision to accept a lower standard of living in order to keep living costs from rising. There are other differences between the two types of index. For example, the CPI includes only the cost of sales and excise taxes that are included in the purchase price o f goods and services, but not income taxes, whereas a cost-of-living index would include both sales and income taxes. 87 EMPLOYMENT COST INDEX CASE STUDY I. Purpose The Employment Cost Index (ECI) measures change in total employee compensation and has been designed as a principal Federal economic indicator by the Office of Management and Budget. The ECI is used in monitoring the effects of monetary and fiscal policies by enabling analysts and policymakers to assess the impact of labor cost changes on the economy, both in the aggregate and by sector. The limitations of the index must be kept in mind. Because the ECI is an index, it only measures change in employee compensation; the index is not a measure of the total cost of labor. Not all labor cost (e.g., training expenses, retroactive pay, etc.) fall under the ECI definition of compensation. II. Sponsors The Bureau of Labor Statistics developed the ECI in 1975 to provide a comprehensive measure of employee compensation. The initial design was started in the early 70's by the Office of Wages and Industrial Relations and the Office of Survey Design of BLS. All data collection and data processing is provided by Bureau staff. III. Sample Design A. Private Sector Sample Design A principle concern of the ECI sample design is to provide an ongoing sample that in some sense represents an outgoing current universe. ECI accomplishes this with what is called replenishment groups. A replenishment group is an establishment sample of SICs which replaces a segment of the current sample. A new replenishment group is introduced each quarter until the entire sample has been replaced; after which, the cycle is repeated (currently every four years). The quarterly replenishment groups each have, approximately, an equal number of establishments. This equality reduces the disruption in the quarterly estimates and is within resource constraints. A replenishment group collection cycle begins every three months and the new sample is introduced into the ECI estimates after the section update. 1) Description of the Private Section Establishment Selection Each replenishment sample is composed of a number of related two-digit SIC subsamples. Within each SIC, the frame (Unemployment Insurance File) may be sorted by Census Region, employment or establishment name. A sample of 450 establishments is selected probability proportionate to employment for the entire replenishment group. Systematic samples of about 300 establishments comprise the main replenishment sample. The remaining 150 establishments are selected for several supplemental groups. The supplemental. 89 groups are held in reserve in case additional sample is required if a larger than expected number of out- of-scope is obtained. To enable variance estimation by replication techniques, the establishments are assigned to two half-samples. 2) Description of the Occupation Selection To measure Major Occupation Group (MOG) compensation change, the Occupational Universe (currently based on the 1970 Census occupations) is partitioned into the MOGS, such as professionals, technical workers, etc. Each MOG may be further partitioned into Entry Level Occupations (ELOs), such as Teachers. There are usually 9 to 13 ELOs, which represent all occupations within an SIC. For each ELO found in the establishment, data is collected to represent that ELO. During the initial visit to a sample establishment each detailed establishment occupation is matched into one of the ELOs. Then a probability proportionate to employment selection is made within each ELO, selecting one specific occupation. Data for wages and benefits is then collected for each of the selected detailed establishment occupations. B. Public Sector Sample Design The public sector sample has been fixed since June 1981, when it was introduced. There is no public sector replenishment system because of the lack of updated frame. An easily accessible frame does not exist for State and local governments. 1) Public Sector Establishment Sample Design The public sector frames were divided into four parts: schools, hospitals, State and large local governments (all SICs except schools and hospital), and small local governments. a. Schools: The public elementary and secondary schools frame, (SIC 821) as well as the higher education (SIC 822) frame, came from 1973-74 National Center for Education Statistics (NCES) listing of all State and local schools. Establishments were stratified by 3-digit SIC; then a sample was selected with probability of selection proportionate to enrollment within the school. A first phase mail survey was conducted to determine ELO employments for the selected schools. Using these ELO employments to obtain measures of size, the second stage sample of 206 establishments employing a two- way controlled selection technique controlling on respondent burden and the number of designated quotes within each selected ELO was selected. 90 b. Hospitals: The hospital frame was the 1976 Health, Education and Welfare (HEW) list of public hospitals. The hospital survey design did not include a first phase occupational survey. Public hospitals were stratified by Census region and ownership and selected systematically using probability proportionate to employment. The occupation selection was essentially a systematic sample (equal probability) within each establishment. The 106 establishments in the final sample were then requested to supply data from the appropriate occupations. c. State and Large Local Governments No universe listing of establishments was available for State and large local governments. A refinement survey was used to develop a sampling frame. The local government jurisdictions in the refinement survey (cities, counties, special districts, etc.) were selected from 1972 Census of Government file provided by the Bureau of the Census. Only jurisdictions with more than 100 employees were included in the refinement survey (see "small local governments" below). The 3,729 local jurisdictions were stratified into size class/Census region strata. Forty-six jurisdictions were selected probability proportionate to employment. In addition, sixteen States were selected probability proportionate to employment land included in the Refinement Survey. Once the refinement was completed, a probability proportionate to employment sample of 780 refined units were selected for a first phase occupational employment survey. Occupational employments were requested for nine occupational groups within each of the 780 units. The final sample includes 350 units. d. Small Local Government Due to their small size (units with less than 100 employees), no refinement or first phase survey was done for small local governments. Instead, the list of small local governments was stratified by Census Region and then a probability proportionate to employment sample of 30 units was selected. Any refinement required was accomplished by BLS field representatives at the time of collection. 91 IV. Survey Design and Content A. Design 1) Reporting Unit The ECI reporting unit is the physical location of a business (establishment). Sometimes data can only be collected for a unit which is larger than the original designated establishment, Usually this is acceptable and a weighting adjustment is made later. It is also possible that data is much more accessible at a finer level than an establishment; in this case, subsampling procedures are available to randomly select a subunit. 2) Following Movers If the collection unit is essentially unchanged after a physical move, then it is followed provided it remains within the same State. 3) Weighting The weights for each establishment/ELO is the reciprocal of the selection probability times the ELO employment. There is also a nonresponse adjustment factor applied to the weight. 4) Interview Schedule Each establishment reports wage and benefit data four times a year (March, June, September and December). The typical private sector establishment will be included in the survey for a four year period, at which time the sample is replaced. Currently, there is no definite date when the public sector sample will be replaced. 5) Interview Mode The initial data collection is always a personal visit. During subsequent quarters a mail update form is used. When necessary, telephone calls are made to obtain required data. 6) Questionnaire There are two basic types of ECI collections -- initiation and quarterly update collections. During the initiation, the field representative selects a detail establishment occupation to represent each ELO. Once the establishment occupation is selected, benefit usage, benefit plan, wage and work schedule data are collected for each selected detail establishment occupation. 92 During the quarterly update, wage data and benefit plan change data are collected. When a benefit plan changes, the new plan is incorporated into the database using the initiation usage. B. Content The Employment Cost Index is a relatively new Bureau of Labor Statistics survey measuring the change in the employer cost of employing workers. When the ECI first started its publication in December 1975, it measured quarterly wage change covering the private non-farm sector, excluding Alaska, Hawaii and private households. Publications included overall National, Major Industry Division (MID) like wholesale trade, manufacturing and services; Major Occupation Group (MOG) like Professionals, Managers and Clerical Workers; Census Region (Northeast, South, North Central and West); Union/Non Union and Metropolitan/Non-Metropolitan Area quarterly change numbers. Currently, the ECI is an index measuring total compensation change covering the total non-farm civilian sector excluding private household and the federal government. Compensation is composed of wages and twenty-three benefits (hours related benefits, such as vacation; supplemental pay, such as shift differentials, insurance, such as health benefits; pension and legally required, such as social security). The National series (Overall National, MID, MOG indices) use Laspeyres (fixed weight)/industry/occupation estimates. For each of the non-National series.1/ (Census Region, Union/Non Union and Metropolitan/Non- Metropolitan), estimates (e.g.,union/industry/occupation) are obtained by allocating the fixed weight industry/occupation estimates using current sample data; so that the non-national series cannot be considered Laspeyres. V. Response A. Determination of Private Sector Replenishment Cycle Assuming the sample is completely replaced after n, 2n, 3n,..., quarters and that the response and attrition rates are equal across replenishment, then the response rate obtained after n quarters should be maintained each quarter thereafter. We call this the maintainable response rate. The determination of the appropriate time length for the complete replenishment cycle can be made by computing the maintainable response rates for various cycles and comparing the rates. To compute the maintainable response rate, the following wage information from the original sample is used: proportion of initial sample in scope, 0.85; proportion of initial in scope sample responding, 0.82; proportion of sample remaining each quarter, 0. 98; and __________________________ .1/ For an economic interpretation of the non-national estimates see: Estimation Procedures for the Employment Cost Index, G. Donald Wood, Jr., Monthly Labor Review, May 1982. 93 number of establishments required at the end of the replenishment cycle 2000. Using the above information the following table on quarterly sample size and maintainable response rate is determined. Estimated Replenishment Number of units Maintainable response Cycle (Years) initiated per quarter rate (wages) 2 385 0.76 3 267 6.74 4 208 0.71 Considering the initial work required introducing an establishment into the survey, a two year cycle was not considered cost effective. A 0.71 wage response rate with a four year cycle is lower than desired considering the fact that the benefit response rate would be closer to 0.6 than to 0.7. A three year cycle would keep respondents in the survey for a reasonable length of time and provide a benefit response rate at least close to 0.65. Therefore, the initial decision was to proceed with a three year cycle. After the first year of replenishment samples, it became apparent that field resource constraints would not allow a three year cycle. We are currently working on a four year cycle. B. Public Sector The Public Sector does not have a replenishment system in place at this time. The initial response rate, in June 1981, was 81%. Since then the attrition rate has averaged 0.3% each quarter. These numbers are considerably better than the private sector. Even though there is no replenishment system, the response rate does not decrease quickly. In addition, the number of establishment births and deaths within the public sector should be much less than the number within the private sector. The universe, therefore, should remain relatively stable until 1990. C. Imputation Schemes There are three levels of imputation in the ECI. The first level is a weight adjustment to compensate for the initial nonresponse. The second level is an imputation for temporary nonrespondents. (Those establishments that will respond next quarter, but for some reason cannot respond this quarter). This imputation is done at the item level. Its purpose is to serve as a link for periods when there is a response. The third level of 94 imputation is at the estimation cell level, whenever there is no data for the entire estimation cell. This imputation assures that the same cells are being compared each quarter. VII. Data Product At the present time, no public use tapes of micro ECI data are available. The only data available to researchers are that contained in the quarterly news release which is available on Labstat. The feasibility of developing a public use tape is being explored. 95 CASE STUDY 4 NATIONAL LONGITUDINAL STUDY OF THE HIGH SCHOOL CLASS OF 1972 I. Purpose: The basic purpose of NLS-72 is to provide data on the experiences that affect the development and attainment of a current generation of young people. Specifically, this study provides data on: . the transition of young people from high school to postsecondary education . the transition from high school to the world of work, . persistence in postsecondary education (as opposed to dropping out), . the transition from postsecondary education to the world of work. II. Sponsor NLS-72 has, since its inception, been sponsored by the National Center for Education Statistics (NCES) within the U.S. Department of Education. The principal contractors who have played major roles in WLS- 72 are: 1. Education Testing Services (ETS) -- Base-year survey in 1972. 2. Research Triangle Institute (RTI) -- First four follow-up surveys 1974, 1975, 1977, and 1980. 3. National Opinion Research Center (NORC) -- Fifth follow- up survey and Postsecondary Transcript Study in 1984-85. III. Sample Design The sample design for NLS-72 is a stratified multistage probability sample of students from all schools, public and private, in the 50 states and the District of Columbia, which contained a 12th grade class. Stratification variables were: type of control (public vs. private), geographic, region, enrollment size, proximity to a college, percent minority, income level of community, and urbanicity. The original sample design for the base-year survey called for selecting a probability sample of 1,200 schools from the population of schools with a 12th grade, and within each school random selecting 18 seniors. Since 231 of these schools refused to participate and 21 had no seniors enrolled, the number of schools actually participating was 948. The number of students participating was 16,683. At the time of the first follow-up, in 1974, 205 of the nonparticipating schools were induced to participate and former seniors from those schools were administered retrospective surveys. Ultimately the reconstituted base-year sample consisted of 22,652 students from 1,318 schools. IV. Survey Design and Content In the base-year survey, questionnaires and cognitive tests were administered to groups of students in each participating school. Information on courses taken and grades earned was extrated from school records. 97 Follow-up surveys have been conducted primarily by mail but when repeated reminders failed to elicit a response, resort was had to personal interviews, either by telephone or face- to-face. About one third of the mail respondents in each follow-up survey were telephoned to resolve response inconsistencies. The fifth follow up, which is now in the field-test stage, is being funded by NCES with the help of a consortium of interested agencies. It will also be conducted primarily by mail. To reduce costs only a subsample of the original sample will be used. The various questionnaires tap numerous content areas, including: background characteristics, cognitive ability, socioeconomic status, home background, community environment, relative importance of significant others, current and planned educational and occupational activities, school characteristics, performance in school, work performance and satisfaction, goal orientations, marriage and family, opinions of school, et al. A more detailed listing of survey content areas is displayed in the attached "Table 2." The content areas for the fifth follow-up survey are being reduced somewhat in order to make room for certain new topics. Education and work history items are retained, however. In addition, special new questionnaires are included to be filled out by hose respondents who have become teachers, or parents. V. Response Rates As a result of extraordinary tracking efforts and intensive data collection activities, the response rate to the various student questionnaires' has remained quite high over the 12 years of RLS-72 operation. Student responses rates for each of the surveys thus far completed were: Base year 87.8%* 1st FU 94.2% 2nd FU 92.1% 3rd FU 88.7% 4th FU 82.2% * This figure is the percentage supplying data, based on all targetted students in participating schools in the original base-year survey. The corresponding figure for the reconstituted sample was 73.6%. VI. Evaluations To maximize the validity and reliability of the data, several procedures were followed: 1. For each of the surveys thus far completed, the student questionnaire was first pretested on a sample of 1971 seniors. (This will not be possible for the 5th follow up because tracing efforts for those students were not adequate to retain a sufficiently large subsample). 98 2. For the base-year survey, a reliability check was conducted in which 500 respondents were asked to reanswer 10 questions 3 months later. 3. For the base-year survey, a validity check was conducted by asking the parents of 500 students to confirm or correct the student's report of family income. 4. To improve the quality of mail responses, all questionnaires were checked for completeness and consistency. Respondents whose forms failed these edit checks were telephoned for clarifications. VII. Data Products and Analysis NCES makes all NLS-72 data files available to the public at cost. As each new data file becomes available, an Announcement to that effect is widely disseminated to potential users. As of 1981, over 320 research reports based on NLS-72 data had been published. These are listed and annotated in the following publication: National Longitudinal Study of the High School Class 1972; Study Reports Update: Review and Annotation by M. E. Taylor, C. E. Stafford, and C. Place. Research Triangle Institute, June 1981. Click HERE for graphic. 99 Click HERE for graphic. 100 CASE STUDY 5 HIGH SCHOOL AND BEYOND I. Purpose: High School and Beyond is a longitudinal study of a nationally representative sample of 1980 high school sophomores and seniors in the United States. Its basic purpose is to replicate, eight years later,the National Longitudinal Study of the High School Class of 1972. Specifically HS&B; would provide updated information on: . factors influencing persistence vs. dropping out of high school or college, . the transition of young people from high school to postsecondary education or to the world of work, . persistence in postsecondary education, . the transition from postsecondary education to the world of work . courses taken and grades received, both at the high- school and the college level. II. Sponsor Since its inception HS&B; has been sponsored by the National Center for Education Statistics (WCES) within the U.S. Department of Education. The principal contractor who has been primarily responsible for the details of research design and for data collection, coding, and storage,has been the National Opinion Research Center (NORC). III. Sample Design HS&B; employs a two-stage, highly stratified sample design. In the first stage 1,122 schools that had either 10th or 12th grade students (or both) were drawn. To make the sample more useful for policy analysis, the following types of schools were oversampled: alternative public schools, public schools with high percentages of Hispanic students, Catholic schools with high percentages of minority group students, and high performing private schools. In the second stage, 36 sophomores and 36 seniors were randomly selected, school size permitting, yielding total samples of 30,030 sophomores and 28,240 seniors. In the first follow-up survey, conducted in spring 1982, all sophomore cohort members who were still in the same schools were included with certainty, as were all dropouts and other subgroups of policy interest, yielding a sophomore cohort sample size of 29,737. Of these, a subsample of 18,000 was selected for a detailed study of high school transcripts. In the first follow-up survey a subsample of 11,995 of the 1980 senior sample were selected. The second follow-up survey took place in spring, 1984. At that time, samples of 15,000 members of the sophomore cohort, and 11,995 members of the senior cohort were selected for further data collection. 101 IV. Survey Design and Content In the base-year survey, questionnaires and cognitive tests were administered to groups of students in each participating school. The administrator in each school filled out a questionnaire about the school; teachers in each school were asked to comment on students in the sample; and a sample of parents of sophomores and seniors (about 3,600 for each cohort) was surveyed primarily for information about their plans for financing their child's postsecondary education. The first follow-up survey of the sophomore cohort took place in spring 1982 when most respondents were seniors. Questionnaires and tests were group administered to all base- year sample members still attending the same school. Dropouts, and transferees were contacted by mail or as a last resort, by personal interview. For the second follow-up of the sophomore cohort and for all follow-ups of the senior cohort, contact was by mail or, when necessary, by personal interview. The student questionnaires cover a large number of content areas, including: school work, gainful employment, demographic characteristics, physical condition, parental characteristics, social relations, and life plans. Marital and fertility history are also covered in the follow-up questionnaires. V. Response Rates A total of all (72 percent) of the 1,122 eligible schools selected for the base-year survey actually participated. Of the 311 schools that were unable or unwilling to participate, 204 were replaced with schools which matched them with regard to geographical area, enrollment size, community type, and other characteristics. This brought the total number of participating schools to 1,015, or 90 percent of the 1,122 target. The student-level base-year response rate within participating schools was 85 percent. The first follow-up survey response rate was about 94 percent for each cohort. Response rates for the second follow-up survey were 92 percent and 91 percent for the sophomore and senior cohorts, respectively. VI. Evaluations To maximize the validity and reliability of the data, several steps were taken: (1) all data collection instruments were pretested on a group of respondents similar to those who would participate in the main survey. (2) Ambiguous or inconsistent responses to mail questionnaire items were clarified by means of telephone calls. (3) A special analysis was performed by NCES to compare the estimates of family income given by the students with those given by the parents. 102 VII. Data Products and Analysis NCES makes all HS&B; data files available to the public at cost. As each new data file becomes available, an Announcement to that effect is widely disseminated to potential users. As of summer 1984, over 150 different research studies based on HS&B; data had been published. The principal contractor of HS&B;, NORC, is developing a computerized bibliography of all HS&B-based; publications. 103 CASE STUDY 6 NATIONAL LONGITUDINAL SURVEYS OF LABOR MARKET EXPERIENCE I. purpose The National Longitudinal Surveys of Labor Market Experience (NLS) were designed to identify factors that influence the labor market behavior and experience of a group of workers (Parnes:12). Five cohorts were selected to represent workers with labor market problems of special concern to national policy makers. The NLS was the first national survey of employment-related phenomena to focus on individual labor market behavior through time. Since 1940, cross-sectional data on labor force participation had been available from the Current Population Survey. Since the 1950's, information on earnings and employer characteristics had been available from the Continuous Work History Sample, based on a sample of the Social Security Administration's records. Longitudinal data on associated topics is available from the Panel Survey on Income Dynamics, the Longitudinal Retirement History Study, and the Continuous Longitudinal Manpower Survey of CETA participants. None of the other surveys, however, has provided data like those from the NLS on individual gross flows linked to attitudes and experience. II. Sponsors In 1965 the U.S. Department of Labor's Manpower, Development and Training Administration (now the Employment and Training Administration) undertook a series of longitudinal studies of the labor force. The Department of Labor (DOL) set up a contract with the Ohio State University Center for Human Resource Research (OSU) under which OSU was responsible for planning and analyzing the surveys. The DOL set up a separate contract with the U.S. Bureau of the Census for data collection for the original cohorts. Data collection for the new youth cohorts was subcontracted to the National Opinion Research Center (NORC). III. Sample Design Respondents in the original four cohorts were selected from an area probability sample of the non-institutionalized civilian U.S. population. Primary Sampling Units were selected on the basis of the 1960 Census. For each cohort reliable statistics for Whites and Blacks were ensured by selecting about 1,500 Black respondents and 3,500 White respondents in each cohort. This was accomplished by classifying enumeration districts by race, and using a sampling rate between 3 and 4 times higher in predominately Black ED's. Forty-two thousand housing units were contacted for screening interviews in early 1966. From these, interviewers identified just over 22,000 eligible respondents in 13.500 households. (A number of households contained more than one respondent, sometimes belonging to more than one cohort.) 105 The new youth cohort selected in 1979 is arranged in 8 strata, by race, ethnicity, income, age and sex. For these cohorts, the Census Bureau drew a sample from an area probability sample of the U.S. stratified so as to produce segments of varying size but equal with respect to the characteristics of the target sample (OSU,1979:11). Seventy five thousand addresses were selected for screening interviews and from these the WORC identified a final sample of about 12,000 respondents between 14 and 21 years of age. The new young men's cohort includes respondents who are serving in (or returned from) the armed forces. The Department of Defense provided lists of persons on active military duty to NORC for sample selection. In the first stage a sample of military units was drawn, then within these units separate samples of males and females were selected, including some respondents not living on military bases. IV. Survey Design and Content A. Design 1. Respondent Rules Proxy responses are only accepted from relatives or other members of a sample person's household, if the sample person is temporarily incapable of answering questions. Specific questions eliciting opinions or attitudes are excluded from proxy interviews. 2. Reporting Units Separate questionnaires are completed for each respondent in a multiple respondent household. Separate household record cards are also prepared, but data from one may be transcribed to another by the interviewer. Household composition is recorded at certain interviews. CPS definitions are used for "household members." Household characteristics are tabulated as respondent attributes at each wave. OSU has prepared special tabulations of multiple respondent households, such as a fathers-and-sons tape, a siblings tape, etc. 3. Following Movers Local government agencies, the Postal Service, neighbors and relatives, and others recorded at the first interview as knowledgeable about the respondent's whereabouts, are among the contacts that may be questioned to obtain the current address of a sample person who has moved. Respondents who have moved are con- tacted through the field office closest to their new location. 4. Weighting The basic weight for each sample case is a reciprocal of selection probability, and reflects the differential sampling ratio by race. The samples have been weighted so that the characteristics for each wave match the known distribution of the characteristics in the population. 106 5. Interview Schedule The original NLS plan called for annual interviews of each cohort for five years. To reduce costs, after 1968 the cohorts of adult men and adult women were interviewed only every other year. In 1972 all four cohorts were extended by including two annual telephone surveys and a personal interview on the tenth anniversary (1976-77). The entire survey was extended an additional 5 years in 1977, on the recommendation of a group of analysts and data users convened by the department of labor. After 1983 the older and younger men's cohorts were dropped, and the older and younger women's cohorts were extended 5 years (along with the new youth cohorts). 6. Interview Mode For the original four cohorts the first and final waves consisted of personal interviews. Four of the intervening waves were conducted by telephone (5 for mature women), and one mail questionnaire was sent in 1968. The interview, schedule for the new youth cohort called for persona interviews in each year from 1979 to 1964. B. Content The NLS was originally composed of 4 separate longitudinal cohorts: Adult men, adult women, young men and young women. The cohorts represent four groups important to policy makers: men in the years leading to retirement (between 45 and 59 years old in 1966); women likely to be re-entering the labor market (between 30 and 44 years old in 1967); and young men and young women likely to be finishing their education and entering the labor market (boys between 14 and 24 years old in 1966 and girls between 14 and 24 years old in 1968). The longitudinal survey of adult men was planned to answer specific research questions about retirement decisions, about skill obsolescence, about the duration of unemployment in this age group, and about the relationship between health and labor market experience. The sample of adult women was designed to study women's entry or re-entry into the labor force after a period spent primarily in raising children. Special attention was paid to attitudes toward employment in general and towards the propriety of labor market activity for women in particular. The cohorts of young men and young women were planned to provide information on the extent of occupational knowledge among teenagers, and on attitudes toward education and toward employment experiences. The new youth cohort was developed in 1979 to study employment patterns in low income and minority groups, and to look at changes since 1960. Many of the interviewing procedures and labor force concepts used in the NLS were similar to those used in the Current Population Survey (CPS) and the Census Bureau's CPS interviewers were often assigned to do NLS 107 interviewing as well. Coding of occupation and industry continue to conform to the definitions used in the 1960 Census. Although for most recent 1980 codes are used as well. Older Men's Cohort: In each wave data were collected to measure employment and unemployment. For all jobs held since leaving school, the interviews collected occupation, industry, location and duration of employment. In addition, annual income and earnings were collected for each job, along with measures of job satisfaction. Mature Women's Cohort: The surveys of adult females contained similar questions about background and labor force participation. But in place of questions about retirement, there were questions designed to study the process of leaving and re-entering the labor force. Background questions for women were designed to distinguish labor market participation before and after any interregnum that began with marriage. A large number of questions dealt with household structure and responsibilities for dependents,, including attitudes toward child care, costs and preferences for child care, the husband's health limitations, and husband's attitudes toward women working. Young Men's and Young Women's Cohorts: The questionnaires for the original youth cohorts Were similar in most ways the adult questionnaires. Among the unique variables were an inventory of current job characteristics which included variety and autonomy of tasks, feedback from supervisors, and opportunities for contact and friendships on the job. Union membership was measured in several waves, and a large number of questions measured educational performance and experiences. These included curriculum preferences in high school and college, college finances, and reasons for leaving school. For young men, only, retrospective data on military service were collected, including military job series. For the young women's cohort, questions were asked relating to household dependents and child care responsibilities. These were identical to questions asked in the survey of adult females,, including the repeated measures of attitudes toward women working. Intermittent Questions: For the adult males, questions were asked in some waves pertaining to physical health, retirement plans, and attitudes toward women working. In other waves questions were asked about commuting times and costs, collective bargaining coverage, training after leaving school, spouse's health limitations, and military service. In two-waves there were questions calling for retrospective evaluations of career experiences, including perceptions of age, sex and race discrimination, perceptions of individual career progress, and perceptions about job pressures. 108 For the adult women's cohort, there were questions in some waves about volunteer activities, and questions on attitudes toward women working were repeated at intervals. A number of attitude measures were collected intermittently for the young men's cohort. In the first interview a score for occupational knowledge was compiled, and in the final interview a standard index of job satisfaction was derived for young men. Questions were asked at intervals to evaluate job aspirations and expectations about education and training. Data from Administrative Records: For the adult cohorts, the size of the local area labor force, and the annual local unemployment rate were recorded in each file at each wave. In addition, for the adult female cohort, an index of local demand for female labor was also included. For the youth cohorts, a standard IQ test was administered once to each respondent. The presence of an accredited college in the local area was recorded in each file during the first interview. An index of local demand for female labor was included in six waves for young women. For all the youths, background data were collected on the quality and curriculum of the schools that the respondents were attending at the tine of their selection for the sample. V. Response The possibility of sample attrition worried the designers of the NLS, but it does not appear that any major attrition biases have detracted from the reliability of generalizations about the populations which the MLS cohorts represent (OSU, 1982). Over all, after 12 years of the survey, an average 80 percent of the eligible respondents were still being interviewed (U.S.:321). When a 5 year extension was considered for the original 4 cohorts, the Census studied the known characteristics of non- respondents, and concluded that after 15 years those still being interviewed were not significantly different from those who had dropped out of the survey, judging by most socio-demographic characteristics (OSU, 1982). The attrition rates have differed by cohort. Three years after the first interview for adult males, almost 5 percent of these respondents were no longer eligible (through death or institutionalization) and about 92 percent of the remainder were interviewed. The worst attrition has been in the original young men's cohort, perhaps due to the exclusion of those serving in the armed forces (Parnes:25). Of those interviewed in 1966, 1.4 percent were dead or institutionalized in 1968, and an additional 12.4 percent were out of scope because they were in the armed forces. Just under 89 percent of the remainder were interviewed. 109 The figures for women and girls were slightly better. One percent of the women were ineligible after 2 years, and almost 94 percent of those eligible were interviewed. For girls, over 93 percent of the eligible respondents were interviewed in 1970, 2 years after selection. To monitor sample attrition in the four original cohorts, every 5 years the distribution of such characteristics as occupation, educational attainment, age and marital status was compared to national estimates. To compensate for attrition, interviews and non-interviews are stratified by race, education, and residential mobility, and the weight of interviews in each cell is adjusted for the proportion of non-interview cases in each wave. A final adjustment is made for the re-entry of young men serving in the armed services during the year the sample was selected (1965- 66). There are no allocations or amputations for missing data to prevent inconsistencies with data from other waves. Only when missing data are clearly due to a record-keeping error are data from one item used to replace those from another. In 1982, the characteristics of respondents still in the sample were compared to the characteristics of the sample interviewed in the initial year. Age, race, educational attainment, employment status, industry, occupation, marital status, SMSA, and annual income were all compared. For most cohorts, the differences in distribution of characteristics between the 2 samples were less than 2 percent. It was concluded that attrition had not seriously distorted the representativeness of the cohorts, and that any potential bias could be dealt with through weighting (Rhoton:7). VI. Evaluations To reduce attrition in the new youth cohort, several procedures were modified, based on experience with the 4 original cohorts. First, some questionnaire items that had caused response problems were changed. Second, more information was collected at the first contact that could be used in tracing mobile respondents. Third, more information about the NLS was provided to respondents, both before and after the interviews, and a newsletter is mailed to respondents to report on survey results. Finally, the NORC traced and contacted persons who were non-respondents in earlier waves. (Previously nonrespondents were dropped from the sample after 2 years of noninterviews.) This tracing was successful in over one- third of attempted cases (Rhotor :2-12). VII. Data Products and Analysis The Ohio State University makes WLS data files and documentation available to other researchers at cost. By 1979, data files were available for adult males 1966-76, for adult females 1967-76, for young men 1966-75. and for young women 1968-75. The data at any release point are composed of the entire longitudinal record, and include revisions to remove errors found in previous releases. 110 CASE STUDY 7 RETIREMENT HISTORY STUDY I. Purpose The Social Security Administration's Retirement History Study ( RHS) is a multiwave panel survey designed to address a number of policy questions relating to the causes and consequences of retirement. Among these questions are: Why do individuals retire before age 65? How well does income in retirement replace proretirement earnings? What happens to the standard of living after retirement How do Social Security and other laws affect retirement patterns? Until the RHS was undertaken, data bearing on these Issues were based on retrospective questions from cross sectional surveys. A prospective longitudinal study permits accurate analyses of the factors influencing the retirement decision and an accurate description of the complex of personal adjustments required during preretirement and postretirement years. II. Sponsors The RHS was sponsored by the Social Security Administration under direction of staff in the Division of Retirement and Survivor Studies, Office of Research and Statistics. Early consultation was provided by an outside advisory committee. Data Collection was performed by the Bureau of the Census. III. Sample Design The original sample of 12,549 persons was a multi-stage area probability sample selected from members of households in 19 retired Current Population Survey rotation groups. The sample was nationally representative of persons age 58 through 63 in 1969. The sample included men of all marital status categories and women with no husband in the household. Married women were excluded because they were found in early pretests to have no independent retirement plans. Institutionalized persons were also excluded from the original sample. IV. Survey Design and Content A. Design 1. Respondent Rule Proxy responses were accepted only for that part of the questionnaire dealing with spouse's labor force history. Sample persons who were not interviewed in the first wave (1969) were dropped from the survey. Respondents who were institutionalized 90 days or more at the time of subsequent waves were kept in the sample. All other noninterviews in later waves were dropped from the sample. 111 2. Reporting Units The reporting units were designated sample members (individuals) only. 3. Following Movers A year before each interview (after the first) the SSA provided the Census with current address listings for all sample persons and/or Spouses who were benefit recipients. In addition the Census checked all previous addresses with the post office to identify movers. Both these procedures reduced the number of unanticipated movers (especially between data collection regions) encountered at the time of interviewing. All movers were followed except those who emigrated or who lived more than 50 miles from any PSU. 4. Weighting The weighting procedure began with a basic weight based on factors relating to the original CPS rotation groups and was followed by several stages of ratio estimation. Weighting for noninterviews was adjusted after 1969. No further weighting adjustments were made because by 1979 SSA ha,d determined that the differences between weighted and unweighted estimates were too small to justify the procedures. 5. Interview Schedule Initial interviews were conducted in 1969 and then in alternate years through 1979. In each wave the interviews were conducted over a 3 to 4 month schedule (usually February to-June). 6. Interview Mode The interview mode was personal and face-to-face. At each wave contact began with a letter from the Census informing the sample of the upcoming interview. Interviewers were encouraged to use telephone contacts to schedule their visits, but all interviews were by personal visit. Questionnaires with missing information could be completed by telephone. B. Content The interview schedule was designed to elicit a wide range of Information about preretirement lives and attitudes of sample members. The schedule was divided into six sections: (1) respondent's labor force history; (2) preretirement and retirement plans; (3) health; (4) household, family and social activities; (5) income, assets and debts for respondent, spouse and children under age 18; and (6) spouse's labor force history. Base-line labor force history was collected only in the first 112 interview (1969). This explains why all noninterviews In the 1969,wave were dropped from the sample. By collecting labor force history for the sample person's spouse, longitudinal data was available If a surviving spouse later replaced a deceased sample person as respondent. Survey data were also supplemented with individual Social Security earnings and benefit records, yielding information on the continuity of work history and the amount or benefits to which the workers were entitled. V. Response Of the original sample Of Just over 12,500 selected in 1969, 8,700 were interviewed in 1977. This included over 1,000 surviving spouses who were eligible to serve as respondents after the death of a sample person. At each wave nonresponse (composed of refusals, no contact, and persons institutionalized) seldom rose over 4 percent. The remaining attrition was caused by deaths among the sample. The low nonresponse rate was in part attributable to efforts made to contact respondents: no limits Were placed on the number of attempts interviewers should make. Some refusals were related to the length of the interviews. The first averaged an hour and 15 minutes long. In subsequent years the length of the Interview was the Most frequently cited reason for refusal to respond. VI. Evaluation (Unknown) VII. Data Products and Analysis Most of the published analyses have been organized into a series of reports that are available from the Social Security Administration. 113 CASE STUDY 8 WORK INCENTIVE EXPERIMENTS I. Purpose Section 505(a) of the "Social Security Disability Amendments of 1980" (Pub. L. 96-265) directs the Secretary of Health and Human Services (HHS) to develop and carry out experiments and demonstration projects designed to encourage disability insurance beneficiaries to return to work and leave the benefit rolls. The objectives of these experiments, specified in the law, are to generate long-range savings to the trust funds and to facilitate the administration of title II of the Social Security Act. Section 505(a) itself contains several suggestions for experimental variables, specifically - Benefit reductions based on amount of postentitlement earnings. - Lengthening the trial work period. - Altering the 24 month waiting period for Medicare benefits. - Changing the manner in which the program is administered. The language in section 505(a) states explicitly that the experiments should be carried out in a way that permits thorough and complete evaluation and on a large enough scale so that the results may be generalized reliably to the future day-to-day operation of the disability program. In addition, the report of the House Ways and Means Committee indicates Congress' desire that no individual be disadvantaged compared to existing law. II. Sponsors This project, mandated by law (Pub. L. 96-265), directs the Secretary of HHS to carry out the experiments. Planning the experiments has been delegated to SSA. The law authorizes the use of disability insurance trust fund monies to pay for the experiments and authorizes the Secretary to waive the present benefit and eligibility requirements of titles II, XVI and XVIII to the extent necessary to carry out the experiments. ____________________________ *Since its mandate in the Disability Insurance Amendments of 1980 (Pub. L. 96-265), the Social Security Administration disability program work incentive experiments have undergone a number of designs. Due to a number of administrative problems and the imminent deadline of the legislative mandate no experimental plan has yet to be implemented. Legislative extension of the experimental authority is now under consideration. For expository purposes the plans developed in the Fall of 1982 are presented. 115 III. Sample/Experimental Design A. The Study Population The study population for the WIE consists of all newly awarded beneficiaries except those who fall in one of the following categories: - Under age 18 or over age 59 at time of award. - Residing outside the 48 contiguous States or in an institution. - Received a closed period award. - Previously entitled to DIB. - Dually entitled to DI and to title II auxiliary benefits. - Statutorily blind. - Career railroad case certified to the ERB for payment. B. Experimental Design 1. Programmatic Changes Sample sizes for each experimental group and the control group have been determined in an attempt to insure the ability to measure important increases in the proportion of work recoveries. Our best estimate is that under current law about three percent of a newly awarded beneficiary cohort will have their benefits terminated after successful completion of a trial work period. We estimate that for the proposed experimental alternatives, a one percentage point increase in the recovery level (that is, a change from three to four percent) would yield significant trust fund savings, on the order of $100 million per year or larger. Thus, the sample sizes we choose insure a good chance of detecting a one percentage point change if this change occurs in any experimental group. The required sample size total 21,000 cases, including 3,000 for each of the five experimental groups and 6,000 for the control group. Schematically, the design of the WIE and the sample sizes and allocations can be pictured as follows: 116 Medicare extension Click HERE for graphic. Group T.1 represents a control group operating under the provision of the current law. For each of the experimental groups, T.2-T.6 inclusive, only the programmatic change(s) specified applies. 2. Administrative Changes Two administrative changes will be instituted to assure that the WIE operates effectively and efficiently. These changes are (1) a face-to-face interview at the start of the experiments that explains the experimental changes to the participating beneficiaries, and (2) use of a quarterly report of work and earnings. With these up-to-date reports it is Possible to minimize the problems benefit overpayments. These changes in themselves may alter beneficiary behavior. The experiments is therefore designed to test whether these administrative changes have a direct effect an recovery. 117 The following experimental group make up this portion of the WIE experimental design: Click HERE for graphic. This scheme takes advantage of the 6,000 cases that will already serve as the control group for the WIE. As a result, only an additional 9,000 cases would be required to study the impact of the two administrative changes being tested. The considerations used in determining sample size and the allocation of cases among the four test groups involved in this portion of the experiment are essentially the same as those discussed for the programmatic revisions. It should be pointed out that none of these cases (the 6,000, as well as, the additional 9,000) will involve either increased benefit payments or Medicare reimbursements. They all operate under present program provisions. C. Sample Design 1. Stratification In order to improve the efficiency of the experimental design the award population will be Stratified by two factors -- age and medical diary status. Since younger beneficiaries are more likely to return to work and leave the benefit rolls, they are likely to take advantage of the experimental provisions than older beneficiaries. Beneficiaries who are scheduled for medical reexaminations might be less likely to be granted trial work periods because they are judged more likely to recover. 118 The following table defines four age/diary strata. Medical Stratum Age diary S1 18-44 Yes (young) S2 18-44 No S3 45-59 Yes (old) S4 45-59 No Taking these strata into account, the full experimental design has the following dimensions: Experimental Stratum group Total S1 S2 S3 S4 Total 36,000 3,600 7,200 3,600 21,600 T1 6,000 600 1,800 600 3,000 T2 3,000 T3 3,000 T4 3,000 T5 3,000 300 900 300 1,500 T6 3,000 T7 3,000 T8 3,000 T9 3,000 T10 6,000 600 1,800 600 3,000 The allocation to stratum will be roughly proportionate to size. 119 Note that an additional experimental group, T.10, is shown. This group represents a "silent" control group. The beneficiaries in this group do not receive any program or administrative changes, as is the case for group T.9. The beneficiaries in T.10, however, will not be processed by the WIE review unit. This allows us to test the experimental effect of establishing the special unit itself through comparison of T.9 and T.10 outcomes. Thus, the total number of beneficiaries with any involvement in the WIE is now 36,000. 2. Geographic Clustering and Stratification by Dat a of Award In order to limit the impact of the face-to-face treatment application on SSA field staff and Costs, SSA's Office of Field Operations has asked that WIE sample cases be in no more than 200 SSA districts. (A district is defined to be an SSA district office and its associated branch offices.) We, therefore, group the WIE population into clusters of SSA districts. The selection of a sample of clusters is the first stage of selection for the WIE sample. The size of these clusters depends on a number of interrelated requirements. The first requirement is our desire to put a full replicate of the experimental design (or multiples thereof) into each cluster of districts as indicated in the following table: Experimental Stratum group Total S1 S2 S3 S4 Total 120 12 24 12 72 T1 20 2 6 2 10 T2 10 T3 10 T4 10 T5 10 1 3 1 5 T6 10 T7 10 T8 10 T9 10 T10 20 2 6 2 10 120 One hundred and twenty cases is the minimum number required to simultaneously satisfy the allocations discussed above among the strata and among the experimental groups . Placing a full replicate In each cluster induces orthogonality between treatment (and strata) and cluster and facilities the analysis of experimental results. In particular, under the assumption of no interaction between treatment and cluster in producing experimental outcomes, the association between treatment and outcome can be measured by tabulating treatment (and strata, if necessary) results alone essentially ignoring geographic effects. The ability to display the results of the experiments in an uncomplicated manner is of great importance in presentations to those persons responsible for program and operating policy. The second aspect of the determination of minimum cluster size is that each cluster should have a high probability of providing the necessary number of sample cases in each stratum to complete the design; that is, 12 cases for S1, 24 for S2,, 12 for S3 and 72 for S4. It turns out that a population of 250 will yield the needed cases with a probability greater than .998. The third aspect to be, considered is that the number of districts in the sample must not exceed 200. This constraint has implications for the length of the sampling period. There are about 614 districts contained In the 48 contiguous States with an average of about 350 new awards per district per year. Since the sample each cluster will require 250 awards to achieve a 120 case replicate, about 75,000 awards will have to be available to obtain the full 36,000 case sample. Since 200 districts can supply about 70,000 cases a year, the 200 district constraint implies the need for a 1 year sampling period. The 1 year sampling frame will be divided into 6 bimonthly sampling periods, with a full 120 case replicate of the design going into each cluster of districts in each sampling period. Each cluster will need to supply 1,350 awards in each year. Since each cluster supplies 720 (120 times 6) sample cases, 50 clusters are required for the sample to complete the design in 1 year (50 x 720 = 36,000). IV. Survey Design and Content A. Design 1. Respondent rule. No proxy responses are accepted. 121 2. Reporting units. Individual beneficiary and spouse. 3. Following movers. All movers will be followed. 4. Weighting. The basic weight for each sample case will be the reciprocal of estimation is the probability of selection. No need for ratio anticipated. B. Interview Schedule/Mode and Content In addition to data from administrative records, a baseline questionnaire and followup mail questionnaire will be administered. At the start of the experiment, field personnel will contact all persons (except those in T1.1 and T1.3 and the silent control group) to explain to them in person. At that time the interviewer will administer a short questionnaire designed to obtain data on demographic characteristics, family composition, amount and source of family income and private disability insurance benefits. The questionnaires will be mailed to members of groups that are not contacted for face- to-face interviews. A supplemental mail questionnaire will be sent every 6 months over 4 years to a subsample of 10,000 beneficiaries. The questionnaire will be designed to elicit information that will update the baseline interview and describe how beneficiaries find jobs and the factors involved in the success or failure Of sustained work. V. Response Since all participants will be tracked through administrative records, there will be no actual attrition from the study. Response to the supplemental questionnaires is expected to be high because they will be administered in conjunction with required administrative reports. VI. Evaluation None planned. VII. Analysis Plans (See text discussion.) 122 CASE STUDY 9 NATIONAL MEDICAL CARE EXPENDITURE SURVEY I. Purpose The National medical care Expenditure Survey (NMCES) was designed to assess the use of health care services and to determine the patterns and character of health expenditures and health insurance for the U.S. noninstitutionalized civilian population in 1977. The survey was conducted by the National Center for Health Services Research (NCHSR), as part of a landmark study, the National Health Care Expenditures Study (NHCES), which is providing information on a number of critical issues of national Health policy. Topics of particular interest to government agencies, legislative bodies, health professionals, and others concerned with health care policies and expenditures include: - The cost, utilization, and budgetary implications of changes in federal financing programs for health care and of alternatives to the present structure of private health insurance. - The breadth and depth of health insurance coverage. - The proportion of health care costs paid by various insurance mechanisms. - The influence of Medicare and Medicaid programs on the use and costs of medical care. - How and why Medicaid participation changes over time. - Patterns of use and expenditures as well as sources of payment for major components of care. - The cost and effectiveness of different federal, state, and local programs aimed at improving access to care. - The loss of revenue resulting from current tax treatment of medical and health insurance expenses, particularly with regard to the benefits currently accruing to different categories of individuals and employers, and the potential effects on the federal budget of proposed changes to tax laws. - How costs of care vary according to diagnostic categories and treatment settings. The data for these studies were obtained from the National Medical Care Expenditure Survey (NMCES), which has provided the most comprehensive statistical picture to date of how health services are used and paid for in the United States. The survey was completed in September, 1979. Data were obtained in three separate, complementary stages. About 14,000 randomly selected households in the civilian, noninstitutionalized population were interviewed six times over an 18-month period during 1977 and 1978. This survey was complemented by additional surveys of physicians and health care facilities providing care to household members during 1977 and of employers and insurance companies responsible for their insurance coverage. 123 II. Sponsors Funding for NMCES was provided by National Center for Health Services Research, which co-sponsored the survey with the National Center for Health Statistics. Data collection for the survey was done by Research Triangle Institute, NC, and its subcontractors, National Opinion Research Center of the University of Chicago, and Abt Associates, Inc., of Cambridge, MA. Data processing support is being provided by Social and Scientific Systems, Inc. of Washington, D.C. III. Sample Design The survey sample was designed to produce statistically unbiased national estimates that are representative of the civilian noninstitutionalized population of the United States. To this end, the study used the national multi-stage area samples of the Research Triangle Institute and the National Opinion Research Center. Sampling specifications required the selection of about 14,000 households. Data were obtained for about 91 percent of eligible households in the first interview and 82 percent by the fifth interview. The NMCES area sampling design can be characterized as a stratified three-stage area probability design from two independently drawn national area samples. The fourth stage involved the selection of ultimate sampling units (e.g., housing units and a special class of group quarters). An essential ingredient of this design is that each sample element has a known, nonzero selection probability. Also, the national general purpose area samples from the Research Triangle Institute (RTI) and the National Opinion Research Center (NORC) used in the survey are similar in structure and, therefore, compatible. Except for difficulties associated with survey nonresponse and other nonsampling errors, statistically unbiased national and domain estimates can be produced from each sample or from the two samples combined. The first stage-in both designs consists of primary sampling units which are counties, parts of counties, or groups of contiguous counties. The second stage consists of secondary sampling units which are census enumeration districts or block groups (Bureau of the Census, 1970). Smaller area segments generally consisting of at least 60 housing units constitute the third stage in both designs; a subsample of households was randomly selected from each of these segments in the final stage of sampling. Combined stage specific sample sizes for the two designs were 135 primary sampling units (covering 108 separate localities), 1,290 secondary sampling Units, and 1,290 segments. Here, the number of separate primary areas is less than the sum of the number of primary sampling units in the two national primary samples since units from some of the large Standard Metropolitan Statistical Areas (SMSAS) were selected in both samples. Selection procedures for the fourth stage included a disproportionate sampling scheme to obtain a target of 3,500 uninsured households. IV. Survey Design and Content As noted, about 14,000 households participated in six separate rounds of interviews during 1977 and early 1978. The first interviews began in mid January 1977; subsequent rounds of interviews were conducted at intervals of about three months. The first, second, and fifth rounds of interviews were 124 conducted in person, as were about 20 percent of the third and fourth rounds and about half of the sixth round; the remainder were conducted by telephone. During each of the first five rounds of interviews, information was obtained on use of medical services, charges for services and sources of payment, numbers and types of disability days, and status of health insurance coverage. Data collected during the first interview covered the period from January 1, 1977, through the date of interview. Data collected during the second, third, and fourth rounds covered the period from the immediately preceding interview through the date of the current interview. The fifth interview covered the period from the previous interview through December 31, 1977. Beginning in the second round of interviews and continuing through the fifth, the household respondent was asked to review a computer-generated summary of data previously reported on health care services received and costs. This review permitted a check for accuracy and completeness and provided the necessary information to check continuity among the interview rounds for such data as health insurance coverage and charges for multiple services. The sixth round of interviews consisted of a series of supplemental questions covering limitations of activity, status of income tax filing, and the amount of itemized medical deductions. Supplemental questions also were asked during the second through fifth round interviews. These questions covered employment, health insurance, access to health care, barriers to care, ethnicity, and income and assets. In addition to answering questions, each survey participant was asked to sign a permission form so that each physician or facility that had been reported as providing medical care during 1977 could release information about the patient. In cases where a person had not reported receiving medical care in 1977 from his usual source of medical care, a permission form for his usual source of medical care was requested. Persons with health insurance policies were asked to sign a permission form authorizing release of information by the employer, union group, or insurance company. When employed persons reported no health insurance coverage, they were asked to sign a permission form authorizing the employer to provide information about the insurance coverage that was available. These forms were collected at various times during the survey and provided data which was the basis for the subsequent surveys of medical providers and health insurers. V. Response Rates Data were obtained for approximately 91 percent of eligible households in the first interview and 82 percent by the fifth interview. Of 38,815 participants in the NMCES, 4146, or 10.7 percent failed to respond for the entire time period of 1977 for which they were eligible to respond. For example, a person could have refused participation after initially cooperating -in the first interview by not responding for the remainder of the interviews. Similarly, the inability to reestablish contact with a participant after change of residence would result in this type of nonresponse. This problem of partial nonresponse is not limited or unique to the NMCES, but characteristic of national panel surveys in general. 125 VI. Evaluation Component The NMCES used several methodological innovations to insure data reliability. During each round of interviews, respondents were asked to report the diagnosis, total charge and sources of payment for each inpatient hospital stay, medical provider visit, dental visit, prescription drug, or purchase of eyeglasses or other medical equipment. In addition, respondents were asked to provide information about their health insurance coverage. Data on health care use and expenditures were updated each round through the use of a computerized summary of the information reported in the previous interview. Respondents were asked to review this information and make any needed additions or corrections. In particular, the summary was expected to allow respondents a means to provide more complete charge and payment data at a later date if it was unknown at the time of the interview. All respondents were asked to complete the summary. Approximately 32 percent of household survey respondents were also included in the medical provider survey. The medical provider survey (MPS) was a record check or verification procedure to obtain expenditure and diagnostic data from physicians and hospitals who treated a sample of household respondents during the year. Thus, for each person in the household survey the data obtained from the questionnaire was checked in a subsequent interview through the summary mechanism and in about a third of the cases, subjected to verification through the MPS. In addition, household data on health insurance coverage was verified through the Health Insurance/Employer Survey (HIES) which collected, for each private health insurance plan reported in the household survey, data from employers, insurance carriers or other insuring organizations. VII. Data Products and Analysis NCHSR has developed National Medical Care Expenditures Survey data files and documentation for public use. As of spring 1985, over 100 different research studies based on NMCES data had been published. A detailed Annotated Bibliography of Studies from the National Medical Care Expenditure Survey is available from the National Center for Health Services Research. 126 CASE STUDY 10 NATIONAL MEDICAL CARE UTILIZATION AND EXPENDITURE SURVEY I. Purpose The National Medical Care Utilization and Expenditure Survey (NMCUES) was designed to collect data on health, access to and use of medical services, charges and sources of payment for medical services, and health insurance coverage for the U.S civilian noninstitutionalized population during 1980. NMCUES was developed from a series of surveys concerning health, health care, and expenses for health carp. However, NMCUES drew most heavily from two surveys -- the National Health Interview Survey (HIS) and the National Medical Care Expenditure Survey (NMCES). The HIS is a continuing survey that began in 1957 and is conducted by the National Center for Health Statistics (NCHS). Its primary purpose is to collect information on illness, disability, and use of medical care. Although some medical expenditure and insurance information has been collected in the HIS, a cross- sectional survey design was inefficient for obtaining complete and accurate Information of this type. It was concluded that a panel survey procedure would be required, and a pilot survey was conducted for the NCHS by the Johns Hopkins University Health Services Research and Development Center and by Westat Research, in 1975-76. Based on information obtained during the pilot study, the National Center for Health Services Research (NCHSR) and NCHS cosponsored the National Medical Care Expenditure Survey in 1977 - 78. This was a panel survey for which households were interviewed six times to obtain data for 1977. NMCUES was similar to the NNCES in survey design and questionnaire wording, to allow analysts of change during the 3 years between 1977 and 1980. Both NMCUES and NMCES are similar to the HIS in terms of question wording in areas common to all three surveys. However, each survey is different with special emphasis on different areas. Together they provide extensive information on illness, disability, use of medical care, costs of medical care, sources of payment for medical care, and health insurance coverage at two points in tine. II. Sponsors NMCUES was cosponsored by NCHS and the Health Care Financing Administration (HCFA). Data collection was provided under contract by the Research Triangle Institute (RTI) of Research Triangle Park, North Carolina, and its subcontractors, National Opinion Research Center (NORC) of Chicago, Illinois, and SysteMetrics, Inc., of Santa Barbara, California. The contract was awarded in September, 1974. III. Sample Design NMCUES utilized two frames, the first to provide a national household sample and the second to provide a State Medicaid household sample. The process of selecting each sample was different, and is described separately. 127 A. The National Household Sample: The NMCUES sample of dwelling units is derived from two independently selected national samples; one provided by RTI and the other by NORC. The sample designs used by RTI and NORC are quite similar with respect to principal design features. Both can be characterized as self-weighting, stratified, multistage area probability designs. The principal differences between the two designs are the type of stratification variables and the specific definitions of sampling units at each stage. B. The State Medicaid Household Sample: The November, 1979 Medicaid eligibility files in California, Michigan, New York and Texas Were used as frames to select a sample of cases for the State Medicaid household component of the survey. A case generally consisted of all members of a family receiving Medicaid within the same category of aid. The State aid categories were collapsed into three or four strata, depending on the State. These were: (1) aid to the blind and disabled; (2) aid to the elderly (those with Supplementary Security Income); (3) Aid to Families With Dependent Children (AFDC);and (4) State only aid in California, Michigan, and New York, which provided some Medicaid coverage without Federal reimbursement. Cases in other Federal aid categories were excluded from the target population because the counts were too few to permit separate stratification. Approximately equal numbers of cases were selected from each stratum, and cases were clustered by zip codes for ease of interviewing. The lack of a central automated eligibility file in New York State (outside of the five New York City boroughs and a few other counties) required selection of counties before stratification. Within many of these counties, the lack of automation also required cases to be selected without consideration of zip codes. C. Links to Administrative Records: In addition to the data collected during interviews with sample households, another phase of data collection occurred after the final round of household interviewing was completed. Medicaid and Medicare numbers provided by the household were used to extract data from the Medicaid files of the Federal government. Data from the administrative records were merged with the household data to increase the analysis capabilities of the data. IV. Survey Design and Content A. Design 1. Respondent Rules -- The respondent for the interview was required to be a household member, 17 years of age or older. A non-house- hold proxy respondent was permitted only if all eligible household members were unable to respond because of health, language, Or mental condition. 128 2. Following Movers -- The rules for following movers were slightly different for the national household samples and the State Med4caid sample. First, for the national household survey all persons living in the housing units or group quarters at the tine of the first interview contact became part of the sample. Unmarried students 17 - 22 years of age who lived away from home were included in the sample if the - parent or guardian was included in the sample. In addition, persons who died or were institutionalized between January 1st and the date of first interview were included in the sample if they were related to persons living in the sampled housing units or group quarters. All of these persons were considered "key" persons, and data were collected for them for the full 12 months or 1980 or for the proportion of time they were part of the U.S. civilian noninstitutionalized population. In addition, babies born to key persons were also considered key persons, and data were collected for them from the time of birth. Relatives from outside the original population (i.e., institutionalized in the Armed Forces, or outside the United States between January 1 and the first interview) who moved in with key persons after the first interview also were considered key persons, and data were collected for then from the time that they joined the key person. Relatives who moved in with key persons but were part of the civilian noninstitutionalized population on January 1, 1980, were classified as "non-key" persons. Data were collected for,non-key persons for the time that they lived with a key person. Because non-key persons had a chance of selection in the initial sample, their data will not be used for general analysis. However, data for non-key persons are used for family analysis because they do contribute to the family's utilization of and expenditures for health care during the time that they are a part of the family. For the State Medicaid sample, interviewers obtained information for each eligible member of each case. Case members who d4ed before January 1, 1980, or who were continuously institutionalized between January 1, 1980 and the first interviewer contact, were excluded from the survey. Any related person living with a case member when the interviewer contacted the Household also was designated a key person, and was tracked for the complete year. In addition, babies born to key persons were considered key persons, and data were collected for them from the time of birth. Relatives outside the U.S. noninstitutionalized population between January 1 and the date of the first interview who moved in with a key person after the first interview also were considered key persons. Data were collected for them for the remainder of 1980. Persons who 129 were part of the U.S. noninstitutionalized population on January 1, 1980 and who moved in with a key person after the first interview, were classified as non-key persons; data were collected only for the time that non-key persons lived with a key person. These non- key persons are included only in family analysis. 3. Weighting -- For the analysis of NMCUES data, sample weights are required to compensate for unequal probabilities of selection, to adjust for the potentially biasing effects of failure to obtain data from some persons or households (i.p., nonresponse), and failure to cover some portions of the population because the sampling frame did not include them (i.e., undercoverage). Basic Sample Design Wieghts' -- Development of weights reflecting the sample design of NMCUES was the first step in the development of weights for each person in the survey. The basic sample weight for a dwellIng unit is the product of four weight components which correspond to the four stages of sample selection. Each of the four weight components is the inverse of the probability of selection at that stage (when sampling was without replacement), or the inverse of the expected number of selections (when sampling was with replacement and multiple selections of the sample unit were possible). - Two Sample Adjustment Factor -- As previously described, the NMCUES sample is comprised of two independently selected samples. Each Sample, together with its basic sample design weights, yields independent unbiased estimates of population parameters. As the two NMCES samples were of approximately equal size, a simple average of the two independent estimators was used for the combined sample estimator. This is equivalent to computing an adjusted basic sample design weight by dividing each basic sample design weight by two. In the subsequent discussion, only the combined sample design weights are considered. Ratio Adjustment (Household Level) -- The basic sampling weights were adjusted decrease sampling variation and to compensate for household level nonresponse and undercoverage. In total there were 63 ratio adjustment cells which were formed by cross-classify4ng race, age, and type of household head and size of household. Estimates from the 1980 CPS were used for population controls. - Ratio Adjustment (Person Level) -- The household level adjusted weights were further ratio adjusted at the person level. A total of 59 ratio adjustment cells (based on age, race and sex) were utilized. Population controls, which were provided by the U.S. (Census bureau, were based on projections from the 1980 Census. 130 4. Interview Schedule The sample dwelling units were interviewed at approximately 3 month intervals beginning in February, 1980 and ending March, 1981. The core questionnaire was administered during each of the five interview rounds to collect data on health, health care, health care charges, sources of payment, and health insurance coverage A summary of responses was used to update information reported in previous rounds. Supplements to the core questionnaire were used during the first, third, and fifth interview rounds to collect data that did not change during the year, or that were needed only once. b. Interview Mode Approximately 80 percent of the third and fourth round interviews were conduct by telephone; all remaining interviews were conducted in person. 6. Survey Costs The basic survey design and data collection contract with RTI and NORC cost approximately $18.9 million dollar. B. Content: 1. Core and Intermittent Questions -- The repetitive core of questions for NMCUES included health insurance coverage episodes of illness, the number of bed days, restricted activity days, hospital admissions, physician and dental visits, other medical care encounters, and purchase of prescribed medicine. For each contact with the medical care system, data were obtained on the nature of the health conditions, characteristics of the provider, services provided, charges, sources, and amounts of payment. Questions asked only once included data on access to medical care services, limitation of activities, occupation, income, and other sociodemographic characteristic. 2. Cross-wave Controls Collection of data from the households was facilitated by the use of a calendar and a summary. At the time of the first interval, the household respondent was given a calendar on which to record information about health problems and health services utilization, and to assemble physician and other provider bills between interviews. Following each household interview, information about health provider contracts and the payment of charges associated with them was used to generate a computer summary of information provided. This summary was then printed out in a simple format and mailed to the household for review of its accuracy and completeness prior to the next interview. At the subsequent interview, the interviewers reviewed this information with the household respondent to ensure accuracy and to obtain information not available during a previous interview. V. Response A. Survey Nonresponse Response rates for households and persons in the NMCUES were high, with approximately 90 percent of the sample households agreeing to participate in the survey, and approximately 94 percent of the individuals in the participating house 131 holds supplying information. Even though the overall response rates are high, survey based estimates of means and proportions may be biased if nonrespondents tend to have different health care experiences than respondents, or of there is a substantial response rate differential across subgroups of the target population. Furthermore, annual totals will tend to be underestimated unless allowance is made for the loss of data cue to nonresponse. Two methods commonly used to compensate for survey nonresponse are data imputation and,the adjustment of sampling weights. For NMCUES, data imputation was used to compensate for attrition and for item nonresponse, and weight adjustment was used to compensate for total nonresponse. The calculations of the weight adjustment factors were discussed previously in the section on sampling weights. 1. Attrition Imputation -- A special form of the sequential hot deck imputation method was used for attrition Amputation. First, each sample person with incomplete annual data (referred to as a "recipient") was linked to a sample person with similar demographic and socioeconomic characteristics who had complete annual data (referred to as a "donor"). Secondly, the time periods for which the recipient had missing data were divided into two categories: Imputed eligible days and imputed ineligible days. The imputed eligible days were those days for which the donor was eligible, in scope) and the imputed ineligible days were those days for which the donor was ineligible (i.e., out of scope). The donor's medical care experiences such as medical,provider visits, dental visits, hospital stays, etc., during the imputed eligible days were imputed into the recipient's record for those days. Finally, the results of the attrition imputation were used to make the final determination of a person's respondent status. If more than two-thirds of the person's total eligible days (both reported and imputed) were imputed, then the person was considered to be a total nonrespondent and the data for the person was removed from the data file. 2. Item Nonresponse and Imputation -- Among persons who are classified as respondents, there is still the possibility that they may fail to provide information for some or many items in the questionnaire. In the NMCUES, item nonresponse was particularly a problem for expenditures for health care, income, and other sensitive topics. The extent of missing data varied by question, and imputation for all items in the data file would have been expensive. Imputations were made for missing data on key demographic, economic, and expenditure items across the five data files in the Public Use Data Tape. Table 1 (page 13) illustrates the extent of the item nonresponse problem for selected survey measures which received imputations in the four data files used in this report. Demographic items tend to require the least amount of imputation, some at insignificant levels such as for age, sex, and education. Income items had higher levels of nonresponse, and for total personal income, which is a cumulation of all earned income and 11 sources of unearned income, nearly one-third of the persons required imputation for at least one component. The bed disability days, work loss days, and cut down days have levels of imputation that are intermediate between the 132 demographic and income items. The highest levels of imputation occurred for the important charge items on to various visit, hospital stay, and medical expenses files. Total charges for medic visits, hospital stays, and prescribed medicines and other medical expense records were imputed for 25.9, 36.3, and 19.4 percent of the events, respectively. Among the source of payment data, the imputation rates for the source of payment were small, but the rates for the amount paid by the first source of payment was genera subject to high rates of imputation. Nights hospitalized on the hospital stay file was imputed at a rate comparable to the first source of payment. The methods used to impute for missing items were diverse and tailored to the measure requiring imputation. Three types of imputation predominate: Editing or logical amputations; a sequential hot deck; and a weighted sequential hot deck. The imputation process will be described for two items t o illustrate the nature of imputation for the NMCUES. For Hispanic Origin, two different imputation procedures were used; logical and sequential hot deck. Since Hispanic Origin was not recorded during the interview for children under 17 years of age, a logical Amputation was made by assigning the Hispanic Origin of the head of the household to the child. For the remaining cases which were not assigned a value by this procedure, the data were grouped into classes by race of the head of the house-hold, and within classes the data were sorted by household identification number, primary sampling unit, and segment. An unweighted sequential hot deck was used to impute values of Hispanic Origin for the remaining cases with missing values. The imputations for medical visit total charge were made after extensive edit, had been done to eliminate as many inconsistencies as possible between sources of payment data and total charge. The medical visit records were then separated into three types: Emergency room, hospital outpatient department, and doctor visit Within each type, the records were classed and sorted by several measures which differed across visit types prior to a weighted hot deck imputation. For example, for doctor visits the records were classified by reason for visit, type of doctor seen, whether work was done by a physician, and age of the individual. Within the groups formed by these classing variables, the records were then sorted by type of insurance coverage and the month of visit. The weighted hot deck procedure was then used to impute for missing total charge, sources of payment, and sources of payment amounts for the classified and sorted data file. Since amputations were made for missing items for a large number of the important items in the NMCUES, they can be expected to influence the results of the survey in several ways. In general, the weighted hot deck is expected to preserve the means of the nonmissing observations when those means are for the total sample or classes within which amputations were made. However, means for other, subgroups, particularly small subgroups, may be changed substantially by imputation. In addition, sampling variances can be substantially underestimated when impute values are used in the estimation process. For a variable with one-quarter of its values imputed, for instance, sampling variances based on all cases will be based on one-third more values than were actually collected in the survey for the given item. That is, the variance would be too small by a factor of one-third, at least. Finally, the strength of relationships between measures which received imputations can be substantially attenuated by the imputation. 133 VI.Analysis and Evaluation Since 1980 NCHS has awarded a number of contracts for the review and analysis of NMCUES data to evaluate the quality of the data and the data collection and processing methods. This includes a contract with Westat (of Rockville, Maryland) to evaluate NMCUES data collection,and data processing and a series of 3 contracts with the University of Michigan to analyze findings related to physicians charges, patient expenditures and sources of payment. Another contract, with Applied Management Sciences, examined family characteristics and expenditures for healthcare. VII. Data Products Data from the NMCUES are available with documentation on public use tapes from the National Technical Information Service, a division of the Department of Commerce in Springfield, Virginia. Additional information concerning the public use tapes is available from the Utilization and Expenditure Statistics branch, NCHS. Findings from the survey were presented in official publications primarily from the government's Public Health Service and Health Care Financing Administration 1983 - 85. A number of analyses of NMCUES appeared in a Working Paper series published by the NCHS which now has over 20 titles, as well as in professional journals dealing with public administration and public health. 134 Table 1. Percent of Data Imputed for Selected Survey Items in Four of the NMCUES Public Use Data Fi1es Tape Location Survey Item Percent Imputed Person File (n = 17,123) Age 0.1 Race 20.1 (1) Sex 0.1 Highest Grade Attended 0.1 Perceived Health Status 0.8 Functional Limitation Score 3.2 Number of Bed Disability Days 7.9 Number of Work Loss Days 8.9 Number of Cut Down Days 8.2 Wages, Salary, Business Income 9.7 Pension Income 3.5 Interest Income 121.6 Total Personal Income 30.4(2) Medical Visit File (n = 86,594) Total Charge 25.9 First Source of Payment 1.8 First source of Payment Amount 11.6 Hospital Stay File (n = 2,946) Nights Hospitalized 3.1 Total Charge 36.3 First Source of Payment 2.2 First Source of Payment Amount 17.6 Medical Expenses File (n = 58,544), Total Charge 19.4 First Source of Payment 2.9 First Source of Payment Amount 10.0 (1) Race for Children under 14 imputed from race of head (2) Cumulative across 12 types of income 135 CASE STUDY 11 LONGITUDINAL ESTABLISHMENT DATA FILE Historically the economist has relied upon aggregate economic information from various sources (including the Census of Manufactures and Annual Survey of Manufactures (ASM) programs) to investigate the changing structure of the manufacturing sector of the United States economy. It has not been possible to observe the variations in behavior among establishments (plants) or to determine how changes in the behavior of individual establishments affected the enterprise (firm) or the aggregate statistical totals. The Census Bureau has developed a Longitudinal Establishment Data (LED) file which, when coupled with recent advances in econometric computer software, makes possible a wide range of empirical analysis at the manufacturing establishment level. The LED file was developed in cooperation with the National Science Foundation under the general direction of Nancy and Richard. Ruggles of Yale University. The LED file is a time series of economic variables collected from manufacturing establishments in the Census of Manufactures and Annual Survey of Manufactures programs. The LED file contains establishment level identifying information; basic information on the factors of production (inputs, such as levels of capital, labor, energy and materials) and the products produced (outputs); and other basic economic information used to define the operations of a manufacturing plant. The LED file resides in a random access database environment which facilitates immediate access to individual data values. History The ASM program was initiated in 1949 and provides detailed economic information on the functioning of manufacturing plants in intercensal years. Since the inception of the ASM program the Census Bureau has understood the potential of linking establishment records across ASM survey years to create a longitudinal micro level data file suitable to perform time series analysis. The Ruggles' were particularly interested in developing such a file for various types of macroeconomic studies. The first real attempt at creating such a file was undertaken in the late 1950's using the 1954 Census of Manufactures as a starting point. This first attempt tried to match establishments across time using survey identification numbers as keys. While a significant portion of the establishments had retained their identification numbers for several years, many identification numbers had been changed and no audit trail was maintained. There was really no way of linking such establishments except by laborious search of the name and address records in the mailing directory. In those days, shuttle forms were used and thus the linkage of identification numbers in different years was not critical in order to measure year-to-year change in manufacturing establishments. This first attempt at a matching of identification numbers required a labor intensive effort to ensure accurate matches. This experience led to modifications in the ASM processing that placed greater responsibility on the directory to document identification number changes and to link old and new identification numbers. It also led to the introduction of the concept of the permanent plant number that would be assigned to an establishment throughout its life in the ASM program. This permanent identification number became critical not only to the directory controls but also to new methods of editing and tabulation. 137 Considerable staff and computer time were expended on this first effort and a large segment of the ASM file was successfully matched for the years 1954-1962. However, since the computer record for many establishments did not include all corrections resulting from the survey review, and because many nonmatches were left unresolved, the file was not developed to the extent necessary to be usable for a wide variety of longitudinal studies. The first effort at creating a time series file of establishment level microdata was discontinued in 1968 because of budget restrictions. However, the experience gained from the first effort added significantly to the directory, editing and tabulation techniques used in the ASM; specifically the computer edit of the Census and ASM programs were modified to incorporate more year-to- year analysis. During the 1970's several major advances were made at the Bureau which made it possible to renew the effort to develop a longitudinal establishment file. First, the Industrial Directory was started in 1972 which solved the problems of linkage of identification numbers due to changes in ownership. Second, the establishment correction system introduced into the Census and ASM programs in 1979 assures that all corrections made by the staff during the review of the data are applied to the data records. Prior to 1975, budgetary constraints prevented the complete correction of the computer data files, although the corrected data were included in the official published statistics. The current effort to develop the LED file was undertaken as a joint effort by the Census Bureau and Richard and Nancy Ruggles of Yale University, with funding provided by the NSF and the Small Business Administration. The Census Bureau has created a longitudinal data file of individual manufacturing establishment data from the Census of Manufactures and ASM for the years 1972 to 1981. This process required the linkage of establishment level records based upon identification numbers. This linkage process was complicated by the numerous plant closings, plant openings, mergers and acquisitions that transpired during the decade covered by the file. A computer match was performed to link establishment records over time, linkage problems were resolved by the data analysts so that a consistent series of economic surveys is available for each establishment in operation during the period covered. The linked data were reformatted into a data structure suitable for such a file and extraction routines were developed so that data can be removed from the file. Contents of the File The basic unit of collection for the Census of Manufactures and the ASM is the manufacturing establishment. Thus the establishment is the basic unit of data storage in the LED file. An establishment is defined as a single,physical location engaged in one of the categories of industrial activity in the Standard Industrial Classification (SIC) system. The SIC system is used in the classification of manufacturing establishments by type of activity in which they are engaged; it facilitates the collection, tabulation, presentation and analysis of census data relating to establishments. The data are stored as a time sequence of survey responses for establishments rather than as a time series of annual observations for variables. The data are sorted by a permanent establishment identification number and survey year. 138 The data for a particular year are stored in modular sets of fixed length records; data for a module (a set of variables) have a consistent format for all years. The variables available from the LED file are presented in Table 1, the LED Directory. As this table indicates, basic economic information on the factors of production (inputs) such as employment, payrolls, supplementary labor costs, worker hours, cost of fuels and electricity, cost of materials, capital expenditures, rental payments, inventories and on the products produced (outputs), such as value of shipments and value added, are available for all years. In recent years, a number of new items have been added, including the consumption of specific types of fuels, methods of valuation or inventories, purchases of used structures and machinery, retirements, and depreciation. The detailed information obtained in census years on materials consumed and on products shipped are not available from the ASM, thus a continuous time series is not available for those variables. Methodological Problems Data Comparability through Time: The main objective of survey processing is to identify "significant errors", i.e., those that affect the quality of the aggregate data or the test for confidentiality. We cannot afford the cost of cleaning up "insignificant" data errors. Therefore, we do not always insist on complete and correct data for each establishment, even in a sample, and rely instead on our computer edit to maintain the completeness of the record, to "estimate" data for establishments that fail to report, and to identify "significant" errors (edit failures) that are referred to the analysts for review. This means that some data errors remain in the records of the individual establishments. It should be noted that data "flags" included in the longitudinal file will indicate which cells have been computer changed or analyst corrected. Most importantly, because of cost, we have concentrated on year-to-year comparisons of establishment data. Our computer edit has been designed to work with only two periods of data; current year and previous year. Our aggregate review focuses on two years of data, current and previous, although trends are also considered. For economic research purposes, where micro data for several years are needed this type of editing and review may not be sufficient. Different problems will come into focus when establishment data are edited and reviewed over a long-period of time as compared to using only two years. Another factor that affects data comparability over time involves the errors that are identified during the survey processing, but which are not carried back to the file because of cost considerations. As noted earlier, this situation was virtually eliminated with the introduction of an establishment correction system for the 1975 ASM. For the, 1972 Census and the 1973 and 1974 ASM, this system was not available, but efforts were taken to assure that most of the corrections were carried back to the file. Therefore for these years a tabulation of the computer file will yield results very close to the publication totals. Data comparability over time may also be affected by two other factors. The first involves a change in the definition of an individual item. An example of this will occur for the 1982 Census of Manufactures in regard to inventories. 139 Prior to 1982, information on the book value of inventories was Collected. Investigations of methods used by individual companies to compile inventories indicates that the best way to obtain consistent data among different companies and even among individual establishments of the same company is to request LIFO (last-in- first-out) inventories before the application of the LIFO adjustment or reserve. Therefore, the inventories inquiry has been revised for 1982 ,to collect data on a pre-LIFO basis (i.e., gross value before any LIFO reserve or adjustment). However, since we will be requesting additional information including the amount of the LIFO reserve, we will be able to "estimate" book value for 1982. The second factor that would affect data comparability involves modification of the computer editing procedure used for a particular item. An example occurred in the 1977 census when the addition of retirements and detailed capital expenditure items to the report form resulted in a complete change of the editing procedure used for the assets-expenditures-retirements complex. Assets data continue to be collected as in the past, but the new computer editing procedure probably resulted in a "break" in the series for a few establishments whose assets data were edited differently for 1972 through 1976 as compared to 1977 and subsequent years. Availability of "Processed" rather than "Raw" data: In analysis of an establishment file, some researchers feel that the actual data reported by the respondent are preferable to the data that have been edited and changed (without verification by the company). However, the data files used for the development of the time series file include a mixture of "raw" (originally reported) and computer-corrected data. The "raw" data are no longer available for all establishments. Therefore, researchers who advocate economic research based only on "raw" microdata will find the Census/ASM LED to be of limited use. We have already noted that data "flags" included in the longitudinal file will indicate which cells have been computer changed or analyst corrected. As a result, researchers may choose to isolate only the "raw" microdata that remain unchanged as a result of Census Bureau processing procedures. Disclosure The last problem to be discussed, and the most complex, involves disclosure implications. Data collected by the Bureau of the Census are protected by Title 13 of the U.S. Code from disclosure to outside parties. All tabulations and analysis of longitudinal data must be analyzed to ensure that no individually identifiable confidential data are released to outside users. Bureau of the Census policy also requires that the Center for Economic Studies prevent actual estimation or close approximation of individual confidential data from released statistics. This is accomplished by applying the Census Bureau's respondent and concentration rules, which may require suppression of individual data cells. Additional suppression of nondisclosure cells may be required in cross-tabulations to avoid complementary or indirect disclosure of confidential data. After a request for tabulation or analysis is received by the Center, a comprehensive analysis of possible disclosure of sensitive information will be performed . The user will be notified of Possible disclosure which would require 140 the suppression of information. Due to the complex nature of the LED file, each disclosure analysis will be handled on a case by case basis. Under no circumstances will the Bureau release names or addresses of establishments in the file. Also the Bureau will not release microdata in any format which would allow identification of individual establishments. The results of each project must be carefully scrutinized in terms of disclosure implications before the data can be released to the researchers. The effects of ownership changes, industry changes, corrections made as a result of reviewing the establishment data, and so forth, must be taken into consideration. Furthermore, if the time-series data are subject to regression analysis or other mathematical analysis, interesting questions are raised on what information can be released. Finally, the results of each project must be compared against the results of previous studies in order to avoid complementary disclosure problems. This is quite an undertaking, and, at present, a systematic approach to handling disclosure problems has not been developed. How will the File be Used Users of the LED file will work through the staff of the Center for Economic Studies (CES). A major purpose of the CES is to make industrial data available to the data user community of economic policymakers and researchers to facilitate analysis and research. The result of that analysis and research will then help the Bureau to improve its economic measurement programs. The Census confidentiality policies and the U.S. Code limit direct access to individual establishment data to Census employees who have sworn to protect their confidentiality. This regulation precludes direct access to the LED data by outside researchers only sworn Census employees will have direct access to the LED file. The CES will act as the interface between the data user community and the LED file by processing requests by outside researchers for tabulations and analyses of the LED file. The CES is creating a computer environment that will permit low-cost expeditious processing of user requests. It will be possible for an outside analyst to request cross-tabulations of aggregate statistics, estimations of econometric models, and other economic and statistical relationships based on the establishment level data. These tasks will be performed on a cost-reimbursable basis. The types of tasks that can be performed using the LED file include: 1. Analysis of a wide range of issues from the field of industrial organization including diversification, concentration, ownership patterns and changes, and monopolistic and oligopolistic industries. 2. Analysis of productivity, technological change and efficiency and their diffusion within and across establishments, enterprises and industries. 3. A wide range of descriptive statistics such as cross- tabulation of important variables (productivity value added, wage rates) by size of establishment or enterprise, by industry or by geographic area. 141 4. A wide range of studies of various economic surveys by comparing detail and summary statistics across surveys. 5. Analysis of the sources and nature of productivity growth, including geographic, size and industry dimensions. 6. Analysis of geographic patterns in input markets, especially labor and energy markets. 7. Analysis of energy use in manufacturing establishments. 8. Analysis of the geographic dimensions of, for example, labor and energy markets. The data user/research community benefits by analysis of a rich longitudinal data base for manufacturing establishments and (through integration with other economic survey results) whole enterprises. The Bureau's economic survey programs will benefit from validation and evaluation studies through time and across economic surveys. Feedback on the scope of the surveys, uses of the data, and data anomalies discovered during analysis will improve both the content and the quality of the survey data and statistical products based on theory. Also generalized data manipulation and analysis software produced for analytical uses of the file can be made available for use in the economics division for their use in production processes. 142 Click HERE for graphic. 143 Click HERE for graphic. 144 Click HERE for graphic. 145 CASE STUDY 12 STATISTICS OF INCOME PROGRAM I. Purpose The internal Revenue Service, in addition to its primary mission of enforcing the Federal tax laws, is also charged with publishing statistics on the operation of the tax laws. The data, based on tax returns, are released in a series of reports called Statistics of Income (SOI). The SOI reports from the very beginning (1916) have been used extensively for tax research and for estimating revenue, especially,by officials in the Department of the Treasury. The main emphasis of the annual statistics has always been individual and corporation income tax data. Other subjects based on other types of returns for which data have been tabulated either annually or periodically have been partnerships, estates and gifts, fiduciaries, farmers' cooperatives, and foundations and other tax exempt organizations. Data are also published on the international income and taxes of U.S. persons and corporations. Traditionally, the SOI Program has been based on cross- sectional samples. However, these statistics told very little about the relationships between events that were being described. For example, was it the people who moved who achieved increases in income? Did people whose tax rates went down give more or less to charitable organizations? Only with longitudinal studies has IRS been able to relate status at one point in time to status at another. This is done by focusing on specified observational units in one Year, and following their status through successive (or preceding) years. In addition, when dealing with attitudes, such as the response of taxpayers to tax law and economic changes, longitudinal samples are as close as SOI can come to performing controlled experiments. Most of the longitudinal studies have been panel studies. The same variables are measured for the same observational units at different periods in time. This is done by creating a file of individual tax return data for a group of taxpayers for each of a succession of years. The IRS has also done transtemporal studies, in which different variables have been measured in different years for the same taxpayers. An example would be the matching of individual income tax returns filed during a taxpayer's lifetime with the estate tax return (which indicates the taxpayer's wealth) filed after his or her death, A third type of longitudinal study is the non-identical study, in which one set of variables is measured for one set of observational units at one time, and another set of variables is measured for a related but not identical group of observational units at another. This occurs when the estate tax return of one individual is matched to the income tax returns filed in later years by his or her heirs. Because IRS is dealing with administrative files, one more set of distinctions deserves to he made. Each of the types of longitudinal studies mentioned above can be either prospective or retrospective in nature. In other words, the historical data can be built by going either backwards From a paper presented to the American Statistical Association by Robert A. Wilson and John DiPaolo, and a presentation to the Joint U.S. and Canadian Conference on Tax Modelling by Peter J. Sailer. 147 or forwards in time from the point at which the sample was selected. The SOI Division has created both types of files, as well as hybrids which move in both directions. II. Sponsorship The SOI program is the responsibility of the Statistics of Income Division of the IRS Office of Returns and Information Processing. The Statistics of Income Division is responsible not only for SOI, but also for conducting special statistical studies and providing advice on sample designs for use in helping other organizations in IRS to conduct studies of their own. III. Sample Design The SOI program has the following basic character. Returns filed with the ten service centers are processed for administrative purposes to determine the correct tax liability. During processing, the returns are entered on tape for eventual posting to the IRS Master File. It is when the return records are on tape that they are designated for SOI After the returns are designated, they are subjected to additional editing and relational testing for the SOI program. A. Design Problems The first task is to identify the same observational units. In the case of individual taxpayers, this is not too difficult, at least in theory. All records are identified by social security number (SSN), and most of the electronic files are sorted in SSN order. There are many reasons, however, which can cause non-matches. Deaths (in the case of prospective studies) and births (in the case of retrospective studies) guarantee that not all records will match to a record for another year. (Births and deaths mean coming into the system or leaving the system. This leads to the phenomenon that a taxpayer can be born into the estate tax system only by dying.) Unfortunately (for the SOI program), many taxpayers show a tendency to die only temporarily, and then to be reborn a few years later. However, neither processing errors, nor births, nor deaths create as many problems as marriages. When a male in an SOI panel gets married, he will generally start filing a joint return with his wife, using his SSN as the primary SSN on the return. This means that he will still be in the panel but, in contrast to earlier years, he may well have a second persons's income and taxes mixed in with his. On the other hand, when a female gets married, she is generally lost to a panel, especially if the sample selection is performed at the service centers, where secondary SSN's are not always key-entered. No matter how much effort is made to keep all the observational units from one year to the next, the fact remains that it will not be possible to include completely comparable data items, since joint returns always combine data items for both taxpayers. 148 The problem of marriages is compounded when one is trying to establish a panel of corporations. While multiple marriages do occur among individuals, at least they occur serially. In the case of corporations, the frequent and cumulative merging of observational units often with units from totally unrelated industrial groupings, can wreak havoc with corporation panels. For that reason, corporation panel studies undertaken by the Statistics of Income Division have been confined to very small pilot efforts. Although setting up a panel file may be much more complicated than simply selecting a series of cross-sectional samples, panel files have one additional benefit. While the sampling variability of the estimates for each year should be about the same as they would be for a cross-sectional sample of the same size for each year, the sampling variability of the changes from one year to the next should be considerably smaller. This happens because the differences between one year and the next truly are differences, not the results of selecting different samples. IV. Survey Design and Content A. The 1967-73 Individual SOI Panel The 1967-73 panel was created by incorporating two four-digit social security number endings in each stratum of each Statistics of Income sample for those years. In other words, anybody whose SSN ended in one of those two combinations of digits was included in the larger, stratified sample selected to produce the annual Statistics of Income report. In theory, at least, this created a general-purpose panel at a very low cost. The cost of abstracting, keying, and testing important data items from selected tax returns was absorbed as part of the regular statistical processing. One problem arose because an annual 2 percent delinquency rate added tip to quite a few incomplete observational units over a seven-year period -- over 10 percent, as a matter of fact. Further complications arose because of the many tax law changes and consequent redesign of the tax forms over the 7-year period of the panel. Because of these changes, the file format changed considerably over the period, with old items being dropped and new ones added. IRS finally decided to create a completely new file format, which would work for all the years in the panel. Fields were created for all items that existed over the 7-year period, and were filled in for those years for which they existed. When the completeness of the file was evaluated, going back only one year (i.e., to 1972), returns for 11.7 percent of the taxpayers in the sample were missing. Going back another year, some of the lost taxpayers reappeared, while others dropped out, for a net loss of 18.4 percent. By the time IRS had gone back 6 years to the beginning of the panel, no returns could be found for 32.6 percent of the 1973 taxpayers. The number for which IRS did not have complete records was closer to 50 percent. In spite of its limitations, the file proved useful in studying a number of issues. B. The Capital Gains Panel Beginning with Tax Year 1973, the Statistics of Income Division began assembling "capital gains panels." These are 5-year, retrospective/prospective panels, with the base year in the middle. A highly stratified sample of Schedule D returns (Capital Gains and Losses) with sampling rates ranging from 1/48,000 149 to 1/5, is selected for the middle year. The IRS Individual Master File is then used to locate the returns for the two previous years and, eventually, for the two following years. The returns are pulled, and details on each capital transaction are edited and transcribed. C. The Estate Collation Study While a panel of Forms 1040 can provide information about the realization of capital gains, and a panel of Schedule D data can indicate what type of assets have been traded and how long they have been held, neither shows how these relate to the total wealth of the taxpayer. Wealth, in fact, is reported at most once for any given taxpayer -- on Form 706 (Estate Tax Return), by the taxpayer's estate, after he or she has died. The purpose of the SOI's estate collation studies is to establish a connection between the income and the wealth of taxpayers, and to trace the transfer of wealth (and consequent changes in income) when a taxpayer dies. This is done by matching a decedent's estate tax return first to his or her income tax returns prior to death, then to the beneficiaries' income tax returns both before and after the death. In other words, this is a hybrid of every type of longitudinal study mentioned above: a retrospective and prospective, non- identical, transtemporal panel. For the 1976 Estate Collation Study, IRS matched estate tax returns filed in 1977 with the decedent's income tax returns filed for the two previous years. In addition, IRS matched the income tax returns for nonspousal heirs to whom a bequest of $50,000 or more had been made, obtaining data for the two years before and the three years after the bequest. D. Taxpayer Migration Data This project is probably one of the largest panel studies ever undertaken. It is not done by the Internal Revenue Service, but, it involves data files that are provided by IRS to the Bureau of the Census. The Census matches every computer record of individual income tax returns filed from January through September of a given year to the previous year's record. The Census Bureau is given access to return records, among other things, to make intercensal population and income estimates, and to provide county and minor civil division level data to the Treasury Department for the Federal Revenue Sharing program. The matching of return records is in part an operational necessity. Taxpayers frequently use a business or Post Office Box address on their returns. Therefore, the Bureau persuaded IRS to put a question on the return about the exact governmental unit in which a taxpayer lives. However, this is done only once every few years -- the most recent year was 1980. Among the series of data which Census creates from these files are matrices which show from where to where the population is shifting; and county migration data which show how many taxpayers entered and left each county within a given period of time, how many exemptions they claimed, and, for some years, the amount of income for the in-migrants, the out-migrants, and the non-migrants. 150 E. Department of Defense (DOD) Salary Study The DOD Salary Study is the result of a public law passed by the U.S. Congress which requires the Department of Defense to perform an evaluation of the military pay structure at least once every four-years. Part of this study entails following the earnings of persons who leave the Armed Forces-separatees, as DOD calls them -- to learn what the "opportunity costs" are for persons who stay in the Armed Forces. The sample of separatees is chosen by DOD. New separatees are sampled each year. Once selected for the sample, the individual stays in it forever. DOD gives IRS the social security numbers of the new designees, along with codes indicating their DOD characteristics. By going to Forms 14-2 (Wage and Tax Statements), rather than to income tax returns, IRS gets only the salaries of the individuals in the sample. Because of the taxpayer's right to privacy, no identifiable data are returned to DOD. All SSN's are removed from the data before they are sent back to DOD. Furthermore, DOD supplies IRS with at least three individuals with any given combination of DOD characteristics codes, so that there will not be any way to match back to the SSN's. One of the limitations of this panel is that of missing data. There are no indicators on the Form W-2 to indicate whether a person for whom data are missing is self-employed, unemployed, retired, or dead, or whether IRS has made a processing error. At this point, there is no alternative to simply leaving these individuals out of the analysis. F. The Individual Panel Beginning with Tax Year 1979 The Tax Year 1979 sample was designed to study certain questions related to mortality and morbidity rates by occupation of taxpayer. Funds had been made available for this purpose by the Social Security Administration and the National Cancer Institute. Since future links to certain data items from the Social Security Administration's Continuous Work History Sample (CWHS) were anticipated, five SSN endings were chosen to overlap with the CWHS sample. There is now a 3-year panel of some 45,000 randomly selected tax return records, and a 4-year panel of 9,000 records. G. Corporation Tax Adjustment Study (CORTAX) This study is intended to quantify the effects of adjustments (through carrybacks of net operating losses and unused credits, IRS examination activity, etc.) to corporate tax liability after the corporation's original tax return (Form 1120 series) has been filed. By linking SOI corporate sample EIN's to their Business Master File -(BMF) accounts, SOI expects to tabulate these adjustment amounts for all tax years on the BMF extract -- usually the most recent five or so. For example, CORTAX 86 will commence in 1986 by extracting these adjustment data for Tax Years 1978 - 1982, using the Tax Year 1982 sample file of EIN's as the extract or link variables. While a significant portion of the SOI corporate on sample (like other SOI sampling frames) is already longitudinal , CORTAX will lend an additional longitudinal aspect with its five years of 151 adjustment data for each CORTAX year's record. In addition, CORTAX will show cumulative adjustment effects (and, thus, annual changes) for certain tax years over time for the longitudinal "core" of records in the SOI corporate samples. CORTAX 87 is expected to provide tax liability adjustment data for an accounting period range ending with Tax Year 1985, and may expand tabulations to include interest and penalty assessment amounts as well. Thereafter, CORTAX studies are planned for annual occurrence, and should continue to provide Treasury's Office of Tax Analysis and Congress' Joint Committee on Taxation with the supplemental data bases necessary for the development of more current and detailed tax policy/legislation analyses. V. Future Studies There is no doubt that longitudinal studies are essential to the IRS mandate to produce statistics on how the internal revenue laws are operating. A new estate collation study is being planned for 1982 decedents. In this new, improved study, wealth transferred to trusts and other estates, as well as to individuals, will be traced. One of the most ambitious plans is the study of Intergenerational Transfers of Wealth. The only time an actual accounting is available for an heir's wealth will be when that heir, in turn, passes away. This is what the study of intergenerational transfers is all about. By linking estate tax returns filed by succeeding generations of heirs a classic non- identical longitudinal study -- it is possible to study changes in the concentration of wealth during the history of the tax system, and the role intergenerational transfers of wealth have played in this process. Additional plans for the future include improved individual panel studies using data from the Individual Master File of all tax return records, including one in which the postal ZIP code will be used to trace migration patterns; Also planned are additional capital gains panels, and a panel study of large private foundations. 152 REFERENCES ANDERSON, T.W. 1957 "Maximum Likelihood Estimates for a Multivariate Normal Distribution When Some Observations are Missing." Journal of the American Statistical Association, 51, 200-203. ARTZROUNI, MARC 1980 "Tracing Respondents in Longitudinal Surveys: A Bibliographic Overview." Unpublished Ms., U.S. Bureau of the Census, Statistical Research Division. BARTHOLOMEW, D.J. 1973 Stochastic Models for Social Processes, John Wiley and Sons. BENUS, J. 1975 "Response rates and data quality." Five Thousand American Families -- Patterns of Economic Progress, Vol. III, Ann Arbor: Institute for Social Research, G.J. Duncan and J. N. Morgan (eds.). BIDERMAN, A., CANTOR, D. and REISS. A. 1982 "A Quasi-Experimental Analysis of Personal Victimization Reporting by Household Respondents in the National Crime Survey." Paper prepared for the Joint Statistical Meetings of the American Statistical Association, Cincinnati, Ohio. BISHOP, YVONNE M.M.; FIENBERG, STEPHEN E.; HOLLAND, PAUL W.; 1975 Discrete Multivariate Analysis, The MIT Press. BLALOCK, H.M. 1970 Causal Models in the Social Sciences, Chicago: Aldine. BURKHEAD, D., and CODER, J. 1985 "Gross Changes in Income Recipiency from the Survey of In come and Program Participation". Proceedings of the Social Statistics section, American Statistical Association. BYE, BARRY V. and SCHECHTER, EVAN S. 1980 "Estimating Response Variance from Latent Markov Models: An Application to Self Reported Disability Status", ORS Staff Paper no. 37. BYE, BARRY V. and SCHECHTER, EVAN S. 1986 "A Latent Markov Model Approach to the Estimation of Response Errors in Multiwave Panel Data", Journal of the American Statistical Association, forthcoming, June, 1986. CAMPBELL, RICHARD T. and MUTRAN, ELIZABETH, 1982 "Analyzing Panel Data in Studies of Aging, Research on Aging, Vol.4 , no. 1 , 3-41 . CITRO, C.F. 1985 "Alternative Definitions of Longitudinal Households and Poverty Status in the ISDP", Proceedings of the Survey Methods Research Section, American Statistical Association. 153 CLOGG, CLIFFORD C. 1979 "Latent Structure Models of Mobility" The Pennsylvania State University. CODER, J., and FELDMAN, A.; 1984 "Early Indications of Item Nonresponse on the Survey of Income and Program Participation", in Proceedings of the Survey Methods Research Section, American Statistical Association. COLEMAN, JAMES 1981 Longitudinal Data Analysis, Basic Books Inc. COOK, MARTIN A., and ALEXANDER, KARL L. 1982 "design & Substance in Educational Research" Adolescent Attainment, A Case in Point" in Sociology of Education: 53 no. 4: 197-202. COX, B., and BONHAM, G. 1983 "Sources and Solutions for Missing Data in the NMCUES," in Proceedings of the Survey Research Methods Section, American Statistical Association, Washington, D.C. COX, B. and COHEN, S. 1985 Methodological Issues for Health Care Surveys. Marcel Dekker, New York. DAVID, M. (ed.); 1983 Technical, Conceptual and Administrative Lessons of the Income Survey Development Program. New York, Social Science Research Council. DAVID, M. and LITTLE, R., and McMILLEN, D. 1983 "Weighting Adjustments for Nonresponse in Panel Surveys." Unpublished working paper, U.S. Bureau of the Census. DAVID, M., and LITTLE, R. 1983 "Concepts and Strategies for Imputation of ISDP and SIPP." Unpublished working paper, U.S. Bureau of the Census. DUNCAN, G.J., JUSTER, F.T. and MORGAN J.N. 1982 "The role of panel studies in a world of scarce research resources." Paper prepared for Social Science Research Council Conference on Designing Research with Scarce Resources, Washington, D.C. DUNCAN, G. & KALTON, G. 1985 Issues of Design and Analysis of Surveys Across Time, paper presented to the I.S.I., August, 1985, Amsterdam. DUNTEMAN, GEORGE H. and PENG, SAMUEL S. 1977 "Some Analysis Strategies Applied to the National Longitudinal Study of the High School Class of 1972," Research Triangle Institute, ELANDT-JOHNSON, REGINA C., and JOHNSON, NORMAN L. 1980 Survival Models and Data Analysis, John Wiley and Sons. ERNST, I., HUBBLE, D., and JUDKINS, D. 1984 "Longitudinal Family and Household Estimation in SIPP". Proceedings of the Survey Research Methods Section, American Statistical Association, Washington, D.C. 154 FELDMAN, A., NELSON, C., and CODER, J.; 1980 "Evaluation of Wage and Salary Income Reporting on the 1978 Income Survey Development Program Test Panel", in Proceedings of the Section on Survey Research Methods, American Statistical Association. FERBER, R., and FRANKEL, D, 1981 Evaluation of the Reliability of the Net Worth Data in the 1979 Panel: Asset Ownership on Wave 1. Prepared under contract with the Survey Research Laboratory, University of Illinois. FIENBERG, S.D., and TANUR, J.M. 1983 "The Design and Analysis of Longitudinal Surveys: Controversties and Issues of Cost and Continuity." Technical Report no. 289, Department of Statistics, Carnegie-Mellon University, Pittsburgh,,Pennsylvania. FOX, ALAN 1976 "Work Status and Income Change, 1968-72: Retirement History Study Preview," Social Security Bulletin. FRANKEL, D. 1985 Survey of Income and Program Participation: Selected Papers given at the 1985 Annual meeting of the American Statistical Associating. Las Vegas, Nevada, 1985. GINSBERG, RALPH B. 1972a "Critique of Probabilistic Models: Application of the Semi-Markov Model to Migration," Journal of Mathematical Sociology, Vol. 2, 63-82. GINSBERG, RALPH B. 1972b "Incorporating Causal Structure and Exogenous Information with Probabalistic Models" With Special Reference to Choice, Gravity, Migration and Markov Chains," Journal of Mathematical Sociology, Vol. 2, 83-103. GROVES, R. M. and KAHN, R.L. 1979 Surveys by Telephone: A National Comparison with Personal Interviews. New York: Academic Press. HAUSER, ROBERT M. 1978 "some Exploratory Methods for Modeling Mobility Tables and other Crossclassified Data", CDE Working Paper 78-19. HECKMAN, JAMES J. and SINGER, BURTON 1982 "The Identification Problem in Econometric Models for Duration Data." Advances in Econometrics, Cambridge University Press. HENNESSEY, JOHN C. 1982 "Testing the Predictive Power of a Proportional Hazards Semi- Markov Model of Postentitlement Work Histories of Disabled Male Beneficiaries", Social Security Administration ORS Working Paper no. 29. JEAN, A. and McARTHUR E. 1984 "Some data collection issues for panel surveys with application to SIPP." Proceedings of the Survey Methods Section, American Statistical Association 155 JONES. B. 1982 "Development of Sample Weights for the National Household Component of the National Medical Care Utilization and Expenditure Survey." Research Institute, Research Triangle Park, NC. RTI/1815/05-01F. J™RESKOG, KARL G., and S™RBOM, DAG 1976 "Statistical Models and Methods for Analysis of Longitudinal Data", In D.J. Aigner and A.S. Goldberger (eds.), Latent Variables in Socioeconomic Models, pp. 285-325. Amsterdam, Holland. 1978 LISREL User's Guide: Version IV, International Educational Services. 1979 Advances in Factor Analysis and Structural Equation Models, Abt Books. KALACHEK, E. 1978 "Longitudinal surveys & labor market analysis" background paper no. 6. National Commission on Employment A Unemployment Statistics, Washington-, D.C. KALTON, G., KASPRZYK, D., and SANTOS, R. 1981 "Issues of Nonresponse and Imputation in the Survey of Income and Program Participation " in Current Topics in Survey Sampling. Krewski, D, Platek, R., Rao, J.N.K. (eds), Academic Press, New-York. KALTON, G. and LEPKOWSKI, J. 1992 "Longitudinal Weighting in the ISOP? Chapter 12 in David, op. Cit. Lessons of the ISDP, D,C., SSRC. KALTON, G and LEPKOWSKI, J. 1983 "Cross-Wave Imputation," in Technical, Conceptual and Administrative Lessons of the Income Survey Development Program (ISDP). M. David, (ed.), Social Science Research Council, New York. KALTON, G., LEPKOWSKI, J., and LIN, T. 1985 "Compensating for Wave Nonresponse in the 1979 ISDP Research Panel" in Proceedings of the Survey Research Methods Section, American Statistical Association. Washington, D.C. KALTON, G., LEPKOWSKI, J., and SANTOS, R. 1981 "Longitudinal Imputation." Survey Research Center/University of Michigan, Income Survey Development Program. Unpublished report, of the ISDP. Department of Health and Human Services, Washington D.C. KASPRZYK, D., and FRANKEL, D. 1985 Survey of Income and Program Participation and Related Longitudinal Surveys: 1984; Selected Papers Given at the 1984 Annual Meeting of the American Statistical Association. Philadelphia, Pa. KASPRZYK, D., and KALTON, G. 1983 "Longitudinal Weighting in the Income Survey Development Program," in Technical. Conceptual and Administrative Lessons of the Income Survey Development Program (ISDP). M. David (ed.), Social Science Research Council, New York. 156 LAND, K.C. 1971 "On the Definition of Social Indicators", in American Sociologist. 6:322. LANDIS, RICHARD J., and KOCH, GARY G. (N.D.)"The Analysis of Categorical Data in Longitudinal Studies of Behavioral Development", (source unknown). LANDIS, RICHARD J.; STANISH, WILLIAM M.; FREEMAN, JEAN S.; KOCH, GARY G. 1976 "A Computer Program for the Generalized Chi-Square Analysis, of Categorical Data Using Weighted Least Squares (GENCAT)", Computer Programs in Biomedicine, 6: 196-231. LITTLE, R. 1984 "Survey Nonresponse Adjustments* in Proceedings of the Survey Research Methods Section, American Statistical Association,.Washington, D.C. LITTLE, R. 1985 "Nonresponse Adjustments in Longitudinal Surveys: Models for Categorical Data." Paper prepared for the meeting of the International Statistical Institute, August 1985. MARINI, M., OLSEN, A., and RUBIN, D. 1980 "Maximum-Likelihood Estimation in Panel Studies with Missing Data," in Sociological Methodology. Schuessler,, K.F. (ed.), Jossey-Bass, San Francisco. McARTHUR, E., and SHORT, K.; 1985 "The Characteristics of Sample Attrition in the Survey of Income and Program Participation" in Proceedings of the Survey Research Methods Section, American Statistical Association. McMILLE14, D., and HERRIOT, R.A.; 1984 "Toward a Longitudinal Definition of Households", in Proceedings of the Social Statistics Section, American Statistical Association. McMILLEN, D., and KASPRZYK, D.; 1985 "Item Nonresponse in SIPP", in Proceedings of the Survey Research Methods Section, American Statistical Association. MOORE, J., AND KASPRZYK, K. 1984 "Month-to-Month Recipiency Turnover in the ISDP", in Proceedings of the Survey Research Methods Section, American Statistical Association. NELSON, D., McMILLEN, K., and KASPRZYK D. 1983 "An overview of the survey of income and program participation." U.S. Bureau of the Census, Washington, D.C. OHIO STATE UNIVERSITY, The; 1979 The National Longitudinal Surveys Handbook. First Edition. Columbus: Center for Human Resource Research. 1982 The National Longitudinal Surveys Handbook. Second Edition. Columbus: Center for Human Resource Research. 157 PARNES, H.S. 1972 Longitudinal Surveys: Prospects and Problems" in Monthly Labor Review, 95 no.2:11-15. RHOTON, P. 1983 "Attrition and the National Longitudinal Surveys of Labor Force Behavior: Avoidance, Control and Correction". Unpublished mss., RUBIN, D. 1974 "Characterizing the Estimation of Parameters in Incomplete Data Problems." Journal of the American Statistical Association, 69, 467-474. SATER, D., 1985 "Enhancing Data from the Survey of Income and Program Participation with Data from Economic Censuses and Surveys". SIPP Working Paper series no. 8505, Bureau of the Census. SINGER, BURTON. 1983 "Longitudinal Data Analysis", in N. Johnson and S. Kotz (eds.), Encyclopedia of Statistical Sciences, Vol. IV, John Wiley and Sons. SINGER, BURTON and SPILERMAN, SEYMOUR 1976 "Some Methodological Issues in the Analysis of Longitudinal Surveys", The Annals of Economic and Social Measurement, Vol. 5 no. 4 Fall, 447-474. TUMA, NANCY BRANDON 1976 "Rewards Resources, and the Rate of Mobility: A Nonstationary Multivariate Stochastic Model," American Sociological Review, Vol. 41, 338-360. TUMA, NANCY BRANDON and HANNAN, MICHAEL T. 1984 Social Dynamics: Models and Methods., Academic Press. U.S. BUREAU OF THE CENSUS; 1982 Wage and Salary Data from the Income Survey Development Program, Current Population Reports, Series P-2 , no. 11 . U.S.G.P.O., Washington, D.C. 1983 Economic Characteristics of Households in the United States: - third Quarter, 1983, Current Population Reports, Series P-70, no. 1. U.S.G.P.O., Washington, D.C. (This series is published with quarterly information. no. 5 in the series, containing data for fourth quarter 1984, was released November, 1985.) U.S. DEPARTMENT OF COMMERCE; 1978 A Framework for Planning U.S. Federal Statistics for the 80's. Office of Federal Statistical Policy and Standards. Washington, D.C. U.S. SOCIAL SECURITY ADMINISTRATION, ORLANDO, FLORIDA 1982 "disability Insurance Work Incentive Experiments: Project Statement", SSA/OP/ORDS/DDS, March. U.S. SOCIAL SECURITY ADMINISTRATION (N.D.)"Retirement History Study Report Series", Social Security Administration Publication no. 73-11700. 158 VAUGHN, D., WHITEMAN, T., and LININGER, C.; 1984 "The Quality of Income and Program Data in the 1979 ISDP Research Panel: Some Preliminary Findings", in Review of Public Data Use,"Vol. 12, no. 2, pp. 107-131. WHITE, G.D. JR., and HUANG, H. 1982 "Mover Followup Costs for the Income Survey Development Program" paper presented at the Joint Statistical Meetings of the American Statistical Association et al., Cincinnati, Ohio, August. WHITMORE, R. , COX, B. , and FOLSOM, R. 1982 Family Unit Weighting Methodology for the National Household Survey Component of the National Medicaid Care Utilization and Expenditure Survey. Research Triangle Institute, Research Triangle Park, N.C. RTI/1898/06-03F. WILLIAMS, W.H. and C.L. MALLOWS 1970 "Systematic Biases in Panel Surveys," in JASA 65: 1338-1349. YCAS, MARTYNAS A. 1982 "survey Design and Panel Attrition". Paper no. 11 in David, Op. Cit. pp. 147-154. YCAS, MARTYNAS A. and LININGER, C.; 1981 "The Income Survey Development Program: Design Features and Initial Findings" in Social Security Bulletin, vol. 44, no. ii. 159 Reports Available in the Statistical Policy Working Paper Series 1. Report on Statistics for Allocation of Funds; GPO Stock Number 003-005-00178-6, price $2.40 2. Report on Statistical Disclosure and Disclosure-Avoidance Techniques; GPO Stock Number 003-005-00177-8, price $2.50 3. An Error Profile: Employment as Measured by the Current Population Survey; GPO Stock Number 003-005-00182-4, price $2.75 4. Glossary of Nonsampling Error Terms: An Illustration of a Semantic Problem in Statistics (A limited number of copies are available from OMB) 5. Report on Exact and Statistical Matching Techniques; GPO Stock Number 003-005-00186-7, price $3.50 6. Report on Statistical Uses of Administrative Records; GPO Stock Number 003-005-00185-9, price $5.00 7. An Interagency Review of Time-Series Revision Policies (A limited number of copies are available from OMB) 8. Statistical Interagency Agreements (A limited number of copies are available from OMB) 9. Contracting for Surveys (Available through NTIS Document Sales, PB83-233148) 10. Approaches to Developing Questionnaires (Available through NTIS Document Sales, PB84-105055) 11. A Review of Industry Coding Systems (Available through NTIS Document Sales, PB84-135276) 12. The Role of Telephone Data Collection in Federal Statistics (Available through NTIS Document Sales, PB85- 105971) 13. Federal Longitudinal Surveys (Available through NTIS Document Sales, PB86-139730) Copies of these working papers, as indicated, may be ordered from the Superintendent of Documents, U.S. Government Printing Office, Washington, D.C. 20402 (202-783-3238) or from NTIS Document Sales, 5285 Part Royal Road, Springfield, VA 22161 (703-487-4650).
(November 1985)
| Page Last Modified: April 20, 2007 | FCSM
Home Methodology Reports |