Abstracts for EIA’s Fall 2008 Meeting

with the

ASA Committee on Energy Statistics

1.  Preliminary Results of Energy Consumer Price Index Research, Janice Lent, Statistics and Methods Group (SMG)

The Energy Information Administration (EIA) is researching estimation methods with the goal of developing an experimental Energy Consumer Price Index (ECPI), based almost entirely on EIA data. For some major energy sources, EIA collects universe or large-sample price and sales data, which can be used to compute price indexes with very low sampling error. Also, EIA’s model-based projections of future energy prices and consumption levels can be used to develop CPI forecasts for some energy components. Because the experimental indexes are being computed in a research environment rather than in a large-scale production environment, the process of incorporating data from new energy surveys will be streamlined.  In this paper, we provide background information and preliminary results of EIA’s ECPI estimation research. We also discuss some producer price indexes that could be estimated using EIA data.Questions for the Committee:

1) Given that our experimental ECPI closely tracks the energy component of the BLS CPI for the period examined, should EIA move forward to produce the ECPI on a regular basis? 

2) Which of the producer price index targets discussed would be most relevant for policy makers and the general public?

ASA Committee Recommendations:

Because the experimental ECPI series closely tracks the energy component of the BLS CPI for the period examined, the Committee was asked to comment on the value of producing the ECPI on a regular basis. Also, the Committee was asked to judge the relative value of two proposed PPI series: (1) a PPI to track prices of fuels used for electricity generation and (2) a PPI to track prices of residential solar energy equipment.

The Committee’s reaction to the experimental ECPI series presented was mixed. Some members suggested that, due to the similarity between the ECPI and the energy component of the BLS CPI, EIA should not pursue the goal of publishing a new index series. Others, however, saw the differences between the two series as potentially important. It was noted, for example, that, during times of structural change in the energy industries, the EIA series would be likely to detect changes sooner than the BLS series. (The weights used in computing the BLS CPI are based on data from the Consumer Expenditure Survey; the reference periods for the weights are generally two to three years prior to those for the indexes.)

With regard to the suggested PPI series, Committee members generally favored the solar energy equipment series over the PPI for fuels used in electricity generation.  Because electric power generators incur other costs in addition to fuel costs, some members suggested that a PPI reflecting only fuel costs may seem incomplete. (BLS produces a PPI for the electric power industry that covers other costs in addition to fuel costs.) One Committee felt strongly that EIA should try to measure changes in the cost of wind energy equipment.

EIA Intended Response(s):

EIA will continue research on the ECPI. The PPI for solar energy equipment will also be investigated as resources permit. EIA currently does not collect data on the cost of wind energy equipment. The option of calculating a synthetic CPI for wind energy equipment will be investigated.  The synthetic statistic would be based on various commodity components of the BLS PPI (e.g., turbine parts) with weights reflecting the relative importance (or expenditure share) of each component in the production of wind energy equipment.

2.  Estimating Monthly Ethanol Consumption in the U.S. , Carol Joyce Blumberg and Michael Conner, Petroleum Division (PD), Office of Oil and Gas (OOG)

Presently, EIA has two methods of estimating ethanol consumption in the U.S. from published data.  These are:

Method 1: Consumption is estimated using a Product Supplied formula of: Consumption for Ethanol = (Net Production for ethanol + Imports of ethanol) –– (Stock Change in ethanol)

Method 2: Consumption for ethanol = .09∙(Monthly-from-Weekly estimate of reformulated gasoline with alcohol production volume + Monthly-from-Weekly estimate of conventional gasoline with alcohol production volume)

A third method (Method 3) is available that uses unpublished data that are at a finer level than the published data.

Method 3: Consumption for ethanol = .09∙(Monthly unpublished reformulated gasoline with alcohol production volume + Monthly unpublished conventional gasoline with alcohol production volume)

Method 2 uses a sample each week and does not include corrections by respondents after the data collection date, while Methods 1 and 3 are based on a census and allow for respondent corrections.

Using multiple linear regression, it was found that Method 1 and a variation on Method 3 give nearly identical estimates and that Method 2 produces biased estimates. 

ASA Committee Recommendations:

Only one member of the Committee made a suggestion. He suggested that we revise our weekly estimates of motor gasoline volumes once we have the monthly estimates of motor gasoline volumes. This way, we would have the final weekly estimates better match with the monthly estimates.

EIA Intended Response(s):

While the suggestion from that committee member has merit, the official revision policy is that “EIA will disseminate revised weekly data only if the revision is expected to substantively affect understanding of U.S. petroleum supply. The decision to disseminate a revision to weekly data will be based on EIA’s judgment of the revision’s expected effect. While revisions are expected to be rare, if one is necessary, it will be disseminated in the next regularly scheduled release of the weekly products.” (quoted from explanatory notes in Weekly Petroleum Status Report)

3.  Time Limits For Protecting Company Level Data, Jacob Bournazian, SMG

EIA protects a majority of the survey information that it collects by using Exemption (4) four under section 552 of the Freedom of Information Act. Exemption four allows the government to withhold from release to the public certain documents in its possession if the information would likely cause substantial harm to a survey respondent’s competitive position. This presentation will discuss how the passage of time affects the potential that a respondent may suffer competitive harm if EIA released historical company level data that was reported on an EIA survey. The presentation discussed how the passage of time affects the risk that a respondent may suffer competitive harm if EIA released historical company level data that was reported on an EIA survey. Feedback was collected during the session on the factors that affect a company’s competitive position in an energy market and whether EIA should consider releasing more historical company level survey data after a certain period of time has passed.

QUESTIONS FOR THE COMMITTEE

1) How does the age of the survey data affect the need to protect it?
2) Should time limits be considered for protecting company level data where EIA is not obligated by statute to protect it?
3) What economic factors should be considered when considering a time limit for protecting company level data?
4) What interrelationships between economic factors should be considered?
5) How may the interrelationship between economic factors that may exist in an energy market increase or decrease the risk of causing competitive harm to a survey respondent by releasing historical company level data?

ASA Committee Recommendations:

The committee members recommended asking specific energy industry groups if they would object to EIA releasing their data after a certain specified time period and then deciding whether to set a time limit and what the time period should be. In the meantime, EIA should consider placing company level data in a data enclave for researchers or releasing more public use data files with direct identifiers removed to prevent the identifiability of respondents.

EIA Intended Response:

EIA decided not to set time limits for protecting the confidentiality of company level data and will explore alternatives for expanding researcher access to survey data.

4A.  Analyzing the STB’s Carload Waybill Sample, Vlad Dorjets, CNEAF

EIA recently obtained a database from the Surface Transportation Board with a statistical sample of all railroad waybills for movements of coal and other energy commodities throughout the country for 2000-2006. This presentation discusses characteristics of  the database, the approach this office has taken to analyze it and the products they hope to create from it. It also discusses the challenge of using this particular sample to represent a population.

REPORT OF COMMITTEE RECOMMENDATIONS AND EIA RESPONSE IS FORTHCOMING

4B.  Modeling International Renewable Trade Sources, Brian Murphy, OIAF

This is a presentation of the author’s work on improving our modeling of international renewable energy sources. In the past, our long-term projections for the International Energy Outlook have aggregated hydroelectricity and all other marketed renewable energy sources. The work involves regional projections of individual renewable sources, including hydroelectricity, wind, solar, and geothermal for use in our IEO2009. The presentation addresses the modeling approach and preliminary results of this effort. It discusses the new “bottom-up” approach that is being used for renewable electricity in this year’s International Energy Outlook, and the challenges of building a model for an industry (renewables) shaped primarily by policy, not cost.

ASA Committee Recommendations:

Committee members recommended improving the renewable electricity forecast by incorporating resource maps of renewable resources and separating the analysis of newer renewable electricity technologies (i.e., wind, solar, and geothermal) from mature technologies (large hydroelectricity). The committee also recommended conducting multiple renewable electricity forecasts scenarios to better account for a variety of model assumptions. Rapid and slow technology improvement scenarios and various carbon tax scenarios were recommended. In addition, the committee advised investigating the correlation between growth in renewable electricity equipment manufacturing and the growth in renewable electricity generation, as well as the relationship between pro-renewable government policies and the renewable electricity cost premium (compared to conventional sources).

EIA Intended Response(s): 

4C.  Modeling International Biofuels Supplies, Emre Yucel, OIAF

Liquid biofuels are becoming more and more popular as a transportation fuel. In order to better project world liquids supply the biofuels component of it must be modeled accurately. Different initial modeling approaches were presented, one that relied on available land use, and one that utilized supply curves created using cost data.

ASA Committee Recommendations:

The committee recommended that the costs and effects of converting idle land to crop land be considered. It was also recommended that the availability of water resources be evaluated.

EIA Intended Response(s):

A model that has a number of modules will be implemented. This will include a transportation and food crop module to better model the competition between the two for fuel use. The amount of feedstock produced will be dependent on the cost of using the land for feedstock production, and feedstock growth will be dependent on the cost to convert new land.

5. Subcommittee Meeting (Closed Session)

6. Sensitivity Analysis of EIA Forecasting Systems, Preston McDowney, SMG and George Lady, Consultant to SMG

This paper reports upon sensitivity analyses conducted with the National Energy Modeling System (NEMS) and the Regional Short Term Energy Model (RSTEM). Sensitivities were developed for NEMS consuming sector modules (residential, commercial, industrial, and transportation) using a regression analysis for data extracted from NEMS solutions for which energy prices and activity drivers were systematically varied. These results were reported to the ASA Committee in October 2007. For this report, sensitivities to weather were also investigated. Based upon the regressions and weather variable analyses, error decomposition was conducted for the AEO1998-AEO2003 versions of NEMS with respect to projections for the year 2006 for residential and commercial sector electricity and natural gas consumption. A sensitivity analysis was conducted at the census region level of detail for price, driver, and weather variables for RSTEM forecasting equations for residential and commercial sector electricity and natural gas consumption. Similar sensitivities for were found for NEMS at the census region level of detail. These results are reported here for comparative purposes.

ASA Committee Recommendations:

The Committee was favorable to the goals of the project and the methodology utilized. It was pointed out that, in addition to explaining the basis for differences between forecast values and eventually realized values, the method  would also serve a model validation function by revealing how NEMS accounted for weather and the other exogenous variables. It was additionally proposed that the roster of influences on forecast accuracy included many variables that the methodology was not accounting for. Subject to resource and other constraints, it was recommended to consider expanding the list of exogenous variables to account for. Finally, it was noted that a number of recent inquiries about the “accuracy” of NEMS projections could have been substantially responded to if the proposed methodology had been in place. The Committee recommended that the methodology presented be established and provide regular reports on the basis for forecast accuracy.

EIA Intended Response(s):

EIA has continued to sponsor the project. Current goals are to expand the scope of variables included and apply the method to each AEO version of NEMS as archived versions  become available. The first version of NEMS used for the project was NEMS2007. Stand alone NEMS projections for this version do not begin until 2010. When actual data for the exogenous variables in the year 2010 become available, the first impact analysis using the methodology will be implemented.

7.  Assessment of RSTEM Forecasts, Andrew J. Buck, Consultant to SMG

This paper reports on forecast analyses conducted with the Regional Short Term Energy Model (RSTEM). Forecasts were developed for consuming sectors (residential, commercial and industrial) using the October 2007 release of RSTEM. The forecasted fuels were natural gas, electricity, and distillate. For natural gas and electricity the forecasts were generated for the nine census regions. The commercial and industrial distillate forecasts for geographic regions were based on the set of fourteen areas created from the nine census regions. Residential distillate consumption forecasts were developed for four geographic aggregates of the nine census regions. For each fuel -- region doublet the October forecast was compared to the realized data reported in May 2008, using % error. The October model was rerun using the realized values for the exogenous variables and the endogenous variable add factors. The % forecast error was again calculated. Finally, the October model was rerun using the realized values for the exogenous variables and no add factors for the endogenous variables. The % forecast error was again calculated.

ASA Committee Recommendations:

1. The Los Alamos Laboratories have experience with complex computer model validation
2. Clarification of the use of add factors in out of sample forecasts was requested, as was the use of levels and changes of the dependent variable.  The add factors serve two purposes: The need for them even when the exogenous data is known implies model inadequacy. When used for an out-of-sample forecast they provide an evaluation of the modeler's ability to compensate for unobservables.  If the model is in levels then it begs the question of stationarity of the dependent variable.
3. There is a literature on the use of preliminary data that should be reviewed.
4. Are there private sources of weather data?
5. ANOVA might be a useful way to identify the important sources of forecast errors.
6. Are there other regional energy models in use and what is their performance record?
7. Is there a process to find better specifications?

EIA Intended Response(s):

At bottom, the ASA Committee felt that there were no serious problems in the RSTEM Forecast Assessment project and recommended that the project continue.

 8.  Results of 2008 EIA 30^th Anniversary Energy Conference Survey, Howard Bradsher-Fredrick and Phillip Tseng, SMG and Michael Salpeter, Summer Intern

On April 7-8, 2008, EIA hosted an energy conference to commemorate and celebrate the thirtieth anniversary of its inception. The presenters included the current Secretary of Energy, the first secretary, representatives of Congress, former EIA administrators and other notable experts in the field of energy. A total of 1,559 persons registered for the conference and 1,150 persons attended the conference. Shortly after the conclusion of the conference, a web-based survey was administered to all of the attendees (a total of 1,060) who provided usable e-mail addresses. This paper describes the survey methodology employed, the results and the final conclusions. A response rate of 41.5% was achieved through the use of three e-mailings (the original and two follow-ups). The survey included both closed-ended and open-ended questions. These results indicated that respondents were generally very favorable toward the conference and will be of value in planning the next EIA energy conference. 

ASA Committee Recommendations:

Good survey, good methodology, great response rate. It is good to be using a current e-mail list and good to keep the questionnaire short. As another piece of information concerning the demographics of the respondents, it might be useful to know who the respondent works for (a general classification of some type). It is important to separate renewable energy from alternative fuels so one can differentiate between them in the analysis. Also, how far the person traveled in order to attend the conference might be interesting. Also, it would be interesting to know which respondents are EIA employees, so that question should be asked. It was thought that a higher response rate would be attained through a web-based survey rather than using a paper survey. However, the results from a paper survey might provide better results since it would involve a more immediate reaction to the conference.

EIA Intended Response(s):

EIA will attempt to keep its customer surveys short and concise preceded by a well thought-out research design. We will also attempt to use a current e-mail list whenever possible so as to optimize the response rate. If EIA staff members are allowed to respond to a customer survey, we should attempt to stratify their responses for the purposes of analysis. We will also use web-based customer surveys whenever possible as opposed to paper surveys.