Food

Summary Report of the U.S. Food and Drug Administration's Initial Test of the Accuracy of the Emergency Contact/U.S. Agent Data in the Food Facilities Registration Database

October 2006

Return to Registration of Food Facilities

Background
Sample Size And Data Set Selection
Protocol For Conducting The Notification Test
Notification Test Results
Analysis And Discussion Of Test Results
Recommendations And Follow-Up Action Plan

Appendix A: E-Mail Test Message
Appendix B: Fax Test Message
Appendix C: Phone scripts for Emergency Contact Test

A. Background

The United States (U.S.) Food and Drug Administration (FDA) established a Food Facilities Registration Database in response to Section 305 of the Public Health Security and Bioterrorism Preparedness and Response Act of 2002 (the Bioterrorism Act), as implemented by the Registration of Food Facilities Final Rule co-issued by the Secretaries of the Department of Health and Human Services (DHHS) and the Department of Homeland Security (DHS) on October 10, 2003 (Title 21 Code of Federal Regulations (CFR) Part 1, Subpart H). Section 305 of the Bioterrorism Act requires, in part, that the Secretary of DHHS compile and maintain an up-to-date list of facilities that are registered under this section; the Secretary has delegated this responsibility to FDA. Registration provides FDA with information about facilities that manufacture/process, pack, or hold food for consumption in the United States. In the event of an outbreak of food-borne illness, such information will help FDA and other authorities determine the source and cause of the event. In addition, the registration information will enable FDA to notify more quickly the facilities that might be affected by the outbreak.

The Bioterrorism Act and §§ 1.225 and 1.230 of the final rule specifically require owners, operators, or agents in charge of domestic (U.S.) and foreign facilities that manufacture, process, pack, or hold food for human or animal consumption in the U.S. to register with the FDA by December 12, 2003, unless the facility begins operations after that date. In those instances, the final rule requires registration with FDA before the facility begins operations. [Note: all section citations to the final rule are found in 21 CFR Part 1, Subpart H.] As of September 14, 2006, 126,210 domestic food facilities and 171,837 foreign food facilities have registered with FDA. A breakdown of the number of domestic facilities in each State within the U.S., as well as the number of foreign facilities registered by country as of September 1, 2006, is available on FDA's website.

As stated in 21 CFR 1.232(e), the registration for domestic facilities must include an Emergency Contact phone number that is accessible at all times. In addition, domestic facilities may choose also to provide, as stated in § 1.233(d), the Emergency Contact's name, title, and e-mail address.

Section 1.232(d) of the final rule requires the registration for foreign facilities to include the name, address, phone number, and Emergency Contact phone number of the facility's U.S. Agent (if there is no other Emergency Contact listed under § 1.233(e). Under § 1.227(b)(13) of the final rule, a U.S. Agent must be a person (as defined in section 201(e) of the Federal Food Drug and Cosmetic Act (21 U.S.C. 321(e))) that resides or maintains a place of business in the United States whom a foreign facility designates as its agent for purposes of this final rule. A U.S. agent cannot be in the form of a mailbox, answering machine or service, or other place where an individual acting as the foreign facility's agent is not physically present. FDA further states in the final rule that the U.S. agent acts as a communications link between FDA and the foreign facility for both emergency and routine communications and will be the person FDA contacts when an emergency occurs, unless the registration specifies under § 1.233(e) another Emergency Contact. FDA further states that we will treat representations by the U.S. agent as those of the foreign facility, and will consider information or documents provided to the U.S. agent the equivalent of providing the information or documents to the foreign facility. 21 CFR 1.227(b)(13). Foreign facilities also may choose to provide the title, fax number, and e-mail address of their U.S. Agent, as provided in § 1.233(f).

Section 1.234 of the final rule requires owners, operators, or agents in charge of domestic and foreign facilities to update any of the information that is required in an initial registration within 60 calendar days of any change to the information submitted to FDA. The rule also requires owners, operators, or agents in charge of a registered facility to cancel a registration when ownership changes or a facility ceases operations within 60 days of the change; new owners are required to re-register with a facility that will continue operations.

From July 10, 2006, through August 2, 2006, FDA conducted an initial test to verify the accuracy of the Emergency Contact information contained in our Food Facility Registration Database. FDA's objective in conducting the test was to verify the accuracy of the Emergency Contact information in the registration database, and assess whether FDA can use this information to quickly notify facilities that may be affected by an actual or potential threat to the U.S. food supply, as intended. Specifically, our test was intended to allow us to estimate with 95% confidence the answers to two questions:

(1) The accuracy of the primary mode of transmission^[1] - i.e., whether an emergency alert that FDA sends to domestic and/or foreign facilities in our Food Facilities Registration Database by primary e-mail, facsimile (fax), or phone would actually reach its intended recipient; and
(2) The accuracy of the identity of the Emergency Contact/U.S. Agent information in the Food Facility Registration Database - i.e., whether the intended recipient of FDA's alert is actually the Emergency Contact and/or U.S. Agent for the facility.

The specific data FDA reviewed for accuracy during this test was limited to that provided to FDA by the registrants pursuant to §§ 1.232 and 1.233 of the final rule (i.e. the Facility Emergency Contact Information and/or United States Agent Information). To conduct the test, FDA randomly selected a statistically representative sample of 800 facilities (400 domestic facilities and 400 foreign facilities) from the Food Facilities Registration Database to be the subject of this test. We refer to these 800 facilities as the "test facilities" hereinafter.

This report summarizes the methodology we used and the results of our initial test. It also identifies corrective actions that owners, operators, or agents in charge of all facilities subject to the final rule may need to take, and the next steps FDA plans to take to ensure that the data within the registration database can provide needed information in the event of an actual or threatened public health emergency involving food under FDA's jurisdiction.

B. Sample Size and Data Set Selection

FDA's statistical objective in conducting the test was to estimate within ± 5%, with 95% confidence, the true percentage of the food facilities in the Food Facilities Registration Database that FDA could potentially notify in the event of an actual or threatened public health emergency involving food under FDA's jurisdiction. Before spending the resources to conduct an Emergency Contact test of all registered facilities, FDA wanted to obtain an estimate of the accuracy of the data that have been provided to FDA by owners, operators, or agents in charge of affected domestic and foreign facilities.

FDA used a statistically representative, stratified, random sample of the Food Facilities Registration Database; the data were stratified by state for domestic facilities and stratified by country for foreign facilities, such that FDA had 400 domestic facilities and 400 foreign facilities included in our test facilities matrix. Assuming that the accuracy of the Emergency Contact information for the population of food facilities in the registration database is within the range of 30 to 60%, FDA determined that a simple random sample (SRS) of n=400 is sufficient to establish a pilot study estimate of the true percentage of facilities that have provided accurate Emergency Contact information within the population of registered foreign or domestic facilities to within 5%, with 95% confidence.

For administrative convenience, in attempting to achieve a representative random sampling, FDA allocated n across the population of domestic registrants in proportion to the percentage of facilities in each state that are registered; i.e.,

n_h =	nN_h	=	400N_h
	N		N

where n_h= the number of facilities randomly sampled within state h, N_h= the number of facilities within state h, and N is the total number of facilities within the population (database), i.e., the sum of the N_h.

We used the same procedure to achieve a representative random sampling for foreign registrants, except we allocated n across the population of foreign registrants in proportion to the percentage contribution of each country's number of facilities to the total number of facilities across all foreign countries, with one exception. For the twelve (12) countries that had the smallest number of registered facilities, it was necessary to group these countries into six (6) units of two (2) countries each in order to have enough registered facilities per unit to justify the selection of one (1) facility from each of these units for inclusion in the sample set.

FDA notes, however, that during preparations for this test, we discovered that 1.4% of all domestic facilities in the Food Facilities Registration Database did not have any Emergency Contact data. This was due to a coding error in the domestic registration module that allowed domestic registrations under certain circumstances to be accepted by the electronic system as complete, even though the registrant had failed to submit all of the mandatory data. (The Food Facilities Registration Database generally is designed to not accept a registration until all mandatory information has been submitted.) FDA has corrected this coding error. In selecting the 400 domestic test facilities, FDA used only the 98.6% of the facilities that had submitted data in all mandatory fields.^[2] This was done because the 1.4% of the domestic facility population without Emergency Contact data does not need to be estimated; it is already fully known. Selecting the sample from the remaining 98.6% of the population allowed for a better estimate of that portion of the population. As a result, the domestic test facilities results presented in the tables below are based on 98.6% of the domestic registrations.

FDA's final conclusions about the estimated accuracy of the transmission mode and Emergency Contact data for domestic facilities in the Food Facilities Registration Database, however, do account for the 1.4% of the domestic facilities that did not provide any information for the Emergency Contact. Accordingly, our final conclusions regarding FDA's ability to communicate with Emergency Contacts/U.S. Agents at domestic and foreign facilities using information provided to FDA in facilities' registrations are based on 100% of both the domestic and the foreign facilities in the Food Facilities Registration Database.

C. Protocol for Conducting the Notification Test

FDA developed detailed procedures for notifying the test facilities by e-mail, fax, and telephone. We also developed detailed procedures for tracking the data obtained during this test, which were used by those performing these tasks. These procedures included template messages for e-mail and fax notifications and scripts for telephone notifications. FDA requested a reply from those who received an e-mail or fax notification affirming whether he/she is or is not the Emergency Contact/U.S. Agent for the facility named in the notification. FDA also requested a verbal response from Emergency Contacts/U.S. Agents who were notified by telephone affirming whether he/she is or is not the Emergency Contact/U.S. Agent for the named facility. If FDA did not receive a response to an e-mail or fax notification within 14 days, we subsequently contacted the Emergency Contact/U.S. Agent by telephone. For the purposes of this test, FDA considered the test "successful" if an Emergency Contact/U.S. Agent who received an e-mail or fax, or who was contacted by telephone using the information provided to FDA in Section 5 (Facility Emergency Contact Information) and/or Section 7 (United States Agent Information) of the facility's registration verified that he/she is the Emergency Contact/U.S. Agent for the named facility.

Of the domestic and foreign test facilities, 325 (81.3%) and 345 (86.2%), respectively, provided an e-mail address for the Emergency Contact in the facility's registration and FDA used the e-mail addresses as the primary mode of contacting these facilities. Less than 20% of all test facilities provided only the required telephone number for their designated Emergency Contact in the facility's registration (specifically, 75 (18.7%) domestic test facilities and 46 foreign test facilities (11.5%), respectively); for these facilities, FDA considered the telephone number the primary mode of contact. Nine (9) foreign test facilities (2.3%) opted to also provide to FDA a fax number for their designated U.S Agent, and for these facilities, FDA considered the fax number the primary mode of contact. These statistics are summarized in Table 1 below.

Table 1. Contact Information Provided in the Registrations of Test Facilities
FACILITY:		DOMESTIC	FOREIGN
Number of test facilities:		400	400
Emergency contact information	Phone only	75 (18.7%)	46 (11.5%)
	E-mail and phone	325 (81.3%)	345 (86.2%)
	Fax and phone	0	9 (2.3%)

On July 10, 2006, FDA began notifying by e-mail all domestic and foreign test facilities that had provided an e-mail address in their registrations; we also began telephone notification for all foreign and domestic test facilities that had provided only the required phone number. Both of these modes of primary notification ended on July 26, 2006. On July 17, 2006, FDA began notifying domestic and foreign test facilities by fax and initiating follow-up telephone calls (secondary contact) to those Emergency Contacts/U.S. Agents that we had sent an e-mail that was deemed undeliverable, and those to whom we had sent an e-mail or fax notification but who had failed to respond. We concluded the test on August 2, 2006. FDA documented and tracked all contact attempts and responses to e-mails, faxes and phone calls.

A summary of the procedures for the three modes of notification that FDA used for this test is provided below.

1. Summary of Protocol for E-mail Notification

FDA sent e-mails to the Emergency Contacts/U.S. Agents for all foreign and domestic test facilities whose registration included an e-mail address. The e-mail requested each recipient to click the appropriate button indicating whether he/she is or is not the Emergency Contact/U.S. Agent for the named facility. A copy of the e-mail notification message is provided in Appendix A. If FDA did not receive an e-mail response from the recipient within 14 days of a successful transmission of the e-mail, FDA telephoned the Emergency Contact/U.S. Agent for the foreign or domestic facility at the phone number for the Emergency Contact/U.S. Agent provided in the facility's registration. FDA also telephoned those Emergency Contacts/U.S. Agents for domestic and foreign test facilities that were sent an e-mail notification by FDA if FDA's e-mail was returned as undeliverable.

2. Summary of Protocol for Fax Notification

FDA faxed test messages to U.S. Agents for foreign test facilities that had provided a fax number but no e-mail address in their registrations. (As stated above, if the facility's registration included an e-mail address, FDA's first mode of contacting the facility was by e-mail.) FDA faxed test messages during the evening hours, between 7:00 pm and 11:00 pm Eastern Time, utilizing the fax notification message provided in Appendix B. (As noted earlier, under the final rule, U.S. Agents must reside or maintain a place of business in the United States.) FDA's fax notification requested a reply from the recipient to verify whether he/she is or is not the Emergency Contact/ U.S. Agent for the facility. If we did not receive a response within 14 days of a successful transmission of the fax notification, FDA contacted the U.S. Agent by telephone. If the fax line was busy, we attempted a maximum of six times to fax the notification to the facility over a two day period.

3. Summary of Procedures for Phone Notification

On the first day of our test, FDA initiated contact by telephone for those foreign and domestic test facilities that provided only the required telephone information for the Emergency Contact/U.S. Agent in their registrations. A copy of the script FDA used is provided in Appendix C. FDA called between 9 a.m. and 5 p.m. local time based on the location of the Emergency Contact/U.S. Agent. In addition to notifying this subset of the test facilities by telephone, FDA also called Emergency Contacts/U.S. Agents who had received an e-mail/fax notification from us, but failed to respond to FDA within 14 days, and Emergency Contacts/U.S. Agents to whom we sent e-mails during this test that were deemed undeliverable.

FDA made multiple attempts to contact the Emergency Contacts/U.S. Agents by telephone if a voicemail system was unavailable, there was no answer, or the line was busy. If a voicemail system was available, FDA left a message requesting that the Emergency Contact/U.S. Agent contact the FDA to verify that he/she is or is not the Emergency Contact/U.S. Agent for the named facility. If after two successive days and six attempts to contact the Emergency Contact/U.S. Agent by telephone, FDA was unsuccessful, the test was concluded and we recorded the attempted contact as unsuccessful.

D. Notification Test Results

1. Results For Domestic Test Facilities

As indicated above in section B of this report, FDA randomly selected 400 domestic test facilities from the 98.6% of all domestic registrations in the Food Facilities Registration Database that were unaffected by the coding error. In summary:

FDA was able to send a primary e-mail or make a primary or secondary (follow-up) telephone call to 361 of the 400 (90.3%) domestic test facilities; of these, FDA received a response from, or talked with, only 289 (72.2%) of the Emergency Contacts listed in the facility's registration.
- 244 of the persons who FDA contacted at these domestic test facilities who are designated as the Emergency Contact for the facility verified that he/she is the Emergency Contact for the facility (61.0% of all domestic test facilities).
- 41 of the persons who FDA contacted at these domestic test facilities who are designated as the Emergency Contact for the facility stated that they were not the Emergency Contact for the named facility (10.2% of all domestic test facilities).
- 4 of the persons who FDA contacted at these domestic test facilities who are designated as the Emergency Contact for the facility did not speak English and therefore FDA could not verify whether the person is or is not the Emergency Contact for the facility (1.0% of all domestic test facilities).
For 72 of the 400 domestic test facilities (18.0%), the phone number provided for the Emergency Contact was determined to be associated with the correct facility; however, FDA did not receive a response to the voicemail we left requesting a return confirmation by the end of the test period. Accordingly, we could not verify if the Emergency Contact had been reached.
For 39 of 400 domestic test facilities (9.8%), the telephone number provided in the Food Facilities Registration Database for the Emergency Contact was not correctly formed or was for a non-existent line or an out of service line, which precluded FDA from being able to complete the phone call.

A more specific breakdown of the responses to our notification to domestic test facilities is provided in Table 2 below.

Table 2 - Domestic Test Facilities Results (n=400)
Mode of Contact			Verified Is Emergency Contact	Is Not Emergency Contact	Undeliverable E-mails	No Response After 14 Days	Response in Foreign Language	Phone Correct - No Response in Test Period	Phone Problems
E-mail Primary Mode	Number of Domestic E-mails	325	146 (36.5%)	9 (2.3%)	62 (15.5%) *	108 (27.0%) **
	*Follow-up Phone Calls to 62 Undeliverable E-mails		13 (3.3%)	17 (4.2%)			1 (0.3%)	13 (3.3%)	18 (4.5%)
	** Follow-up Phone Calls to 108 Facilities With No Response in 14 days		51 (12.7%)	13 (3.2%)			1 (0.3%)	34 (8.5%)	9 (2.3%)
Total E-mail/ Follow-up Phone		325	210 (52.5%)	39 (9.7%)			2 (0.5%)	47 (11.8%)	27 (6.8%)
Telephone Primary Mode		75	34 (8.5%)	2 (0.5%)			2 (0.5%)	25 (6.2%)	12 (3.0%))
Total Domestic Test Facilities Results		400	244 (61%)	41 (10.2%)			4 (1.0%)	72 (18.0%)	39 (9.8%)

2. Results for Foreign Test Facilities

FDA also randomly selected 400 foreign test facilities from all foreign registrations in the Food Facilities Registration Database. In summary:

FDA was able to send a primary e-mail or fax or make a primary or secondary (follow-up) telephone call to 351 of the 400 (87.8%) foreign test facilities; of these, FDA received a response from, or talked with, only 235 (58.8%) of the Emergency Contacts/U.S. Agents listed in the facility's registration. In summary:
- 192 of the persons who FDA contacted at these foreign test facilities who are designated as the Emergency Contacts/U.S. Agents for the facility verified that he/she is the Emergency Contact/U.S. Agent for the facility (48.0% of all foreign test facilities).
- 43 of the persons who FDA contacted at these foreign test facilities who are designated as the Emergency Contacts/U.S. Agents for the facility stated that they were not the Emergency Contact/U.S. Agent for the named facility (10.8% of all foreign test facilities).
- 37 of the persons who FDA contacted at these foreign test facilities who are designated as the Emergency Contacts/U.S. Agents for the facility did not speak English and therefore FDA could not verify whether the person is or is not the Emergency Contact/U.S. Agent for the facility (9.2% of all foreign test facilities).
For 79 of the 400 foreign test facilities (19.7%), FDA determined that the phone number provided for the Emergency Contact/U.S. Agent was associated with the correct facility; however, FDA did not receive a response to the voicemail we left requesting a return confirmation by the end of the test period. Accordingly, we could not verify if the Emergency Contact/U.S. Agent had been reached.
For 48 of the 400 foreign test facilities (12%), the telephone number provided in the Food Facilities Registration Database for the Emergency Contact/U.S. Agent was not correctly formed or was for a non-existent line or an out of service line, which precluded FDA from being able to complete the phone call.
FDA determined that one (1) of the 400 foreign test facilities (0.3%) had gone out of business.

A more specific breakdown of the responses to our notification to foreign test facilities is provided in Table 3 below:

Table 3 - Foreign Test Facilities Results (n=400)
Mode of Contact			Verified Is Emergency Contact	Is Not Emergency Contact	Undeliverable E-mail/ Fax	No Response After 14 Days	Response in Foreign Language	Phone Correct - No Response By Test Period	Phone Problems	Out of Business
E-mail Primary Mode	Foreign E-mails	345	128 (32.0%)	18 (4.5%)	55 (13.7%) *	144 (36%) **
	*Follow-up Phone Calls to 55 Undeliverable E-mails		16 (4.0%)	3 (0.8%)			6 (1.5%)	16 (4%)	13 (3.2%)	1 (0.3%)
	**Follow-up Phone Calls to 144 Facilities With No Response in 14 days		36 (9.0%)	16 (4.0%)			20 (5%)	47 (11.7%)	25 (6.3%)
Total E-mail/Phone		345	181 (45.2%)	36 (9%)			26 (6.5%)	63 (15.7%)	38 (9.5%)	1 (0.2%)
Fax Primary Mode	Fax	9	4 (1.0%)	1 (0.3%)	2 (0.5%)*	2 (0.5%) **
	*Follow-up Phone Calls to 2 Undeliverable Fax								2 (0.5%)
	**Follow-up Phone Calls to 2 Facilities With No Response in 14 Days							2 (0.5%)
Total Fax/Phone		9	4 (1.0%)	1 (0.3%)				2 (0.5%)	2 (0.5%)
Phone Primary Mode		46	6 (1.5%)	6 (1.5%)			11 (2.7%)	14 (3.5%)	9 (2.2%)
Total Foreign Test Facility Results		400	191 (47.8%)	43 (10.8%)			37 (9.2%)	79(19.8%)	49 (12.2%)	1 (0.2%)

E. Analysis and Discussion of Test Results

FDA conducted this initial test to allow us to estimate with 95% confidence the answers to two questions:

The accuracy of the primary mode of transmission - i.e., whether an emergency alert that FDA sends to domestic and/or foreign facilities in our Food Facilities Registration Database by primary e-mail, fax, or phone would actually reach its intended recipient; and
The accuracy of the identity of the Emergency Contact/U.S. Agent information in the Food Facility Registration Database - i.e., whether the intended recipient of FDA's alert is actually the Emergency Contact and/or U.S. Agent for the facility.

Affirmative answers to both questions are prerequisites for FDA to have an effective and efficient means of contacting registered facilities using our primary mode of transmission in the event of an actual or potential threat to the U.S. food supply.

As stated earlier, while preparing for this test, FDA discovered that 1.4% of all domestic facilities in the Food Facilities Registration Database did not have any Emergency Contact data. Since FDA excluded these facilities from consideration when selecting which facilities to include in the 400 domestic test facilities subset, the results of the test reported herein are with respect to the 98.6% of the population from which they were sampled. In drawing final conclusions about the total domestic facilities population, however, FDA has appropriately weighted the domestic test facilities population estimates by 0.986 and incorporated into the proper categories the 1.4% of the domestic population that had no data. Accordingly, the answers to the two questions above regarding the estimated accuracy of the mode of transmission and identity of the Emergency Contact within the Food Facilities Registration Database are for 100% of their respective populations in the database.

1. Estimated Accuracy of the Primary Mode of Transmission

In the event of a potential or actual threat to the U.S. food supply, FDA would attempt to send the alert message to the affected facilities' respective Emergency Contacts using their primary mode of transmission. In this context, FDA considers a registered facility's primary mode of transmission to be "accurate" if:

The e-mail address is for an existing, working account and FDA could send successfully the e-mail to the intended recipient;
The fax number is for an existing, working fax line and FDA could send successfully a fax to the intended recipient.
The phone number is for an existing, working phone line and FDA could call successfully the intended recipient.

FDA considers a registered facility's primary mode of transmission to be "inaccurate" if:

The e-mail address is not correctly formed or is for a non-existent account, rendering the e-mail undeliverable;
The fax number is not correctly formed or is for a non-existent line or a non-fax line, rendering the fax undeliverable; or
The phone number is not correctly formed or is for a non-existent line or an out of service line, which precluded FDA from being able to complete the phone call.

Table 4 groups the raw results for domestic and foreign test facilities into the two major categories based on whether the Emergency Contact/U.S. Agent data provided in a facility's registration was accurate (FDA was able to transmit an alert message to our intended recipient) or inaccurate (FDA was not able to transmit an alert message to our intended recipient). The test facilities results show a similar accuracy for domestic and foreign facilities.

Table 4 -Accuracy of the Test Facilities' Primary Mode of Transmission
Domestic Test Facilities (n=400)		Foreign Test Facilities (n=400)
Accurate primary e-mail address, fax number, or phone number	Inaccurate primary e-mail address, fax number, or phone number	Accurate primary e-mail address, fax number, or phone number	Inaccurate primary e-mail address, fax number, or phone number
81.5%	18.5%	83.5%	16.5%

Based on the test facilities results and accounting for the 1.4% of the domestic facilities known to have no Emergency Contact data, FDA can estimate with 95% confidence that the primary mode of transmission is accurate for 75.4% to 85.3% of all registered domestic facilities in the Food Facilities Registration Database. Conversely, FDA estimates with 95% confidence that the primary mode of transmission is inaccurate for 14.7% to 24.6% of all registered domestic facilities in the Food Facilities Registration Database.

Similarly, based on the test facilities results, FDA can estimate with 95% confidence that the primary mode of transmission is accurate for 78.5% to 88.5% of all registered foreign facilities in the Food Facilities Registration Database. Conversely, FDA estimates with 95% confidence that the primary mode of transmission is inaccurate for 11.5% to 21.5% of all registered foreign facilities in the Food Facilities Registration Database. These results are summarized in Table 5.

Table 5 - Estimated Accuracy, With 95% Confidence, of Primary Mode of Transmission for All Registered Facilities
Registered Domestic Facilities		Registered Foreign Facilities
Accurate primary e-mail address, fax number, or phone number	Inaccurate primary e-mail address, fax number, or phone number	Accurate primary e-mail address, fax number, or phone number	Inaccurate primary e-mail address, fax number, or phone number
75.4 - 85.3%	14.7 - 24.6%	78.5 - 88.5%	11.5 - 21.5%

2. Estimated Accuracy of the Identity of Emergency Contact/U.S. Agent

FDA evaluated not only whether the primary mode of transmission for contacting domestic and foreign test facilities provided in the facility's respective registrations was accurate, but also where the mode of transmission was accurate, whether the person named as the Emergency Contact/U.S. Agent in the facility's registration is actually the Emergency Contact/U.S. Agent for the facility. Our test attempted to determine whether the named recipient was correct by sending the e-mail, fax, or telephone test alert to the person designated as the Emergency Contact/U.S. Agent in each facility's registration and requesting that the recipient of the alert verify his/her status. In the case of e-mail test alerts, FDA followed up after 14 days of a non-response to the e-mail by phoning the recipient and attempting to obtain a response. Because providing a response was voluntary and the time frames of the test were limited, it was not always possible for FDA to determine whether the recipient was actually the Emergency Contact/U.S. Agent for the facility.

Table 6 shows the raw results for domestic and foreign test facilities for each accurate mode of transmission, and indicates whether the identity of the Emergency Contact is inaccurate, accurate, or unknown. There is a higher percentage of foreign facilities than domestic facilities at which FDA was unable to determine the accuracy of the Emergency Contact; this may be due to language barriers and the different time zones making it more difficult for FDA to follow-up with foreign Emergency Contacts who are not also the registered facility's U.S. Agent.

Table 6 - Accuracy of the Identity of the Emergency Contact/U.S. Agent for Test Facilities
Facility	Inaccurate primary e-mail address, fax number, or phone number	Accurate primary e-mail address, fax number, or phone number
Facility		Accurate Emergency Contact	Inaccurate Emergency Contact	Unknown whether Emergency Contact is Accurate or Not
Domestic Test Facilities	18.5%	57.8%	6.0%	17.7%
Foreign Test Facilities	16.5%	43.5%	10.3%	29.7%

Based on these test facilities results and accounting for the 1.4% of all domestic facilities in the Food Facilities Registration Database known to have no Emergency Contact data, FDA estimates with 95% confidence that both the primary mode of transmission and the identity of the Emergency Contact are accurate for 52.0% to 61.9% of all registered domestic facilities in the Food Facilities Registration Database. We further estimate with 95% confidence that either the primary mode of transmission or the identity of the Emergency Contact is inaccurate for 20.6% to 30.5% of all registered domestic facilities in the Food Facilities Registration Database. FDA also estimates with 95% confidence that the primary mode of transmission is accurate but the accuracy of the identity of the Emergency Contact is unknown for 12.6% to 22.4% of all registered domestic facilities in the Food Facilities Registration Database.

Similarly, based on these test facilities results, FDA estimates with 95% confidence that both the primary mode of transmission and the identity of the Emergency Contact/U.S. Agent are accurate for 38.5 to 48.5% of all registered foreign facilities in the Food Facilities Registration Database. We further estimate with 95% confidence that either the primary mode of transmission or the identity of the Emergency Contact/U.S. Agent is inaccurate for 21.8% to 31.8% of all registered foreign facilities in the Food Facilities Registration Database. FDA also estimates with 95% confidence that the primary mode of transmission is accurate but the accuracy of the identity of the Emergency Contact/U.S. Agent is unknown for 24.8% to 34.8% of all registered foreign facilities in the Food Facilities Registration Database. These results are summarized in Table 7.

Table 7 - Estimated Accuracy, With 95% Confidence, of the Mode of Transmission and Identity of the Emergency Contact/U.S. Agent Data for All Registered Facilities
Facility	Accurate Primary Mode of Transmission and Accurate Emergency Contact/U.S. Agent	Inaccurate Primary Mode of Transmission or Inaccurate Emergency Contact	Accurate Primary Mode of Transmission; Accuracy of Emergency Contact/ U.S. Agent Unknown
Registered Domestic Facilities	52.0 - 61.9%	20.6 - 30.5%	12.6 - 22.4%
Registered Foreign Facilities	38.5 - 48.5%	21.8 - 31.8%	24.7 - 34.7%

Even if all of the unknown recipients of FDA's test alert were indeed the Emergency Contact/U.S. Agent for the facility, 20% to 30% of the Emergency Contacts/U.S. Agents who need to be alerted in the event of an emergency would not receive the alert due to inaccurate registration data. The test results do not indicate that there is a significant difference between registered domestic facilities and registered foreign facilities in this regard. This is a significant percentage of the database. Because it is unlikely that all of the unknown recipients are indeed the Emergency Contacts/U.S. Agents for their respective facilities, the percentage of facilities that would not receive the alert is probably higher than estimated by the test results.

F. Recommendations and Follow-Up Action Plan

As a result of this test, FDA believes that it is imperative that immediate steps be taken by FDA and owners, operators, and agents in charge of domestic and foreign registered facilities to improve the accuracy of the information in the Food Facility Registration Database. Recommended follow-up actins are outlined below.

1. FDA Follow-up Actions

FDA plans to take the following actions:

Notify by U.S. mail all foreign and domestic test facilities that FDA determined during this test have inaccurate information in their facility registrations. FDA intends to send communications to the U.S. Agents for these facilities, unless the person indicated he/she is not the U.S. Agent for the facility. In those instances, FDA will notify the owner, operator, or agent in charge of the registered facility. (Note - under the final rule, the U.S. Agent acts as a communications link between FDA and the foreign facility for routine communications, such as this.)
Enhance the coding on the registration system to check the validity of the data being entered (e.g., review checks to ensure that domestic phone numbers include both area code and telephone number, and foreign phone numbers include the country access code, city code, and phone number).
Prepare a Quick Check "How To Guide" for industry that addresses common errors that FDA has found when performing quality checks of the data in the Food Facility Registration Database (e.g., many facilities re-register a facility when information changes instead of updating existing registrations, which results in duplicate and triplicate registrations for the same facility).
Determine whether the guidance documents for foreign facility registration needs to be prepared in additional foreign languages, and if so, as resources permit, issue the guidance through the various embassies and/or an outreach campaign.
Post the results on this test on our website and disseminate the summary report to trade associations, foreign embassies, and other fora (e.g., meetings, CFSAN Constituent Update) to encourage all registered facilities to review their data and correct it, if appropriate.
Conduct a second notification test in calendar year 2007 after FDA completes the above follow-up actions to determine whether the accuracy of the Emergency Contact information in the registration database has improved.

2. Follow-up Actions Needed by Registered Facilities

FDA urges all owners, operators, agents in charge and U.S. Agents of domestic and foreign registered facilities to review all of the data provided in each facility's registration to ensure it is current.

You must update any existing facility registrations if any of the mandatory information in the registration has changed and the facility is continuing in business under the same ownership as existed when the initial registration for the facility was submitted;
You should update any existing facility registrations if any of the optional information in the registration has changed and the facility is continuing in business under the same ownership as existed when the initial registration for the facility was submitted;
You must cancel any existing facility registrations if the facility is no longer in business or the facility is under different ownership than existed when the initial registration for the facility was submitted; new owners must submit a new registration for any facility that is subject to the final rule (21 CFR Part 1, Subpart H) that will continue in operation.
You should cancel duplicate registrations for the same facility; only one registration per facility is required.

If you need help:

Guidance on how to accomplish each of these activities is available at Registration of Food Facilities.
Tutorials are available at FDA Industry Systems Index of Help Pages.
FDA Industry Systems Help Desk personnel are available Monday through Friday, 7:30am until 11:00 pm Eastern Time, excluding U.S. government holidays:
- Phone: 1-800-216-7331 (within the U.S.) or 301-575-0156 (outside the U.S.)
- Fax: 301-210-0247
- E-mail: see contact form at FDA Industry Systems Help Desk or send e-mail to FDA at industry@fda.hhs.gov.

Appendix A: E-mail Test Message

Dear <Emergency Contact or U.S. Agent>,

The United States (U.S.) Food and Drug Administration (FDA) is conducting a test to verify the accuracy of the Emergency Contact information contained in our food facility registration database. FDA established this registration database in response to Section 305 of the Public Health Security and Bioterrorism Preparedness and Response Act of 2002 (the Bioterrorism Act) as implemented by the Registration of Food Facilities Final Rule at 21 CFR Part 1, Subpart H. This test will help to ensure that the FDA can quickly notify facilities that may be affected by an actual or potential threat to the U.S. food supply.

You are listed as the Emergency Contact or U.S. Agent for [name of facility], which is registered with the U.S. FDA.

The FDA respectfully requests a reply to this e-mail; please check the applicable statement below to verify that you are or are not the Emergency Contact or U.S. Agent for the above named facility.

"I agree that I am the Emergency Contact or U.S. Agent for the facility listed above. <Button>"

"I am not the Emergency Contact or U.S. Agent for the facility listed above. <Button>"

If the FDA's registration database lists you as the Emergency Contact or U.S. Agent for more than one facility, you may also receive a separate message pertaining to one or more of these facilities as part of this test. Please reply to each of the messages you receive to ensure that our testing is complete.

If we do not receive an e-mail reply from you, we will follow-up by contacting you by telephone.

Thank you for your prompt attention to this important activity intended to help safeguard the U.S. food supply.

Appendix B. Fax Test Message

Dear <Emergency Contact Person or U.S. Agent>,

The United States (U.S.) Food and Drug Administration (FDA) is conducting a test to verify the accuracy of the Emergency Contact information contained in our food facility registration database. FDA established this registration database in response to Section 305 of the Public Health Security and Bioterrorism Preparedness and Response Act of 2002 (the Bioterrorism Act) as implemented by the Registration of Food Facilities Final Rule at 21 CFR Part 1, Subpart H. This test will help ensure that the FDA can quickly notify facilities that may be affected by an actual or potential threat to the U.S. food supply.

You are listed as the Emergency Contact or U.S. Agent for [name of facility], which is registered with the U.S. FDA.

The FDA respectfully requests a response to this fax. Please check one of the two boxes below to verify that you are or are not the Emergency Contact or U.S. Agent for the facility listed above and fax your response to ORA/PPF at 301-210-0247.

"I agree that I am the Emergency Contact or U.S. Agent for the facility listed above. <Box or blank line to check>

"I am not the Emergency Contact or U.S. Agent for the facility listed above. <Box or blank line to check>

If the FDA's registration database lists you as the Emergency Contact or U.S. Agent for more than one facility, you may also receive a separate message pertaining to one or more of these facilities as part of this test. Please respond to each of the messages you receive to ensure that our testing is complete.

If we do not receive a fax response from you, we will follow-up by contacting you by telephone.

Thank you for your prompt attention to this important activity intended to help safeguard the U.S. food supply.

Appendix C: Phone Scripts for Emergency Contact Test

Note: If a phone number is dialed and the operator states that the number dialed has been changed or that the area code of the phone number has been changed, redial using the correct phone number and utilize the applicable phone script below. Record the correct phone number on the data tracking record.

Scenario 1: Person answers the phone; only Emergency Contact information provided for the domestic facility or the U.S. Agent of the foreign facility contains a phone number.

Hello. I am a representative from the U.S. Food and Drug Administration. We are conducting a test to verify the accuracy of the Emergency Contact information contained in our Food Facility Registration Database. FDA established this registration database to comply with the requirements in the Bioterrorism Act of 2002. This test will help ensure that the FDA can quickly notify facilities that may be affected by an actual or potential threat to the U.S. food supply.

You are listed as the Emergency Contact or U.S. Agent for [name of facility], which is registered with the FDA in accordance with the Registration of Food Facilities Final Rule at 21 CFR Part 1, Subpart H.

We are contacting you to verify whether you are or are not the Emergency Contact or U.S. Agent for the above named facility. Are you the Emergency Contact or U.S. Agent for this facility? [wait for response] Thank you for your cooperation and time.

Note: If the person on the phone is not the Emergency Contact/U.S. Agent, ORA/PPF will record this information on the data tracking record.

Scenario 2: Person answers the phone; attempts to e-mail or fax Emergency Contacts/U.S. Agents/U.S. Agents were not successful - e-mail address and/or fax information available is not accurate.

We were unable to successfully transmit [select one: an e-mail test message or fax test message] using the Emergency Contact information that is in our registration database. We are now contacting you by telephone to determine if you are the Emergency Contact or the U.S. Agent for the above named facility.

Are you the Emergency Contact or U.S. Agent for this facility? [wait for a response] Thank you for your cooperation and time.

Note: If the person on phone is not the Emergency Contact/U.S. Agent, ORA/PPF will record this information on the data tracking record.

Scenario 3: Person answers the phone; e-mail or fax test messages transmitted successfully but no response was received from the recipient.

You are listed as the Emergency Contact or U.S. Agent for [name of facility], which is registered with the U.S. Food and Drug Administration (FDA) in accordance with the Registration of Food Facilities Final Rule at 21 CFR Part 1, Subpart H.

FDA has successfully transmitted [select one - an e-mail test message or fax test message] to the Emergency Contact or U.S. Agent of the above named facility and requested a response to verify that the recipient is or is not the Emergency Contact or U.S. Agent for this facility. We did not receive a response as requested. In follow-up to the test message transmitted, we are now contacting you by phone.

Are you the Emergency Contact or U.S. Agent for the facility listed above? [wait for a response] Thank you for your cooperation and time.

Note: If no response, ORA/PPF will record this information on the data tracking record.

Scenario 4: Answering machine answers the phone; leave the following voicemail message. When a return call is received, use the applicable scenario -- 1-3.

We are contacting you to verify that you are or are not the Emergency Contact or U.S. Agent for the above named facility. Please contact the FDA, at your earliest convenience, Monday through Friday, between 9:00 am - 11:00 pm ET, at 1-800-216-7330 to let us know whether you are or are not the Emergency Contact or U.S. Agent for this facility. Again, the number to call is: 1-800-216-7330. Thank you for your time.

Note: If a return phone call is not obtained, ORA/PPF will record this information on the data tracking record.

⁽¹⁾FDA selected e-mail as the primary mode of transmission for this test, as this mode of communication is the most efficient way for FDA to notify facilities quickly in the event of a true emergency. If a domestic or foreign facility did not provide an e-mail address for its Emergency Contact (other than a U.S. Agent), FDA notified the Emergency Contact by telephone and considered the telephone number the primary mode of contact. If the owner, operator, or agent in charge of a registered foreign facility did not provide an e-mail address for its U.S. Agent who also was designated as the facility's Emergency Contact, FDA contacted the U.S. Agent by fax if a fax number was provided, or by telephone, if a fax number was not provided, and considered the fax and telephone, respectively, the primary mode of contact for these facilities.

⁽²⁾ The test was intended to capture the accuracy of the Emergency Contact/U.S. Agent data that facilities provided to FDA in their registrations to determine whether FDA actually could establish both physical contact through e-mail, telephone, and/or fax, and whether FDA could establish actual contact with the true Emergency Contact/U.S. Agent. The latter in particular is contingent upon whether owners, operators, and agents in charge of registered facilities update mandatory information, including Emergency Contact/U.S. Agent information, within 60 days of a change, as required by the final rule. Accordingly, FDA selected for its domestic test 400 facilities that actually had provided contact information, which was 98.6% of the domestic facilities in the Food Facilities Registration Database.