5.0 Project Findings

5.1 Finding 1: 
Identify conditions that should take precedence in EMS outcomes research

During phase one of this portion of the project, a list of EMS conditions was developed. An EMS condition was defined as an illness, injury or combination of signs and symptoms that caused EMS activation. A preliminary list of such conditions was identified using the NHTSA Uniform Data Conference data element items "Provider Impression" (data element 50), "Signs and Symptoms Present" (data element 52) and "Injury Site and Type" (data element 53) [NHTSA, 1993].

In phase two, frequency data were obtained for all the conditions identified. No local, state or federal databases were suitable for use due to inconsistent data definitions, inconsistent data formatting, and variation in inclusion criteria. EMS Data Systems, Inc. (Phoenix, Arizona) was selected to provide frequency data. EMS Data Systems collects data from various EMS systems across the country using optically-scanned data entry forms and data sets similar to that promulgated by NHTSA. Data from July 1, 1995 through June 30, 1996 were obtained from Alabama, Mississippi, Oklahoma, Illinois and eleven central California counties and used for the frequency analysis.

In phase three, the relevance of various outcomes and the potential impact of EMS on these outcomes, for each condition, was determined. Lacking meaningful outcome data, the investigators obtained expert opinions from 37 EMS researchers and leaders regarding the relevance and potential impact of EMS (see Appendix I). Respondents were asked to complete two questionnaires, one for patients less than 15 years of age and another for patients 15 years of age or older (see Appendix II). Two questions were included in the adult and child questionnaire: 1) For each of the following conditions, how would you rate the relevance of the following 6 outcome categories? and 2) For each of these conditions, how would you rate the potential impact of EMS (including both basic and advanced EMT care) on each outcome? The six outcome categories were defined as: survival (death), impaired physiology (disease), limit disability (disability), alleviate discomfort (discomfort), satisfaction (dissatisfaction), and cost (destitution). Respondents were asked to note the relevance and potential impact of EMS for each condition/outcome category on a 5-point scale ranging from (1) low to (5) high impact.

In phase four, a summary index score was developed and the conditions ranked according to this index. For each condition, a standard normal deviate (position of score on a standard normal distribution with mean of 0 and standard deviation of 1) was calculated. This transformation gives all variables the same mean and standard deviation. The frequency count and weighted score for each condition and age category were then multiplied together (in order to have positive signs after multiplication, negative z-scores were eliminated by the linear transformation of adding 4 to each score). Using this summary index score, the conditions were rank ordered for each age category.

The internal consistency of the total score (summed over the 6 Ds) for each condition was determined by calculating Cronbach's alpha. Correlation between impact and relevance scores was determined using the Spearman rank correlation coefficient. Agreement of rankings between respondents was measured in two different manners. First, respondents were divided into three, mutually exclusive, categories: 1) Physicians and non-physicians that were investigators or co-investigators; 2) Other physician respondents; and 3) Non-physician responders. Rankings were calculated within groups, then correlations among the three were calculated (Spearman rank correlation). Second, the average correlation among rankings by all respondents was calculated.

A sensitivity analysis was conducted by calculating index scores and ranking conditions using only survival scores and then using only discomfort scores. The rankings from each calculation were then compared to the original index score ranks that used all outcome categories, and Spearman rank correlation coefficients were calculated. See Tables 1 and 2.

Following data analysis, EMSOP investigators and consultants met to determine what conditions were to be recommended as priorities for EMS. Criteria for selection were based on summary index scores, the proportion of EMS transports represented, and the feasibility of identifying risk adjustment measures and outcome measures for these conditions within the time and resource constraints of the project. For adults, these conditions (in order of importance) are minor trauma, respiratory distress, chest pain, major trauma, cardiac arrest, airway obstruction, and respiratory arrest. For children, these conditions (in order of importance) are minor trauma, major trauma, respiratory distress, airway obstruction, respiratory arrest, cardiac arrest and seizure. For adults, the top quartile conditions account for 65% of adult emergency transports and for children, 85% of emergency transports. Relief of discomfort is the outcome parameter EMS professionals identified as having the most potential impact for the majority of both adults and children in the top quartile conditions. EMS research priorities focusing on these conditions will ensure that scarce resources will be directed to conditions that not only affect a substantial portion of EMS patients, but also have the potential of providing the greatest benefit (see Tables 3 and 4) [Maio, 1999].

5.2 Findings 2 and 3: 
Determine the appropriate risk adjustment measures for the priority conditions identified; Determine the appropriate outcome measures for the priority conditions identified.

5.2.1 Core Risk Adjustment Measures

Risk adjustment allows better judgment about the effectiveness and quality of alternative therapies; it fosters a comparison of apples with apples. By measuring risk adjustment measures (RAMs), researchers account for an important source of variation in their studies (Seidel et al., 1999). Core RAMS are necessary for prehospital outcomes research involving any EMS condition. Core RAMs should also be used, if possible, in retrospective EMS studies and for prospective prehospital investigations where applicable. The selection of the core RAMs by the EMSOP project team was based on four criteria: 1) they are "medically meaningful" for any EMS condition (ie, there is a possible connection between the RAM and any outcome pertinent to EMS); 2) data for the RAMs is readily available or measurable in the prehospital environment or may be obtained through linkage to other healthcare databases; 3) the RAMs have been described as risk attributes for outcomes research in other health settings; and 4) they were agreed to by all the EMSOP investigators and consultants.

The EMSOP investigators and consultants determined that the following core RAMS should be measured at the time of prehospital care: Patient age, sex, race and ethnicity, vital signs, level of responsiveness, GCS, time intervals, and the EMS provider impression (Zuckerman et al., 1998; Garrison et al., 2002; Donabedian, 1987; Fletcher et al., 1988; Shann et al., 1997; Pearson et al., 2001) [See Table 5].

The routine vital signs for the EMS patient are heart rate, respiratory rate, and blood pressure. Vital signs should be obtained and recorded as part of the initial assessment of every EMS patient. Since vital signs may also be used as an outcome measure, they should also be measured before and after an intervention and at the time of final EMS assessment. While there is no universally accepted method for assessing the level of responsiveness, the AVPU method (alert, verbal response, painful response, unresponsive) should be used. Since the GCS may also be used as an outcome measure, it should also be measured before and after an intervention and at the time of final EMS assessment. The time intervals that should be measured as RAMs are the event-to-treatment interval (ETI) and the prehospital treatment interval (PTI). To calculate the ETI, you need the single point in time marking the start of the event (using the time the call is received at a public safety answering point) and the single point in time of the arrival of the EMS providers at the side of the patient. The PTI is calculated from the single point in time of arrival at the patient until single point in time that marks the arrival at the hospital.

The EMSOP investigators and consultants determined that other core RAMs should be obtained through linkage to other data sources, to include principal diagnosis and patient co-morbidity (see Table 6).

Principal diagnosis is a critical element in risk adjustment since risks may differ significantly depending on the diagnosis (Iezzoni, 1994). For most prehospital patients, the diagnosis used in risk adjustment for prehospital outcomes research will be the principal diagnosis received in the hospital. Patient co-morbidity includes all those conditions and complications that may put a patient with them at risk for a different outcome. A patient's co-morbidity can affect both short and long-term outcomes (Iezzoni, 1994).

5.2.2 Core Outcome Measures

In identifying core outcome measures, we used four criteria that were very similar to those used for identifying risk adjustment measures: 1) they are "medically meaningful" for any EMS condition; 2) data for the outcome measures are readily available or measurable in the prehospital environment or may be obtained through linkage to other healthcare databases; 3) the outcome measures have been used in outcomes research in other health settings; and 4) they were agreed to by all the EMSOP investigators and consultants.

The outcome categories identified for which specific outcome measures would be identified are the six "Ds": death, disease (disease progression), disability, discomfort, destitution (cost), and dissatisfaction. Of note is that risk adjustment parameters that measure disease progression and discomfort can also be used as outcome measures and, therefore, core outcome measures. All of these core outcome measures can be measured or collected by EMS professionals in the prehospital setting.

The outcome parameter of mortality has been extensively used in EMS research and should continue to be used. The interval of care during which the patient was pronounced dead should also be obtained. At a minimum, mortality should be described using the following categories: death in the prehospital interval, death in the emergency department care interval, and death after hospitalization but before discharge. Death in intervals other than the prehospital interval will require data linkage outside of the direct control of prehospital systems. An outcome measure that is a crude indirect measure of disease severity and is related to mortality is patient disposition. Categories for this measure are treated/released at scene; treated and released from the emergency department; treated in the emergency department and admitted to hospital; discharged alive from hospital. As with mortality, obtaining information for disposition will require data linkage.

No core measures for disability were recommended for evaluation by the EMSOP investigators and consultants. Collecting measurements on this parameter would be very challenging for several reasons. First, no one functional outcome measure has been validated for all the priority conditions. Second, when disability measurements are obtained they are usually obtained at intervals distal to the prehospital and emergency department service intervals. Third, disability measures are not routinely obtained on emergency patients, whether or not they are brought to the emergency department by EMS. Thus, there are no data sources available for linkage for the broad array of priority conditions. The fact that no core disability measure was recommended for evaluation does not imply that prehospital care researchers should not continue the search for an appropriate core disability measure. Even though such a measure may not be routinely collected on EMS patients, it may have great value for specific research projects. Later in this report, we discuss disability measures that may be applicable to specific priority conditions.

With regard to the outcome category discomfort, an expert panel identified relief of discomfort as : 1) the most relevant outcome parameter in both adult and pediatric priority conditions; 2) the prehospital intervention that may have the greatest impact on patients (Maio et al., 1999). We think that for the overwhelming majority of patients in the prehospital setting, the measurement of pain is the most appropriate measure of discomfort. Due to its importance, we have devoted the entire next section of the report to pain measurement.

Patient satisfaction is an outcome measure that has been used for a variety of purposes in health care (Allen and Rogers, 1996). Satisfaction has been identified as an indicator of quality care (Doering, 1983) and can also be used to assess the performance of health care delivery at the organizational, unit and individual level. Patient satisfaction is also used in the development and evaluation of patient care models. Although we think that a measure of satisfaction should be obtained on all EMS patients, we were not able to recommend a satisfaction measure that should be evaluated in the prehospital care setting. We could find no report in the peer-reviewed literature of the development, evaluation and use of a patient satisfaction instrument, either generic or disease-specific, that has been used in the prehospital setting. Although there have been a number of studies that have addressed patient satisfaction in the emergency department, either as the primary or secondary outcome measure, only two instruments have reported information regarding their validity, reliability and limitations. It is readily apparent the field of prehospital care is in urgent need of an appropriately developed and tested instrument to assess patient satisfaction. One logical approach would be to develop a basic instrument that could be broadly applied, and eventually modified to address specific diseases. Researchers could begin developing prehospital patient satisfaction instruments by modifying existing measures such as the Group Health Association of America (GHAA) consumer satisfaction survey (Davies and Ware, 1991), or the Picker/Commonwealth survey (Pascoe, 1983). Furthermore, future research may be able to utilize the findings from the ongoing Consumer Assessment of Health Plan Study (CAHPS) sponsored by the Agency for Health Care Research and Quality (AHRQ).

The final outcome category, cost, is intended to capture not only the physical and personal outcomes that result from patient care, but also the costs and benefits of such care. The analyses of costs are typically studied using three models: 1) Cost-effectiveness analysis; 2) Cost-benefit analysis; and 3) Cost-utility analysis (Drummond et al., 1997). In a cost-effectiveness analysis, consequences are measured in natural units such as the amount of disability or health care resources consumed. Typically, there is a single effect of interest, common to both alternatives, but achieved to different degrees. Using the cost-benefit analysis, consequences are measured in dollars. This is similar to cost-effectiveness analysis except that there may be single or multiple effects of interest that are not necessarily common to both alternatives and common effects may be achieved to different degrees by the alternatives. In an analysis of cost-utility, consequences are measured in quality adjusted life years. Typically, cost-utility analysis is conducted when effects are weighted by utility measures denoting the patient's preference for, or the overall desirability of, a particular outcome (Gold, et al., 1996). Further complicating the picture is whether only direct costs are considered (for example acute medical care) or indirect costs (days of work missed) and what the perspective is: patient, insurer, or society. Furthermore, when considering direct costs there is controversy with regard to whether to use charges to patients and/or insurers versus "true costs". Charges for similar service may vary based on the insurer and "true costs" for a various episode of care can be variable based on the measures used to determine those costs.

A number of studies have appeared in the peer-reviewed literature that have examined the issue of cost as an outcome of pre-hospital care. All of the studies are cost-effectiveness studies. Cost-benefit and cost-utility studies do not appear in the peer-reviewed literature to date. The topics of published studies include the cost-effectiveness of helicopter transport including an analysis of helicopter EMS for trauma patients (Gearhart, et al., 1997); a comparison of air medical compared to ground transport (Spaite, et al., 1994), and air medical helicopter crash survival enhancements (Dodd, 1994). Another topic of cost-effectiveness studies is the issue of out-of-hospital cardiac arrest (Valenzuela, et al., 1991). Similar studies have considered potential improvements to EMS for victims of out-of-hospital cardiac arrest (Nichol, et al., 1996), and the cost-effectiveness of public access defibrillation (Nichol et al., 1997, 1998). Other topics appearing in the literature include expanding out-of-hospital care and its impact on hospital resource usage (Wade, 1996), the cost-effectiveness of trauma care (Elliott, 1996), and comparing ambulance dispatch protocols for nontraumatic abdominal pain (Lammers et al., 1995). The EMSOP investigators, however, could find no cost measure that could be uniformly applied to all EMS patients across many different systems. Future research is needed to develop, evaluate and implement cost determination methods that can be widely applied through the EMS community.

5.2.3 Pain

Inadequate pain control has also been recognized in the field of emergency medicine, both in the emergency department and in the prehospital setting. It has also been termed the "fifth vital sign". As discussed earlier, among priority conditions identified, discomfort had the highest weighted score for the top 3 adult first quartile conditions (minor trauma, respiratory distress and chest pain) and the first and third highest ranking children's first quartile conditions (minor trauma and respiratory distress). We think two measures of pain intensity should be evaluated for use among adults and older children in the prehospital setting. One measure uses an adjective response scale (ARS) [Jensen et al., 1986; McGuire, 1984; Keele, 1958; Ohnhaus and Adler, 1975; Woodforde and Merskey, 1972; Huskisson, 1974] and the other a numeric rating scale (NRS) [Jensen et al., 1986; Downie et al., 1978; Wilkie et al., 1990]. The first scale can be used to determine the quality of pain intensity and the latter to quantify pain intensity. Both of these scales have been found to be valid and reliable. These scales have also been found to be feasible to use in the emergency department. Two prehospital studies suggest that these instruments may be feasible to use in the prehospital setting. The ARS could be used upon initial assessment by pre-hospital care personnel and prior to relinquishing care of the patient to the emergency department staff (Ricard-Hibon et al., 1997, 1999). The following classification is recommended: None, Slight, Moderate, Severe, Agonizing. If a patient reports any category other than "None" he/she is asked to give a numerical rating to their pain from 0-100, with 0 being no pain and 100 being the worst pain imaginable. The numeric scale should be used at initial assessment and prior to turning over care to the emergency department staff. For younger children, self-report of pain can be utilized using the Oucher Scale (Beyer et al., 1992; Beyer and Knott, 1998; Villarruel and Denyes, 1991). This scale has been found to be valid and reliable but has not been evaluated in the emergency department or prehospital setting. Pain assessment instruments for infants are complex to use and are not currently feasible for use in the prehospital setting (Grunau et al., 1990, 1998; Craig et al., 1973, 1984; Attia, et al., 1987; Barrier et al., 1989; Horgan and Choonara, 1996; Lawrence et al., 1993; Taddio et al., 1995).

5.2.4 Condition-Specific Risk Adjustment and Outcome Measures

Next, the EMSOP investigators identified risk adjustment measures and outcome measures that should be measured or collected for specific EMS conditions. Measures were identified by a systematic literature search and a structured review of articles pertaining to that measure (see Appendix III for a description of the process used to identify measures). In addition, determination of outcome measures included review of several authoritative texts. Measures were evaluated using the following attributes: time taken to complete, cost and training, scaling, reliability, feasibility of use in the prehospital setting, and potential use as an outcome measure. After evaluation, the EMSOP investigators discussed each measure and a decision made to include or discard the measure. The methodology of the search and reviews, findings and recommended measures were presented to and reviewed by the consultant panel of experts.

5.2.4.1 Major/minor trauma

In phase one of the project, we found that 51% of pediatric transports and 36% of adult transports were for minor injury. Defining minor injury may be somewhat arbitrary. In the first portion of this project, the investigators used the Revised Trauma Score (RTS) to distinguish between major and minor trauma. An RTS of 11 or less was considered major trauma and an RTS of more than 11 was considered minor trauma (Champion et al., 1989). This is the criterion that has been recommended for use by field personnel to determine which patients need to go to a trauma center. Using this definition, 51% of pediatric transports and 36% of adult transports of the data obtained in phase one were classified as minor injury. This measure was selected because it is a standard measure in trauma care and evaluation and could be computed from the available data. However, this manner of defining "minor trauma" is a very crude one, in that it divides patients into those with higher and lower risk of death. Those falling into the low risk category based on RTS range from patients with sprained ankles to patients with splenic lacerations who are alert, oriented and have normal vital signs. Although a precise universal definition for minor trauma is lacking, it usually includes patients in whom the risk of death is minimal. While acknowledging the limitations of our method for defining minor and major trauma, we nonetheless will clarify which of the following recommended measures should be evaluated for use in minor trauma, major trauma or both. For major and minor trauma all previously discussed core risk adjustment and outcome measures will be used. It is important to note that the core measure of discomfort (or pain), may be the outcome parameter that is most relevant to those with minor trauma.

5.2.4.1.1 Risk Adjustment Measures: Major/minor trauma

Physiologic measures for trauma can be measured in the prehospital setting and are affected by time and treatment. The first measure, the Revised Trauma Score (RTS) uses physiological parameters to quantify injury severity-Glasgow Coma Scale, systolic blood pressure and respiratory rate. Specific ranges of each parameter are assigned coded values and also weights. Unweighted coded values range from 0-12, with 12 being no significant physiologic derangement. A patient with an RTS of 11 or less is considered at risk for experiencing major traumatic injuries. The RTS has been found to be a valid measure of physiologic injury severity in children (Eichelberger et al., 1989).

The second measure, the Glasgow Coma Scale (GCS) is a physiological injury severity score and is a crude but effective measure of the global function loss before and after resuscitation and prior to definitive treatment. This measure has been previously described under core measures. Although the RTS and GCS are accepted physiologic measures to risk adjust among patients with major injury, there are no physiologic parameters that have been identified for risk adjusting among minor injury patients. Both these measures can be obtained in the prehospital interval.

The Abbreviated Injury Scale (AIS) is the most widely used and accepted measure of injury severity based on anatomic descriptors (AAAM, 1990). It is an ordinal scale ranging from 1 (minor injury) to 6 (maximum injury, non-survivable). For the 1990 version injuries are grouped into 9 body regions. Several studies have confirmed the validity and utility of the AIS. The validity of AIS scoring for pediatric injury has been examined and several modifications made to accommodate scoring for children as well as adults. AIS scores should be determined for each patient. The AIS rates the severity of individual injuries only. The most widely used measure for combining AIS scores across body regions is the Injury Severity Score (ISS). This score was devised by Baker (1976) and was found to correlate better with mortality than did the AIS. The ISS is defined as the sum of squares of the highest AIS score in each of three most severely injured body regions. The highest score possible is 75. Low scores have been correlated with low probability of death and high scores with high probability of death. Although widely used, the ISS has been criticized for not taking into account the combined effect of multiple injuries within a single body region and underscoring the severity of isolated head injuries (Somers, 1981; Copes et al., 1988). The New Injury Severity Score (NISS) has been developed by Osler and Baker (1997) and addresses some of the limitations of the ISS while maintaining simplicity in definition and computation. The NISS is the sum of squares of the three most severe regions, regardless of body region. Initial studies suggest that the NISS offers a modest increase in predictive accuracy regarding mortality. One of the obstacles to determining the AIS and subsequently, the ISS and NISS is the need for complete and precise information on the extent of injury. This will require the sharing of both emergency department and inpatient information by receiving hospitals. Although the ISS score can be used for both major and minor injuries, there is very little variation in the ISS score among injury patients that are treated and released form the emergency department. Future research is needed to develop a valid and reliable minor severity score.

5.2.4.1.2 Outcome Measures: Major/minor trauma

Besides the usual core mortality measures, special mortality measures can be used in the evaluation of trauma care. With some of these methods, anatomical injury severity scores are combined with physiologic injury severity scores to calculate a probability for survival (Ps). One can then do a detailed examination of patients who died that were at or above a certain Ps, such as 0.75. Using TRISS or ASCOT methodology one can also compute various statistics that would allow one to compare the overall survival rate of one system, adjusted for Ps, with survival rates from other systems (Boyd et al., 1987; Markle et al., 1992). The preventable death rate can also be obtained through case review by a panel (Cales et al., 1984; Cayten et al., 1991; Maio et al., 1996; Esposito et al., 1995). Detailed information is collected only on those trauma patient who die. The method can be used even if injury severity scores of trauma patients are not routinely measured by the institutions caring for trauma patients within the EMS system. Using a well-defined structural review process will increase the reliability of the method (Maio et al., 1996).

The RTS and the GCS, which are used as risk adjusters, can also be used as an outcome measure to determine progression of the severity of the injury. These measures can be obtained in the prehospital interval or be obtained by linking to data sources from more distal intervals. This measure would be most useful among patients with major injury.

The Functional Independence Measure (FIM) is a measure of instrumental activities of daily living that is frequently used among patients admitted to trauma centers (Guyatt et al., 1996; Hamilton et al., 1987). It consists of 18 items that summarize level of independence in 6 areas: self-care, sphincter control, transfer mobility, ambulation, communication and social cognition. The FIM score is usually based on direct observation, but can also be derived using a structured questionnaire completed by self-report. The questionnaire can be administered in person or over the phone. It has now been applied to measuring outcomes following spinal cord injury, brain injury and severe orthopedic trauma (Brenneman et al., 1997; Bergner et al., 1981; Heinemann et al., 1997; Hetherinton et al., 1995). A limitation of the FIM that needs to be considered is that it is not sensitive to variations in higher end functioning that would be more typical of many orthopedic and mild to moderate brain injuries. A pediatric version of the FIM (Wee FIM) has also been developed (McCabe et al., 1990). Within the last few years a measure of Functional Assessment Measure (FAM) was developed as an adjunct to the FIM and contains items related to motor, cognitive and psychological aspects of everyday functioning (Hall et al., 1993; Hall et al., 1994). Use of the FIM would require linkage to hospital data. The FIM is not a measure applicable to minor trauma patients.

The Medical Outcomes 36-Item Short Health Survey (SF-36) was designed as a generic health indicator of health status for use in large population based studies (Ware et al., 1992; Ware et al., 1993; Ware et al., 1994). The SF-36 consists of 35 items or questions that are scaled to measure eight health concepts: physical functioning, role limitations due to physical health problems, bodily pain, general health perceptions, vitality, social functioning, role limitations due to emotional problems, and general health. When using the SF-36 to examine post-injury outcome, some modification to questions about change in status or abilities is recommended to ascertain changes that may have occurred since the injury.

Two summary scores can be derived that measure physical and mental health respectively. The SF-36 can be self-administered or administered by an interviewer and takes only 5 to 10 minutes to complete. The SF-36 can also be incorporated as part of telephone interview. There are few published examples of its application to injury (Beaton et al., 1994; Corrigan et al., 1998; Kopjar et al., 1996; Martin et al., 1997; McCabe,1996). MacKenzie et al. (1998) has shown that SF-36 does not discriminate well among major trauma patients with and without head injury. However, by supplementing the SF-36 with 3 or 4 items, one can derive a subscore specific to cognitive function, that when used in combination with the standard eight subscores of the SF-36, provides a health profile more relevant to injury. The SF-36 has also been used to measure changes in functional outcome among patients with minor trauma (Kopjar, 1996). The SF-36 can definitely be used in the evaluation of major trauma and may also have a role in the evaluation of minor trauma. One of the problems with the SF-36 is that it is not routinely collected in all trauma patients. Therefore, it will most likely be a measure that can only be evaluated through specific research protocols that include collection of data that is not routinely obtained by the EMS system.

We think the FIM and SF-36 hold the most promise for measures of dysfunction among prehospital trauma patients. Currently, these are the two main functional outcome measures that are being used in a nationwide trauma evaluation funded by the CDC (National Study on Costs and Outcomes of Trauma Care (N-SCOT). There are other measures of dysfunction that EMS researchers may also want to consider. The Sickness Impact Profile (SIP) consists of 136 statements about limitations in 12 categories: sleep and rest, emotional behaviors, body care and movement, eating, home management, mobility, social interaction, ambulation, alertness behavior, communication, recreation, and work. It is one of the most comprehensive measures of health status, addressing most dimensions of health with considerable depth (Bergner et al., 1981). It is also sensitive to a broad range of levels of dysfunction. The SIP, however, does take 20-25 minutes to administer, making it impractical for many applications. Shorter versions have been proposed for specific applications but have not been widely validated (Gerety et al., 1994; Post et al., 1996; Sullivan et al., 1993). The SIP can be self-administered or administered by an interviewer. The SIP has been used in several studies describing the outcomes of spinal cord injury, head injury and orthopedic trauma (Beaton et al., 1994; Corrigan et al., 1998; Dikman et al., 1995; Fleming et al., 1998; Fuergemann et al., 1993; Gruen et al., 1995; Jurkovich et al., 1995; Richmond et al., 1998). In general, these studies support the validity of the SIP for trauma outcomes research. As with the FIM, the SIP would not be used in the evaluation of minor trauma. The Child Health Questionnaire (CHQ) was originally developed by Landgraf et al (1996) to measure the well-being of children and adolescents five years and older across 14 domains; a version of the CHQ applicable to toddlers is currently under development. Twelve of these domains can be combined into two summary measures of physical and psychosocial health status. Four different versions of the instrument exist consisting of 28 items, 50 items and 98 items respectively. It can be self-administered or interviewer administered and both parent and child versions exist for young and older children. To date, there are no published studies using this measure for injury. The Quality of Well-Being (QWB) Scale is a preference based measure of health that combines patient reported symptoms and disability into a single index that provides an expression of well being that ranges from zero for death to one for asymptomatic full functioning (Holbrook et al., 1998). It has been used to examine trauma outcomes, but still requires further evaluation.

Currently, there are no routine measures of satisfaction that are collected for trauma patients. Nor could we identify any measure that has been specifically evaluated in the trauma patient. As stated previously, we think that satisfaction measures should be applied for every prehospital patient. Future research is required to develop and evaluate such a measure.

We could find no standardized method of cost determinations that could be applied to the broad range of prehospital care trauma patients. Various methods have been used to determine costs among trauma patients who are admitted to hospitals. Some hospitals may routinely collect some of these measures. However, no one measure or set of measures is recognized as the standard measure for cost. Future research will be needed to identify and/or develop and evaluate measures of cost relevant to prehospital trauma care.

5.2.4.2 Respiratory distress

In the first phase of this project we found that 13% of all adult transports and 10% of all pediatric transfers were for respiratory distress; these conditions ranked 2 and 3, respectively, among priority conditions. When considering recommended measures for the evaluation of respiratory distress in the prehospital setting, several considerations were taken into account. We considered that for the condition respiratory distress, a patient may have various underlying diseases such as asthma, congestive heart failure, COPD, and pneumonia. We also considered there were certain diseases specific to young children, to include croup and bronchiolitis. Prehospital care providers are trained, and prehospital protocols are developed to treat symptoms and signs, not specific disease categories. The very nature of emergent care often precludes the accurate diagnosis of the disease that is causing the symptoms. Therefore, we had to identify measures that could be used among several diseases. Another particularly challenging issue in the process was identifying measures that would be useful among a broad age range. Since the specific disease states that would fall under the prehospital condition of respiratory distress have never been reported, we decided to identify measures that could be used for the specific diseases of asthma, COPD and congestive heart failure. The EMSOP investigators and consultants thought that these three conditions comprise the overwhelming majority of adult and pediatric transfers for respiratory distress. As with the condition of major/minor trauma, all core risk adjustment and outcome measures would be obtained for patients with respiratory distress. These core measures will not be discussed further in this section.

5.2.4.2.1 Risk Adjustment Measures: Respiratory Distress

Peak Expiratory Flow Rate (PEFR) is one of a group of forced expiratory flow measures taken at the point of total lung capacity or at the point of maximal inspiration (American Thoracic Society, 1987 and 1991). Wright first described the measurement of the PEFR, and the instrument used to obtain the measurement in 1959 (Wright, 1959). The PEFR is primarily an index of obstruction in large airways. The use of the PEFR to monitor patients with asthma has been widely adapted (Thoracic Society of New Zealand, 1996). Its use in the treatment and evaluation of chronic obstructive pulmonary disease has also been recommended (Aschrift et al., 1969; Tashkin, 1979). Studies in patients with congestive heart failure have found only mild decreases in PEFR in stable patients, but higher decreases in patients with severe symptoms of CHF (Hales et al., 1977; Light et al 1983, Peterman et al., 1987; Eriksson et al., 1987; Eichaker et al., 1988; Pison et al., 1989). Since Wright's first article, various other instruments have been developed to measure the PEFR. These have been inexpensive portable devices that have been calibrated using the Wright flow meter as a standard (Miller et al., 1992). A device that is frequently used today in the United States is the mini-Wright peak flow meter (Wright, 1978). Children as young as four years of age can be taught to use portable devices to measure PEFR (Jones et al., 1990). The validity of the PEFR has been measured using the FEV1 as a gold standard. In general, the PEFR correlates well with FEV1 (Nowak et al., 1982; Vaughn et al., 1989). Others have obtained similar results in stable and asthmatic children and in normals. A study by Vaughn et. al. (1989) noted correlations of 0.74. Studies conducted in emergency departments have found correlations ranging form 0.77-0.83 in asthma and 0.69 in COPD (Emmerman et al., 1989).

Oxygen saturation represents the percent saturation of available bound hemoglobin. The relationship between percent saturation and the partial pressure of oxygen in arterial blood (PaO2) is described by the oxyhemoglobin dissociation curve. The pulse oximeter (SpO2), is a fast growing and commonly utilized transcutaneous methodology that represents a combination of oximetry and plethysmographic technologies for screening, treatment planning and evaluation and research (Fait et al., 1985; Mihm et al., 1989; Yelderman et al., 1983). Evaluation of the accuracy and feasibility of pulse oximetry in the emergency department and prehospital setting have also been conducted (Jones et al., 1988; Aughey et al., 1991; McGuire et al., 1988). These studies have found the device accurate and its use very feasible. Precision of various pulse oximeters were noted be fairly consistent at approximately 2% (Taylor et al., 1988). The anatomic site of choice for pulse ox probe placement is the finger (Webb et al., 1991). Limitations of this device include severe anemia and low-flow states (Webb et al., 1991).

Dyspnea can be defined as the unpleasant sensation of labored or difficult breathing and is synonymous with the term shortness of breath. This is main parameter of discomfort in respiratory distress (Gift et al., 1993). Factors related to the presence of dyspnea can be classified as physiologic, psychologic, and situational or environmental (Gift et al., 1993). The visual analogue dyspnea scale (VADS) is a self-reported measure that has shown to be a valid measure of dyspnea, as well as practical in the critical care setting (Gift, 1986; Gift et al., 1986; Aitken, 1969). The patient is asked to indicate the degree of shortness of breath experienced by marking the line at the level indicating his or her level of discomfort (Gift, 1989). One end represents "not at all breathless" and the other end "worst possible breathlessness". Children must have fully developed communication skills in order to use these instruments. Studies in the non-emergency setting have reported that children as young as four could use the VADS.

The visual analogue scale has been validated in healthy volunteers as well as asthmatics and persons with COPD as well as in cancer patients suffering from a variety of underlying causes of dyspnea (Gift, 1989; Adams et al., 1985; Roberts et al., 1993; Mador et al., 1992; Subratty et al., 1994; Noseda et al., 1992). The test-retest reliability of the VAS has been shown to be high (Muza, 1990). Aitken (1969) was the first to demonstrate that a visual analogue scale was a valid measure of dyspnea. He used a horizontally oriented scale. Gift, Plaut and Jacox (1986) expanded Aitken's work by reorienting the scale to a vertical line and testing the scale in clinical situations using asthmatic patients and COPD. This reorientation facilitated ease and understanding of use by patients. The vertical VAS is a valid, reliable and easy-to-use instrument. It is an instrument that could be easily used in the prehospital setting (Gift, 1986).

5.2.4.2.2 Outcome Measures: Respiratory Distress

No special measures of mortality for respiratory distress were identified. The PEFR and Pulse Ox, risk adjustment measures for respiratory distress, can also be used as outcome measures for disease severity. These outcome measures can be obtained in the prehospital care interval or obtained through linkage of data from more distal service intervals. The VADS, used to initially measure discomfort for prehospital patients, can also be used as an outcome measure for discomfort. This outcome measure can be obtained in the prehospital care interval or, by linkage, from data of more distal service intervals.

No functional outcome measures that could be used for all patients with respiratory distress could be identified. Although there are disease specific measures of functional status that could be used among patients with respiratory distress (examples include the New York Heart Association Classification, St. George's Respiratory Questionnaire, Pulmonary Function Status Scale, Chronic Respiratory Disease Questionnaire). None of these measures are routinely collected on patients with respiratory distress who are treated and released from the emergency department or who are admitted to the hospital. Future research will be needed to determine the feasibility of using these and other specific respiratory disease functional status measures in conducting outcomes research and evaluation of EMS patients who present with respiratory distress.

Currently there are no routine measures of satisfaction that are routinely collected for patients with respiratory distress in the prehospital setting. Nor could we identify any measure that has been specifically evaluated among patients with respiratory distress. As stated in the section on core risk adjustment and outcome measures, we think that every prehospital patient should have a measure of satisfaction obtained. Future research will need to develop and evaluate such a measure.

We could find no standardized method of cost determinations that could be applied to the broad range of prehospital care respiratory distress patients. Future research will be needed to identify, or develop, and evaluate measures of cost relevant to prehospital care of patients in respiratory distress.

5.2.4.3 Cardiac Arrest

Of all outcomes research undertaken in the out-of-hospital setting, the highest quality to date has been in the arena of non-traumatic cardiac arrest. The EMSOP investigators recommend the use of the Utstein Style (Cummins et al., 1991a). Utstein has already enjoyed wide acceptance and a large amount of work is underway using the guidelines for cardiac arrest research (AHA, 2000; Cummins et al., 1991b; Steill et al., 1998; Steill et al., 1999a, 1999b; Hsu et al., 1996; Grubb et al., 1996; Callicot et al., 1995; Cobbe et al., 1996; Nichol et al., 1999; The Brain Resuscitation Clinical Trial II Study Group, 1991; Cobbe et al., 1996).

Almost nothing has been done in the methodological development of cardiac arrest research in children. Thus, much work will have to be done before any cogent recommendations can be made for pediatric cardiac arrest outcomes research. There are rumors of the development of Utstein Style guidelines for pediatrics, but no accepted or authoritative methods are extant at present. Therefore, EMSOP cannot make evidence-based recommendations at this time.

5.2.4.4 Chest Pain

We were unable to identify any risk adjustment or outcome measures specific to chest pain. We recommend that for chest pain, the general advice provided in EMSOP IV, Pain measurement in prehospital outcomes research be followed. Future research should focus on appropriate and specific out-of-hospital risk adjustment and outcomes measures for chest pain.

5.2.4.5 Respiratory Arrest/Airway Obstruction

We were unable to identify any specific risk adjustment or outcome measures for these entities. At the very least we would recommend core risk adjustment and outcome measures for these patients. Future research will need to develop and evaluate appropriate condition specific risk adjustment and outcome measures for respiratory arrest/ airway obstruction.

5.2.4.6 Pediatric Seizures

We were unable to identify any specific risk adjustment or outcome measures for these entities. At the very least we would recommend core risk adjustment and outcome measures for these patients. Future research will need to develop and evaluate appropriate condition specific risk adjustment and outcome measures for pediatric seizures.

5.3 Finding 4: 
Identify stakeholders and constituencies important for EMS outcomes research.

A list of stakeholders was developed in the second year of the project and modified in the fourth year with the input of the steering committee and consultants (see Appendix IV). These stakeholders and constituencies represent physicians, prehospital care providers, EMS and ambulance system administrators, government agencies, insurance companies and legislators. It is important that all project information disseminated be understandable to both the research community and EMS professionals at the state level and in the field.

A dialog was begun with the National Association of EMS Physicians in July, 1997 when the priority conditions were identified to alert leading prehospital researchers of the project and its progress. Since that time, annual updates have been provided to key EMS researchers at the Society for Academic Emergency Medicine Annual Meeting. The network of stakeholders and constituencies should continue to provide input in the development of the EMS outcomes research agenda, as well as serve as a resource for dissemination.

5.4 Finding 5: 
Develop a research dissemination plan.

The manuscripts developed as a result of this project will reach the desired audience and help pave the way for an EMS outcomes research agenda. Further information will be developed for the broad EMS research community, in dialog with the EMS office of NHTSA and using the list of stakeholders that has been developed by the project investigators and consultants. A one-day "think tank" was held on March 20, 2002 on Performance Measures in EMS that included a broad audience of the EMS community including state EMS directors, firefighters and others involved in prehospital care. Dr. Ron Maio presented EMSOP findings to ensure that the objectives of this project, particularly with regard to outcomes in prehospital care, were incorporated into the Performance Measures work group. On June 3, 2002, EMSOP results were presented at the National EMS Research Agenda Implementation Symposium sponsored by the National Association of EMS Physicians. Another goal of the symposium was to draft a plan for the implementation of the National EMS Research Agenda recommendations. Through all of these efforts, the ultimate goal is to conduct EMS outcomes research that will improve the delivery of prehospital care, and will thus benefit the general public.

Continue to 6.0 Limitations | Return to Contents