The CPPOA Model - An Illustration

The following example illustrates how cost data can be collected and analyzed for specific resource-procedure combinations for individual patients. These data were obtained from an aftercare program that was required of patients as part of their probation.

Program Context

The aftercare program usually included 3 months of participation in a therapeutic community. The aftercare component was administered in an office of the therapeutic community facility. A single counselor ran the aftercare component with minimal support from a secretary and supervision from the director of the program. The caseload usually was 30, all of whom were in a work release therapeutic community. Patients attended one group counseling session and one relapse prevention group per week. Patients also participated in one individual counseling session per month. The counselor provided case management services such as referrals, employment monitoring, and coordination with probation and parole officers.

Cost data were collected directly from staff -their best estimates of which relationships existed among all the different resources, procedures, processes, and outcomes.

After defining the essential resource, procedure, process, and outcome variables of the drug treatment program, the evaluator, the program director, and the aftercare worker estimated the relative strength of each possible relationship between each resource, procedure, process, and outcome. The existence and strength of these resource-procedure, procedure-process, and process-outcome relationships were estimated, rather than measured empirically, to conserve time and money.

The strength of these links need not be expressed in monetary units or percentages, but the staff of this program were comfortable doing this. The result carried forward costs from resources all the way through outcomes, making for a unique cost-effectiveness analysis.

Although the numbers the staff provided could have been simplified to make this example easier to calculate, it would have removed the realism of this analysis. Also, the example retains the actual (and perhaps idiosyncratic) resources, procedures, processes, and outcomes that the staff listed for their programs.

Resource-Procedure Relationships

Resources and procedures were easy for staff to specify: Resources were what were consumed in treatment procedures, and procedures were the actions performed on patients.

Resources and Their Costs

Staff seemed surprisingly comfortable with estimating costs. Table 27 shows the estimated costs for 1 month for each major resource type.

Table 27. Staff Estimate of Resources and Costs for One Month

Treatment Procedures

The next step was to ask staff what procedures usually were administered to patients. The procedures the staff listed were -

• Group counseling.

• Relapse prevention.

• Individual counseling.

• Case management.

To provide a structure for putting numbers on resource-procedure relationships, each possible combination of a resource and a procedure was listed in a resource x procedure matrix (table 28).

Table 28. Resource x Procedure Matrix With Cost Estimates for Each Procedure

Amount of Each Resource Used in Each Procedure

The final step in translating resource-procedure relationships into numbers was to put numbers in each cell of the resource x procedure matrix. These numbers were found in two basic steps. First, the time of direct service staff, the time of support staff, and the costs of supplies and of urine testing were distributed among procedures according to estimated use in the procedures. The entire $1,000 cost of urine testing was allocated to relapse prevention because it was not used in any other treatment procedures. Because support staff assisted primarily with relapse prevention and case management, support staff costs were divided equally between these two procedures.

Next, costs of the remaining resources were allocated among all four treatment procedures according to the percentage of time direct staff spent on each procedure:

• 18 percent for group counseling

• 23 percent for relapse prevention

• 23 percent for individual counseling

• 36 percent for case management

These percentages were based on careful estimates made by the program administrator.

Change in Processes for Individual Patients

One of the people conducting this CPPOA, the program director, and the aftercare worker described three types of psychosocial process that were the focus of treatment procedures:

• Self-efficacy expectancies

• The acquisition of necessary skills, specifically:
  • Relapse prevention skills
  • Support access skills
  • Services access skills

• Bonding with:
  • Addicts and ex-offenders
  • Counselors

Illustrative, hypothetical numbers were created for Patients A and B before and after participation in the program's treatment procedures. For example, according to the questionnaire that measured patients' self-efficacy expectancies, Patient A and Patient B scored an 8 before treatment began. After treatment, Patient A scored 12 and Patient B scored 16. Percentage change was used to examine how much processes changed during treatment (table 29).

Table 29. Percentage Change of Psychosocial Processes for Patients A and B

Data were preserved at this individual patient level, as well as being averaged, because process data is combined with outcome data in the analysis of cost-procedure-process-outcome relationships. The following hypothetical data are simplified to make the calculation procedures more obvious.

Procedure-Process Relationships: Patient Level

Table 30 illustrates how the strength of relationships between (a) the degree to which procedures were implemented and (b) the extent to which psychosocial process changed could be described for individual patients. The percentages in the cells of Patient A's procedure x process matrix are the same percentages calculated in table 29 for change in processes. For example, 50 percent was entered in each cell in the self-efficacy expectancies column. These data would be even more precise if the portion of each procedure that was devoted to changing each process were specified. This, however, may be difficult to measure. Some correlational statistical techniques, such as multiple regression, may help to do this.

Table 30. Procedure x Process Matrix for Patient A

Procedure-Process Relationships: Program Level

Staff of the aftercare program estimated the percentage of time that a given treatment procedure focused on modifying specific psychosocial processes (table 31). All four procedures were described as affecting at least two different psychosocial processes; all four procedures contributed via multiple processes to treatment outcomes. Even if one of the processes (say, support access) were not affected by relapse prevention, other processes would be. Staff felt comfortable using percentages (rather than correlations or other measures) to estimate the strength of relationships between procedures and processes.

Table 31. Procedure x Process Matrix for the Program

Adding Costs to Procedure-Process Relationships

The percentages were then used to distribute the total cost of each procedure among the processes (table 32). The total cost of changing each of the five processes can be calculated by totaling costs in the respective column.

Table 32. Procedure x Process Matrix With Cost Estimates

Outcomes

Process-Outcome Relationships

The psychosocial processes listed in the preceding tables were supposed to increase the likelihood that a patient would achieve four primary outcomes, defined by program staff as -

• Being drug free, that is, abstaining from drugs for a month, according to urine tests, self-reports, and peer reports (i.e., from other former users).

• Having stable employment, that is, having a legal full-time job for the past month.

• Being crime free, that is, avoiding all criminal behavior for the past month, according to self and peers as well as reports from family and probation officers.

• Being compliant with probation and parole according to the probation officer (who met weekly with the aftercare worker).

The question was, were these outcomes actually related to changes in the previously mentioned processes? More concretely, what were the connections between the processes and outcomes, according to staff and according to research?

Percentage Contributions of Processes to Outcomes

Staff also were asked to estimate how much each psychosocial process determined each of the four types of program outcomes. It was fairly easy to obtain these estimates after several hours of discussion. Again, the staff wished to use percentages. The results are detailed in table 33, with percentages of each process assigned to different outcomes and summing to 100 percent for each process (each row). For example, staff estimated that 40 percent of the change produced by treatment procedures in self-efficacy expectancies contributed to the outcome of being drug free. Staff viewed relapse prevention skills as entirely (100 percent) focused on drug abstinence. Staff also reported that equal proportions of improved skills for accessing support affected the outcomes of drug abstinence, stable employment, absence of criminal behavior, and compliance with probation and parole.

Adding Costs to Process-Outcome Relationships

The total cost of changing a process was distributed among the outcomes according to the percentages given earlier by staff to describe relationships between processes and outcomes (table 33). The total cost of achieving each outcome was calculated by adding up the cost figures in each column. These costs sum to $5,400, the total cost of the program for the month. This does not reflect the total cost of achieving these outcomes. The total cost per outcome achieved per patient must include the cost of participating in the therapeutic community for 6 to 9 months, plus 3 months of the aftercare program. Unfortunately, data were not available for the therapeutic community program. Although it is tempting to assign a cost to these outcomes of three times the monthly cost, and to then divide the cost by the proportion of patients attaining the outcome to arrive at a cost/outcome ratio, this would seriously underestimate the cost of attaining these outcomes. That ratio completely omits the costs of the therapeutic community.

Table 33. Adding Costs to Process-Outcome Relationships

Qualitative/Quantitative Path Analysis

By constructing bar graphs of the amounts of resources focused on each procedure, process, and outcome, it is easy to see where the costs are and what outcomes they provide. For example, it is evident in graph 8 that the most costly procedures are relapse prevention and case management. Also, some processes absorb far more resources than do others. As shown in graph 9, self-efficacy enhancement, skill acquisition for service access, and both types of bonding are particularly large investments of potentially therapeutic resources.

However, the outcomes associated with these procedures and processes differ in both the cost of resources devoted to them and the degree to which patients achieved what was desired. The outcome toward which the least amount of resources was directed, being crime free (graph 10), was the most likely to be achieved (by 100 percent of patients). The outcome toward which the most resources were directed (stable employment) was the least likely to be achieved (by a relatively low 65 percent of patients). These costs may reflect the program manager's expectation that stable employment would be the most difficult to achieve and thus deserved more resources. Nevertheless, the cost findings for each class of variables in the CPPOA model are of potential value in program management.

Most of the relapse prevention efforts resulted in a 90-percent abstinence rate. The CPPOA model also shows that several other procedures contributed to this outcome. However, the case management procedure produced a less impressive outcome. By connecting procedures to processes to outcomes, it becomes clear that much of the case management effort is related to the employment outcome. Yet, stable employment (steady work sufficient to support the patient and his or her dependents) is the outcome attained by the lowest percentage of patients. Perhaps this outcome would have been worse without case management, but it does call into question the value of this procedure for program outcomes. It also is interesting to note how much bonding to counselors was estimated to contribute to outcomes.

Integrating Qualitative and Quantitative Models for Formative CPPOA

The CPPOA model and its associated costs and outcomes (and cost-outcome ratios) are based on estimated and informally observed findings generated over the short term, rather than entirely objective measures collected using instruments of proven reliability and validity over several months or years.

The result is more of a qualitative and subjective, rather than a quantitative and objective, understanding of treatment. The qualitative CPPOA diagram and its associated estimates of costs and outcomes can be used as a sort of baseline against which to compare more quantitative data during data collection. Regular updates of the model can contrast and replace estimations with observations, showing staff how closely their understanding of the program matches the understanding provided by more objective measures.

With information on cost-procedure, procedure-process, and process-outcome relationships like that shown in the preceding example, the CPPOA model can then be used to make decisions about program changes or developments. In this example, it seems reasonable to keep intact the procedures and processes related to the abstinence outcome. In fact, the model affirms staff efforts in assisting patients in maintaining abstinence. Some staff, for example, questioned the efficacy of urine testing. Here it appears that urine testing is an important part of the procedures that produce the desired processes and outcomes.

Examining the cost-procedure-process-outcome model, staff can see that the case management efforts aimed at improved employment status may not be producing the desired outcomes. Seeing that a different approach, one aimed at skills acquisition and self-efficacy, was more productive in maintaining abstinence, staff may decide to decrease some of the time devoted to case management to allow for a more focused skills-building and problem-solving employment group.

This brief description shows how the CPPOA model can be used to make decisions about program changes. Many other program descriptions are embedded in the sample above. From these descriptions and connections between costs and outcomes, a variety of more informed program decisions can be made.

Many program managers will recognize ways to reduce the cost of treatment as soon as figures show up in a resource x procedure matrix. The decisions inspired by a resource x procedure matrix are, however, only as good as the data on which they are based. Although estimates such as those made above are enticingly quick and (relatively) easy to generate, their validity is suspect. With so much in the balance, there may be some temptation to bias estimates in favor of one's favorite procedures. It also can be very tempting to underestimate the cost of one's own role in providing treatment. Sometimes one does not realize the presence or strength of this bias.

Although even cost data collected with carefully constructed questionnaires administered by persons not directly involved in treatment can be biased, entirely estimated cost data may be more biased. If cost estimates are used, as in the example developed here, the validity of these estimates needs to be supported -perhaps by collecting some cost data in the more careful, expensive way and comparing the estimated to the observed costs.

CPPOA Research Design

If you have been trained in research design, you may be wondering about the role of research design in CPPOA. The answer is that both experimental and correlational designs can provide useful information for CPPOA.

Experimental CPPOA

Most experimental designs carefully manipulate the procedure part of the CPPOA model, usually presenting different procedures to different patients. Sometimes the procedure to which some patients are assigned is to simply wait, whereas others receive treatment immediately. A random lottery is used to decide which patients should wait and which should receive treatment right away. Outcome measures may be administered to the waiting-list control group, so that researchers can tell how much of the improvement in patients who received actual treatment procedures might be due to (a) the effects of repeatedly administering the same outcome measures and to (b) factors other than treatment procedures.

In variations of this experimental design, patients may be assigned randomly to treatment procedures that begin after different delays. Sometimes entirely different procedures are compared for effectiveness; sometimes different mixtures of procedures are compared.

Correlational Designs and CPPOA

In some treatment settings, it makes sense to assign patients to short-term waiting lists; sometimes there is more demand for services than there are services available. In some programs, too, the procedures have not yet been proven to be effective and need to be tested before being used with many patients. In most programs, however, all patients must receive treatment immediately. Patients sometimes can be assigned randomly to different groups of treatment procedures, such as usual treatment versus new experimental treatment.

Often, patients receive mixtures of treatment procedures that have been carefully tailored to their individual needs, problems, and financial and employment situations. In these circumstances, CPPOA becomes a correlational rather than an experimental analysis. Correlational analyses can accurately describe cost-effectiveness and cost-benefit relationships and can provide the basis for systematic improvement. Sophisticated statistical techniques, as well as tables and graphs, can be used to explore the strength of relationships between costs, procedures, processes, and outcomes.

[Manual Index] [Previous Section] [Next Section - References]