Quality of Research
Documents Reviewed
The documents below were reviewed for Quality of Research. The research point of
contact can provide information regarding the studies reviewed and the availability
of additional materials, including those from more recent studies that may have been conducted.
Study 1Wolchik, S., Sandler, I., Weiss, L., & Winslow, E. (2007). New Beginnings: An empirically-based program to help divorced mothers promote resilience in their children. In J. M. Briesmeister & C. E. Schaefer (Eds.), Handbook of parent training: Helping parents prevent and solve problem behaviors (3rd ed., pp. 25-62). New York: John Wiley & Sons.
Wolchik, S. A., West, S. G., Westover, S., Sandler, I. N., Martin, A., Lustig, J., et al. (1993). The children of divorce parenting intervention: Outcome evaluation of an empirically based program. American Journal of Community Psychology, 21(3), 293-331.  Study 2Tein, J. Y., Sandler, I. N., MacKinnon, D. P., & Wolchik, S. A. (2004). How did it work? Who did it work for? Mediation in the context of a moderated prevention effect for children of divorce. Journal of Consulting and Clinical Psychology, 72(4), 617-624.
Wolchik, S. A., Sandler, I. N., Millsap, R. E., Plummer, B. A., Greene, S. M., Anderson, E. R., et al. (2002). Six-year follow-up of preventive interventions for children of divorce: A randomized controlled trial. Journal of the American Medical Association, 288(15), 1874-1881.
Wolchik, S., Sandler, I., Weiss, L., & Winslow, E. (2007). New Beginnings: An empirically-based program to help divorced mothers promote resilience in their children. In J. M. Briesmeister & C. E. Schaefer (Eds.), Handbook of parent training: Helping parents prevent and solve problem behaviors (3rd ed., pp. 25-62). New York: John Wiley & Sons.
Wolchik, S. A., West, S. G., Sandler, I. N., Tein, J., Coatsworth, D., Lengua, L., et al. (2000). An experimental evaluation of theory-based mother and mother-child programs for children of divorce. Journal of Consulting and Clinical Psychology, 68(5), 843-856.
Supplementary Materials Arizona State University Prevention Research Center. (2001). Data collected from schools. New Beginnings Program codebook, version 1. Unpublished document.
Dawson-McClure, S. R., Sandler, I. N., Wolchik, S. A., & Millsap, R. E. (2004). Risk as a moderator of the effects of prevention programs for children from divorced families: A six-year longitudinal study. Journal of Abnormal Child Psychology, 32(2), 175-190.
Reliability and validity information on measures of putative mediators and outcome variables. (n.d.). Unpublished document.
Summary of Quality of Research information submitted to NREPP
Outcomes
| Outcome 1: Diagnosis of mental health disorder |
|
Description of Measures
|
Diagnosis of mental health disorder was assessed using computer-assisted parent and adolescent versions of the Diagnostic Interview Schedule for children (DIS-C). Diagnoses were derived separately for mental disorder and drug abuse or dependence and were based on meeting two conditions: (1) adolescents met symptom criteria for diagnosis of one or more disorders in the past year, based on self-report or parent report; and (2) two or more of the impairment items for the disorder(s) were rated as intermediate or severe, based on self-report or parent report.
|
|
Key Findings
|
At 6-year follow-up, 14.8% of adolescents whose families participated in New Beginnings were diagnosed with a mental disorder during the previous 12 months, compared with 23.5% of adolescents whose families received books on postdivorce adjustment (p = .02). No significant difference was found between the two groups in diagnoses of drug abuse or dependence.
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.7
(0.0-4.0 scale)
|
| Outcome 2: Symptoms of mental health problems |
|
Description of Measures
|
Symptoms of mental health problems were assessed using (1) the Child Behavior Checklist (CBCL), with which mothers reported on 118 items related to child behavior problems; and (2) computer-assisted parent and adolescent versions of the Diagnostic Interview Schedule for Children (DIS-C), which was used to calculate total symptom scores for mental disorders and for drug abuse or dependence.
|
|
Key Findings
|
Among youth with high levels of behavior problems before the program, those who participated in New Beginnings with their families had fewer behavior problems, as reported by their mothers, compared with youth awaiting care (p < .05).
In a second evaluation, among adolescents with high levels of symptoms before the program, those whose families participated in New Beginnings averaged fewer symptoms of mental health problems, compared with adolescents whose families received books on postdivorce adjustment (p < .001).
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.6
(0.0-4.0 scale)
|
| Outcome 3: Externalizing problems |
|
Description of Measures
|
Externalizing problems were assessed by the following instruments: (1) the Child Assessment Schedule (CAS), adapted into an interview format and shortened to include measures of depression and conduct problems; (2) the Youth Report of Hostility Scale, 30 items from the CBCL, adapted to a self-report format to measure aggression; (3) the Child Behavior Checklist (CBCL), completed by parents, which included a 33-item externalizing problems subscale; (4) the Youth Self Report, a 30-item adolescent self-report scale of externalizing problems; (5) the Divorce Adjustment Project Hostility Scale, a 27-item adolescent self-report scale of externalizing problems with items added to assess delinquent behavior; and (6) the Teacher-Child Rating Scale, a six-item acting-out subscale completed by teachers.
|
|
Key Findings
|
Youth whose mothers participated in New Beginnings had lower mean aggression scores compared with adolescents awaiting care (p < .05).
An evaluation of New Beginnings at 6-year follow-up found that for adolescents who were at high risk before the program, teachers, parents, and adolescents themselves reported fewer externalizing problems for New Beginning participants compared with adolescents whose families received books on postdivorce adjustment (p < .001).
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.8
(0.0-4.0 scale)
|
| Outcome 4: Internalizing problems |
|
Description of Measures
|
Internalizing problems were assessed using the following instruments: (1) the Child Assessment Schedule (CAS), shortened and adapted into an interview format to measure depression or conduct problems; (2) the Child Depression Inventory (CDI), a 27-item inventory completed by adolescents that assesses cognitive and behavioral symptoms of child depression; (3) the Child Manifest Anxiety Scale, a 28-item scale that measures chronic anxiety in children; (4) the Child Behavior Checklist (CBCL), a 31-item internalizing subscale completed by parents; and (5) the Shy-Anxious subscale of the Teacher-Child Rating Scale, six items completed by teachers.
|
|
Key Findings
|
Among adolescents who were at high risk before the program, those who participated in New Beginnings with their families had fewer self- and parent-reported internalizing problems at 6-year follow-up compared with youth whose families received books on postdivorce adjustment (p < .001).
However, another evaluation found that among youth who had high levels of depression at baseline, New Beginnings participants had more internalizing problems than a comparison group of youth waiting for care (p < .05).
|
|
Studies Measuring Outcome
|
Study 1, Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.8
(0.0-4.0 scale)
|
| Outcome 5: Alcohol use, substance use, and sexual behaviors |
|
Description of Measures
|
Alcohol and drug use were measured using the Monitoring the Future survey, self-administered to adolescents. Alcohol and marijuana use was measured using a 7-point scale indicating how many times in the past year the respondent used the substances. Other drug use was computed as the sum of ratings on a scale for 13 other drugs, and polydrug use was assessed by counting the number of different drugs, including alcohol, the respondent used in the past year. The Physical Intimacy Scale was used to assess risky sexual behaviors. Adolescents responded to a self-report questionnaire on the number of different sexual partners they had since completion of the New Beginnings Program.
|
|
Key Findings
|
Adolescents who participated in New Beginnings with their families had fewer sexual partners (p = .02) at 6-year follow-up compared with youth whose families received books on postdivorce adjustment. Among adolescents who were at high risk before the program, those who participated in New Beginnings with their families had lower rates of polydrug use (p = .02), alcohol use (p = .005), marijuana use (p = .05), and other drug use (p = .02) at 6-year follow-up compared with adolescents whose families received books on postdivorce adjustment.
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.7
(0.0-4.0 scale)
|
| Outcome 6: Competence (academic/social competence, self-esteem, and activity involvement) |
|
Description of Measures
|
Competence was a composite measure that included grade point average (computed from school data on unweighted GPA), activity involvement, and self-esteem. Children and mothers completed 18 items from the academic, activities, and peer subscales of the Coatsworth Competence Scale (CCS). Children also completed the 6-item Self-Perception Profile for Children (SPPC).
|
|
Key Findings
|
Adolescents who participated in New Beginnings with their families averaged higher GPAs at 6-year follow-up compared with youth whose parents received books on postdivorce adjustment (p < .01). Among adolescents who were at high risk before the program, those who participated in New Beginnings with their families had increased self-esteem (p < .05) and activities involvement (p < .05) compared with adolescents whose families received books on postdivorce adjustment.
|
|
Studies Measuring Outcome
|
Study 2
|
|
Study Designs
|
Experimental
|
|
Quality of Research Rating
|
3.8
(0.0-4.0 scale)
|
Study Populations
The following populations were identified in the studies reviewed for Quality of
Research.
|
Study
|
Age
|
Gender
|
Race/Ethnicity
|
|
Study 1
|
6-12 (Childhood)
13-17 (Adolescent)
|
61% Male
39% Female
|
90% White
10% Race/ethnicity unspecified
|
|
Study 2
|
6-12 (Childhood)
|
50.4% Male
49.6% Female
|
88% White
8% Hispanic or Latino
2% Black or African American
1% Race/ethnicity unspecified
0.8% Asian
0.2% American Indian or Alaska Native
|
Quality of Research Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the Quality of Research for an intervention's
reported results using six criteria:
For more information about these criteria and the meaning of the ratings, see Quality of Research.
|
Outcome
|
Reliability
of Measures
|
Validity
of Measures
|
Fidelity
|
Missing
Data/Attrition
|
Confounding
Variables
|
Data
Analysis
|
Overall
Rating
|
|
1: Diagnosis of mental health disorder
|
3.5
|
4.0
|
3.5
|
3.5
|
4.0
|
3.5
|
3.7
|
|
2: Symptoms of mental health problems
|
3.5
|
3.5
|
3.5
|
3.5
|
4.0
|
3.5
|
3.6
|
|
3: Externalizing problems
|
4.0
|
4.0
|
3.5
|
3.5
|
4.0
|
3.5
|
3.8
|
|
4: Internalizing problems
|
4.0
|
4.0
|
3.5
|
3.5
|
4.0
|
3.5
|
3.8
|
|
5: Alcohol use, substance use, and sexual behaviors
|
4.0
|
3.5
|
3.5
|
3.5
|
4.0
|
3.5
|
3.7
|
|
6: Competence (academic/social competence, self-esteem, and activity involvement)
|
4.0
|
4.0
|
3.5
|
3.5
|
4.0
|
3.5
|
3.8
|
Study Strengths The intervention was developed based on a solid body of knowledge of the mechanisms involved in effective prevention programs for children and youth. Most of the outcome measures used have sound psychometric properties and were appropriate given the issues under study and the age of the sample population. These studies were outstanding in terms of the methods that were in place to monitor fidelity of the intervention. Ample methods were included to evaluate participant attrition. The long-term study had very impressive participant retention at the 6-year follow-up (91% follow-up rate). The authors have done an outstanding job of evaluating and controlling for confounding variables. The use of intent-to-treat analysis contributes to the strength of the findings.
Study Weaknesses Neither study had a sample large enough to justify the high number of analyses undertaken, which reduces the confidence that the findings are robust and generalizable. A fidelity study was undertaken, but the authors did not present psychometric data on their fidelity measures.
|
|
Readiness for Dissemination
Materials Reviewed
The materials below were reviewed for Readiness for Dissemination. The implementation
point of contact can provide information regarding implementation of the intervention
and the availability of additional, updated, or new materials.
Cultural, age, and gender appropriateness of NBP (New Beginnings Program). (n.d.). Unpublished document.
Measure of Behavioral Problems in Mother-Child Relationship Quality (assessment instrument)
Prevention Research Center. (n.d.). New Beginnings Program: Group leader initial training manual. Tempe, AZ: Author.
Prevention Research Center. (n.d.). New Beginnings Program: Group leader weekly training manual. Tempe, AZ: Author.
Prevention Research Center. (n.d.). New Beginnings Program: Supervisor manual. Tempe, AZ: Author.
Wolchik, S., & Sandler, I. (2006). New Beginnings Program: Group leader manual. Tempe, AZ: Program for Prevention Research.
Readiness for Dissemination Ratings by Criteria (0.0-4.0 scale)
External reviewers independently evaluate the intervention's Readiness for Dissemination
using three criteria:
- Availability of implementation materials
- Availability of training and support resources
- Availability of quality assurance procedures
For more information about these criteria and the meaning of the ratings, see Readiness for Dissemination.
Implementation
Materials
|
Training and Support
Resources
|
Quality Assurance
Procedures
|
Overall
Rating
|
|
3.3
|
3.5
|
3.8
|
3.5
|
Dissemination Strengths Implementation materials are comprehensive, well written, and highly scripted. Manuals pay close attention to cultural, age, and gender appropriateness. Training, supervision, and coaching are thoroughly addressed through a detailed training program and developer support provided throughout implementation. A comprehensive, multilevel approach connects training/coaching opportunities to the quality assurance necessary for program implementation. Implementation sites can submit data collection results for technical assistance and adaptation support from the developer.
Dissemination Weaknesses Program materials may be cumbersome for general dissemination outside a university or research setting. Training requirements are extensive, and program use requires certification that is only good for a 2-year period. Fidelity measures, such as videotaping, may also prove cumbersome outside a university or research setting.
|
|
Costs
The cost information below was provided by the developer. Although this cost information
may have been updated by the developer since the time of review, it may not reflect
the current costs or availability of items (including newly developed or discontinued
items). The implementation point of contact can provide current information and
discuss implementation requirements.
|
Item Description
|
Cost
|
Required by Developer
|
|
Mother and Father Program DVD sets
|
$120 per set
|
Yes
|
|
Parent handbooks
|
$30 per set per parent
|
Yes
|
|
Group leader manuals
|
$150 each
|
Yes
|
|
30-hour initial group leader training at the Prevention Center at Arizona State University in Tempe, AZ
|
$5,000 for up to six participants, plus travel expenses
|
Yes
|
|
20 hours of supervision and continued training
|
$2,000 per participant
|
Yes
|
|
Technical assistance and ongoing limited consultation
|
Varies depending on frequency and length of consultation
|
No
|
|
Online quality assurance monitoring tool
|
$500
|
Yes
|
|
Replications
Selected citations are presented below. An asterisk indicates that the document
was reviewed for Quality of Research.
Dawson-McClure, S. R., Sandler, I. N., Wolchik, S. A., & Millsap, R. E. (2004). Risk as a moderator of the effects of prevention programs for children from divorced families: A six-year longitudinal study. Journal of Abnormal Child Psychology, 32(2), 175-190.
* Tein, J. Y., Sandler, I. N., MacKinnon, D. P., & Wolchik, S. A. (2004). How did it work? Who did it work for? Mediation in the context of a moderated prevention effect for children of divorce. Journal of Consulting and Clinical Psychology, 72(4), 617-624.
* Wolchik, S. A., Sandler, I. N., Millsap, R. E., Plummer, B. A., Greene, S. M., Anderson, E. R., et al. (2002). Six-year follow-up of preventive interventions for children of divorce: A randomized controlled trial. Journal of the American Medical Association, 288(15), 1874-1881.
* Wolchik, S., Sandler, I., Weiss, L., & Winslow, E. (2007). New Beginnings: An empirically-based program to help divorced mothers promote resilience in their children. In J. M. Briesmeister & C. E. Schaefer (Eds.), Handbook of parent training: Helping parents prevent and solve problem behaviors (3rd ed., pp. 25-62). New York: John Wiley & Sons.
* Wolchik, S. A., West, S. G., Sandler, I. N., Tein, J., Coatsworth, D., Lengua, L., et al. (2000). An experimental evaluation of theory-based mother and mother-child programs for children of divorce. Journal of Consulting and Clinical Psychology, 68(5), 843-856.
|
|
|
