Print | E-mail

 

Workshop on Evaluating the Impact of Programs Under the Violence Against Women Act

This workshop was held at the Office of Justice Programs on December 15, 2000, and was sponsored by the National Institute of Justice (NIJ), the Bureau of Justice Statistics (BJS), and the Violence Against Women Office (VAWO) of the U. S. Department of Justice (DOJ). The workshop’s purpose, as stated in discussion guidelines provided to participants, was to conduct a "brainstorming session on determining the impact of VAWA programs in terms of the safety of women and the accountability of offenders."

This report first summarizes all of the workshop’s plenary sessions. After the morning sessions, participants worked in small groups of 10-12 to focus on specific questions. The final section of the report presents details from those small group discussions.

• Welcoming Remarks
  Mary Lou Leary, Acting Assistant Attorney General, Office of Justice Programs
• Overview of VAWA Programs
  Catherine Pierce, Deputy Director, Policy Development Division—Violence Against Women Office (VAWO)
  Deborah Goelman, Deputy Director, Policy Development Division—Violence Against Women Office
• What Is Measured in Current Evaluations?
• Statement of Goals and Discussion of What We Don't Know or What Has Not Been Measured Well
• Report-out on Small Group Discussions
  
  
  • Question 1: How Do We Define the "Impact" of VAWA I?
    
  • Question 2: Are Women Safer? Are Offenders Held Accountable?
    
  • Question 3: What Current Measures of the Impact of VAWA I Can Be Used or Augmented?
    
  • Question 4: What Future Measures Should Be Collected to Measure the Impact of VAWA I and II?
• Discussion
• Closing Remarks
  Jan Chaiken, Director, Bureau of Justice Statistics
• Details from the Breakout Group Discussions
  • Question 1: How Do We Define the "Impact" of VAWA I?
    
  • Question 2: Are Women Safer? Are Offenders Held Accountable?
    
  • Question 3: What Current Measures of the Impact of VAWA I Should Be Used?
    
  • Question 4: What Future Measures Should Be Collected to Measure the Impact of VAWA I and II?
    
  • Recommendations on Federal Roles

Welcoming Remarks

Mary Lou Leary
Acting Assistant Attorney General
Office of Justice Programs

Ms. Leary said that during her many years of service as a line prosecutor, she had found it difficult to convince people to look at violence against women as anything but "family issues that did not need formal attention." Since then, the country has had a "sea change," she said, in the laws, justice system orientation, and ideas for supporting victims of domestic violence.

Ms. Leary noted that every state now has laws requiring domestic violence to be treated as a crime. Civil protection orders are much easier to obtain, and agencies are working more closely together on various aspects of the problem. The Office for Victims of Crime (OVC), for example, now offers more than half of its funding to support services for battered women and children. Awards under the STOP grant program are improving the coordination of criminal justice responses to domestic violence, and judicial oversight programs that include enhanced services are being pilot tested at three sites. Still, she said, there are many questions about how to best manage combinations of victim services and offender accountability measures, and how to connect the right services together.

More than a billion dollars have been spent, Ms. Leary said, to improve societal responses in this area, with millions spent on law enforcement arrest programs alone. Family dynamics and a broad range of social attitudes often make the process difficult. It is important to increase our understanding, she said, by conducting good evaluations. Recognizing that need, DOJ has provided money "off the top" of these grants to support evaluation efforts. Attorney General Janet Reno is extremely interested in good research on programs addressing violence against women. In addition, President Clinton this year signed the new Violence Against Women Act ("VAWA II"), with expanded measures for victims of trafficking and better tools to protect immigrant women.

Although juvenile crime has plummeted in recent years, Ms. Leary said, crimes of violence against women have not followed this pattern. According to BJS, the number of women killed by intimate partners has actually gone up, and the incidence of criminal stalking is much larger than previously thought. It is clear, she said, that we have a long way to go, but we must step back and evaluate the impact of the landmark legislation that created these programs and gain knowledge and perspective from our experiences. She thanked those working in the field for their thoughtfulness, dedication, and commitment to protecting victims and holding offenders accountable.

Back to Top

Overview of VAWA Programs

Catherine Pierce
Deputy Director, Policy Development Division
Violence Against Women Office (VAWO)

Ms. Pierce gave brief summaries of the VAWA programs begun in 1995. In that year, 56 awards totaling $26 million were made to the states and District of Columbia. In 2000, there were 440 awards totaling $220 million. On a day-to-day basis, VAWO oversees more than 1,000 active grants in addition to developing new solicitations, conducting careful peer reviews of proposals, and making recommendations to the Attorney General on new awards.

The largest program, Ms. Pierce said, is the STOP formula grant program, begun in 1994. The base amount for each grant was $500 thousand, with additional funding proportionate to population (according to a formula). Part of the program was intended to promote "coordinated community responses" to violence against women. By 1996, there were 400 grants amounting to more than $165 million. The state agencies administering the federal funds were designated by the governors.

The largest discretionary grant program under VAWA is for Grants to Encourage Arrest Policies. To be eligible for these grants, local governments must certify that laws have been enacted that encourage arrest for domestic violence. VAWO regularly requires grantee partnerships with local shelters, legal advocacy groups, and other programs. These partnerships must be formalized with memoranda of understanding prior to receiving grant funds, with the intent of strengthening cooperation.

Under the Judicial Oversight Demonstration Program, three sites were awarded up to $2 million each for 18 months (Dorchester, Massachusetts (Boston); Milwaukee County, Wisconsin; and Washtenaw, County, Michigan (Ann Arbor). Rural domestic violence enforcement grants totaling approximately $25 million currently support 138 active projects that reach between 180 and 200 counties. This program also includes funding for demonstration programs such as OJP’s "Circle Program," which involves Northern Cheyenne, Zuni, and Oglala Sioux tribes. The "Green Book" program addresses the co-occurrence of domestic violence and child maltreatment. Five new awards under the Green Book program were made to counties in New Hampshire, Colorado, Missouri, Oregon, and California. (A sixth site, San Francisco was funded directly by HHS). Another new initiative is the Civil Legal Assistance program with 151 grantees, including partnerships with attorneys providing pro bono services to victims of domestic violence. These grants have been changed under VAWA II to allow 25 percent of funding to go to programming for sexual assault survivors (previously not allowed under the old appropriation). Another group of newer grants targets sexual assault, dating violence, and stalking on 41 college and university campuses around the country (Grants to Combat Violent Crimes Against Women on Campuses).

Back to Top

Deborah Goelman
Deputy Director, Policy Development Division
Violence Against Women Office

Ms. Goelman noted that the new VAWA 2000 legislation received bipartisan support in Congress. "Full faith and credit" provisions have been strengthened to support enforcement of protection orders across jurisdictional boundaries. VAWA 2000 emphasized partnership roles for probation and parole, and state and local courts, and expanded services for battered immigrants and older victims of domestic violence and sexual assault. Grant programs also were expanded under the new legislation to focus on improving services for victims of sexual assault and dating violence.

Under the grants to combat violent crimes against women, VAWA 2000 established a 5 percent set-aside for tribal governments. In addition, the new law created a set aside of funds for state coalitions against domestic violence and sexual assault. To encourage victims to seek legal relief, jurisdictions that receive STOP Formula Grants must certify that they do not charge victims costs related to civil or criminal domestic violence cases. Courts are now eligible as subgrantees under the STOP Formula Grant Program, and statewide coordination was strengthened.

In addition, three new programs were authorized: Safe Havens for Children, a supervised visitation and exchange program for cases involving domestic violence, an elder abuse program, and a program addressing responses to victims of domestic violence with disabilities. Further, VAWA 2000 authorized the Legal Assistance for Victims Grant Program, a program designed to increase legal services for victims of domestic violence, sexual assault and stalking by training attorneys about these issues and by increasing coordination between nonprofit organizations and legal services providers.

Back to Top

What Is Measured in Current Evaluations?

This session was facilitated by Callie Rennison, Statistician, Bureau of Justice Statistics, and featured six brief presentations that reviewed ongoing VAWA evaluations.

Martha Burt, Program Director, Social Services Research Program, The Urban Institute, discussed the National Evaluation of the STOP Program. Dr. Burt said the evaluation has been examining changes based on law enforcement and prosecution data in whole states and counties within states. Four states (Wisconsin, Iowa, Connecticut, and South Carolina) were contacted to gather statewide data (using incident-based reports that go back to 1995). The incident-based reporting format can distinguish categories valuable for tracking purposes, such as whether intimate violence or child/elder abuse was involved, whether the victim was female, whether a female was arrested, and what prosecution or case resolution occurred. Changes in arrest patterns are being compared across communities and with baseline statistics. The study also collected annual information on communities receiving victim assistance funding from other DOJ sources (Byrne, VOCA, etc.). Only two states had ideal data for comparison, including incident and arrest data, statewide five-year data, and county level data.

Data are being used to look at year-to-year changes, although Wisconsin data indicates that incident reports are prepared only when arrest is imminent, and Connecticut has only arrest data. The Institute for Law and Justice, under a subcontract from the Urban Institute has interviewed nearly 100 each of law enforcement and prosecution agencies that have received STOP grant funding. Few agencies were able to supply complete data from the period prior to 1996 through the present. Population information from the Census is also being used in the analysis.

The evaluation is also tracking which local agencies receive STOP, VOCA, and other funding to document the different kinds of activities supported, observe the communities to which the funds have been targeted, and to determine whether changes and overall impact show in the records. In about 600 jurisdictions, victim service organizations have received portions of the awarded funds. The evaluation is using a Sabre database to track over 6,000 awards that encompass about 30,000 projects. To enhance impact data from the victim services projects on responses to victims and perceptions of domestic violence victims ("testimony data"), a survey has been conducted in 40 communities using random-digit-dialing. The survey requests information on victimization, services that had been used, and opinions about the experience. The study will also look at elements of the legal system available to victims.

Brenda Uekert, Senior Associate, Institute for Law and Justice, spoke about the impact evaluation of the Arrest Policies Program and the law enforcement and prosecution grants under STOP. Site visits have been conducted and quantitative and qualitative strategies have been used to build information on the impact of grant programs. Dr. Uekert said there were great weaknesses in the available data; for example, often pre-grant data on domestic violence were not separately coded. To compensate, researchers built a case tracking system to follow pre- and post-grant handling from arrest through disposition and used structured focus groups. Dr. Uekert said some disturbing trends have been observed in connection with arrests of women and racial/ethnic differences. Minority victim groups, including non-English speaking Hispanic and Native American women, were included in the structured interviews.

Eileen Luna, Professor of American Indian Studies, University of Arizona, discussed the STOP grants for reducing violence against Indian women. By the end of the evaluation, 131 projects under STOP’s Violence Against Indian Women (VAIW programs) will have been evaluated. Not much baseline information was available from surrounding law enforcement agencies or the Bureau of Indian Affairs. It was particularly difficult to get numbers from the tribes and Sheriff’s offices, Dr. Luna said. Too little detail had been recorded on domestic violence or assaults against women (most incidents were listed as "simple assaults"). Turnover of tribal leaders/administrators also affected the study. Evaluations have looked at many aspects of coordination of law enforcement and services (including different agencies in surrounding areas). Those projects which took a "coordination approach" generally had more noticeable effects. Researchers also checked what training of law enforcement was occurring, and whether "systemic racism" along borders or in communities was noted. The study also tracked increases in sentences for offenders committing domestic violence crimes.

Mary Ann Dutton, Professor, Department of Psychiatry, Georgetown University School of Medicine, described the evaluation of ten sites working on different priority areas, all supported under VAWA through the Rural Domestic Violence and Child Victimization Enforcement Grant Program. Case studies are using quantitative and qualitative data, including pre- and post- comparisons with available data. Newly funded programs are being assessed using a "single-case, multiple baseline" design. Interviews, focus groups, and examination of aggregate data are included in the work. The "post-hoc" nature of the research has created challenges, as has the "weaving" of funding by the sites (using all possible sources to stay afloat). Increased community ownership of programs has been observed.

Marcia Chaiken, Director of Research, LINC, reported on a program of case studies in four states and four counties, explaining that this is not an evaluation in the same sense as the other projects. The case studies looked at the effects of VAWA in those locations, with the intent of supporting "judgment calls" in those areas, relative to post-VAWA program development and statistics. The idea for the case studies came from Michael Maltz, University of Illinois, Chicago; they were attempting to see what boost had been given to local programs through the VAWA initiatives. Different types of data were involved, including two longitudinal data sets and current site descriptions. The research, using data from Arizona, Oregon, Massachusetts, and Maryland, covered a wide spectrum of local government, victim services, and criminal justice initiatives to gather perspectives and document approaches used to address domestic violence. Many efforts, using a variety of federal funding sources, were launched to bring about collaboration. The study was supplemented by a review of the work of agency administrators and other researchers on arrests and homicide rates (e.g., Joel Garner on arrests, Eric Wish on hotline calls). Dr. Chaiken emphasized the need to anchor research in theory and to find out which groups already have large collections of data covering a long time period.

Anita St. Onge, Research Associate, Edmund S. Muskie School of Public Service, University of Southern Maine, presented highlights of an analysis of the Snapshots Project. Visits have been made to 24 states to gather descriptive information and statistics on the administration of STOP funding and the activities of state coalitions. Cross-state data elements were not often available, she said, but there were many expressions of enthusiasm for the differences made by these programs. Videos were prepared of selected programs, and profiles have been prepared for the first 18 states, with findings from the data and descriptions of some of the remaining needs.

Back to Top

Statement of Goals and Discussion of What We Don't Know or What Has Not Been Measured Well

This discussion was facilitated by Larry Greenfield, Bureau Director, National Statistical Programs, Bureau of Justice Statistics.

Larry said the National Crime Victimization Survey (NCVS), which interviews women above the age of 12 about violent victimization experiences, was redesigned in 1992 and included nearly double the number of individuals (about 11 million). In 1995, the questionnaire added new items concerning victims who lived on campuses. BJS addresses additional questions about victimization experiences to offender populations in jails and to probation agencies.

In the past six years, BJS has asked for national collaboration on case processing statistics, but has not received much support in this. BJS also studies contingencies of crime: how the crime, offender, and environment come together. For over three decades now, BJS has also worked to improve criminal history records around the United States. Much of this thrust has been aimed at improving capacities within the states to conduct instant background checks before guns are purchased, but there has also been greater vigilance toward improving data on offenders sentenced to the community.

BJS has also been part of increased efforts against cyber crime, including initiatives for stalking and identity theft. Adoption of the National Incident Based Reporting System (NIBRS) is critical for accurate crime statistics, Dr. Greenfield said; it is better than the Uniform Crime Reports (UCR). About 18,000 police departments have adopted the NIBRS system.

Jan Chaiken, Director, Bureau of Justice Statistics, emphasized the circumstances that led to the current workshop. Cooperating federal agencies (VAWO, COPS, the Byrne formula grant programs, and others) have given a great deal of funding and assistance to strengthen organizations. OJP perceives that an exceptional amount of effort has gone into violence against women initiatives, Dr. Chaiken said, but national statistics are not showing an impact commensurate with this effort. For example, the drop in victimization rates for males has been much steeper than for women. Overall, interpersonal violence does not appear to have changed significantly. The rates for rape and sexual assault have fluctuated down and up; and with respect to homicides by intimates, murders of women may be going up again. In terms of homicide statistics, Dr. Chaiken said, the benefits of the programs following this legislation (VAWA) have gone to men.

Data that could complete this picture, from the localities around the states, are still lacking. Coding of offenses is often not clear enough to determine whether the incident was domestic violence or not. Information in the public domain concerning VAWA’s impact, Dr. Chaiken said, is not convincing. If any evaluators at this workshop have doubts about their own data or methods, he said, they need to let that be known. It is essential to clarify results that can inform program development over the next few years.

Bernard Auchter, Acting Director of NIJ’s Violence and Victimization Division, began the discussion by asking the group "what we know (or can know)" about the impact of VAWA, both in the long term and in the short term.

David Adams, Program Director, EMERGE, spoke of examining the increased use of systems, such as calls to police and domestic violence cases in the courts. Using those indicators, the situation initially looks worse, he said, especially in the courts, which may need to "aim" some responses better. For example, in some areas, family courts have been accused of "killing the victim" due to the punishing effects of the measures taken (e.g., custody decisions) and minimal or very short-term accountability measures for the offender. Dr. Jan Chaiken added that systems have not been set up to track or investigate negative consequences to victims. Dr. Marcia Chaiken noted that some of the judicial backlash could be traced to a perceived reduction in judicial discretion. This follows an early "counter-movement," she said, which has been based in different social or family theories.

Dr. Burt said that, since the 1980s, a vast "reservoir" of unreported domestic violence has come to attention. The hope is that people will use the system more, but researchers have to look at how individuals are receiving help or redress from resources outside of the criminal justice or legal system. Justice interventions have necessarily gone to assist known victims and not to funding of prevention, so increased use of the system just shows the effort is working, she said. There are about 3,000 projects under STOP, some of them only one year old. To get real change, she added, "men will have to be leaning on men as well."

David Ford, Chair of the Department of Sociology, Indiana University-Purdue University Indianapolis, suggested that an increase in arrests may not be the number society wants to see as a result of VAWA; moreover, data on numbers of arrests do not indicate reduced violence. Funding from the legislation has been stacked in favor of criminal justice interventions, he said, and these may not be most effective against this crime. More practically, evaluations have to measure that the intervention "does what it says it will do." This may not necessarily protect the women. Answers to the question, "Are women safer?" differ from those to the question, "Are offenders held accountable?"

Beth Richie, Associate Professor, Department of Criminal Justice, University of Illinois at Chicago, said it is hard to disentangle different streams of funding support related to VAWA, and it is also difficult to isolate the effects of VAWA from the effects of other legislation. In addition, various policies—for example, immigration rules and welfare reform—are contributing to changes in the overall community context. These are having significant effects on the behavior of individuals in communities.

Dr. Luna noted that women in Indian country had not been using state-supported shelters at all; therefore, any counting will show an increase in Indian country, she said. Margaret Chretien, Program Specialist, New York State Division of Criminal Justice Services, asked for more detailed consideration of what "accountable" means. We have not done race/class analysis, she said, and have not looked at alternative models being used in communities (only criminal justice models). Dr. Dutton suggested that studies try to capture community context and include statistics that show the impact of welfare change and racial/ethnic distributions. Adele Harrell, Director, Justice Policy Center, The Urban Institute, said the studies undertaken for VAWA II could address time series trends and use coding that will capture trends in contextual factors as well. Although there were little or no data in some areas that could have benefited VAWA I evaluations, now there are more data.

Back to Top

Report-out on Small Group Discussions

Workshop participants met in four small groups for three hours in the afternoon. All groups were asked to address the same four questions related to measuring the impact of VAWA I and VAWA II. In the next plenary session, the small group facilitators reported highlights of these discussions. This report-out session was facilitated by Angela Moore Parmley, Program Manager, Violence and Victimization Division, NIJ. Dr. Moore Parmley encouraged reporters to add to, but avoid duplicating, the ideas noted by other groups.

Back to Top

Question 1: How Do We Define the "Impact" of VAWA I?

Dr. Laura Dugan said Group 4 focused more on pre- and post- VAWA awareness of violence against women issues than on short- and long-term impact. She added that while "victim safety" is frequently cited as an individual-level impact measure, the group recommends the broader term, "victim well-being." It is important to remember, she said, that there are three different movements with a special interest in VAWA—the women’s movement, the victim rights movement, and the system effectiveness movement—and each has a different perspective on what outcomes are desirable.

Dr. Dugan said the group generally agreed that adverse impacts and differential impacts (by race, ethnicity, social class, etc.) need to be considered. She noted that VAWA II attempts to address some of the differential impacts by targeting under-served populations (elderly persons, immigrants, people in same-sex relationships, etc.). There has also been differential treatment of offenders by race, she said.

Such information as numbers of conferences sponsored by VAWA and numbers of participants can provide a "macro" look at activities, Dr. Dugan said, but the group also suggests looking at a sample of conferences to see what relationships were formed and what initiatives were launched as a result. Finally, she emphasized that both positive and negative outcome measures are needed. One positive outcome, she added, was that grants under VAWA have helped communities leverage other sources of funds.

Dr. St. Onge, reporting for Group 2, said the group was not sure it was possible to talk about long- and short-term impacts. Initially, concerns were expressed about the need to "be real" about what can be measured, she said, but the group felt it was important to talk about lessons learned.

Dr. St. Onge said the group generally felt the questions about victim safety and offender accountability were too narrow and needed to be re-framed. Currently, they ask only about the criminal justice response, excluding other areas that should be considered in evaluating VAWA. Violence against women is complex and needs complex solutions, she said; one size does not fit all, but VAWA is primarily funding one type of response. She summarized Group 2’s key points as follows:

• Punitive approaches must not be the only response.
• Evaluations need to be more nuanced.
• There is a need to focus more on community-level issues and interventions.
• VAWA has changed the way people are relating to one another.

Dr. Ronet Bachman, reporting for Group 3, said the group could not define impact without acknowledging all of the assumptions underlying VAWA—for example, that the criminal justice system approach is the best approach. Issues of race and ethnicity must be acknowledged, she said. We must also define impact not in terms of broad goals related to violence against women, but in terms of the efforts that were funded. Finally, she said, more informative are the things we can’t use; stories and local context are needed.

Back to Top

Question 2: Are Women Safer? Are Offenders Held Accountable?

Ms. Menard said Group 1 spoke broadly to these questions. For example, what is meant by women’s safety? In part, this means well-being, but safety also has different meanings with respect to sexual violence. Moreover, different people have different definitions of safety; for example, immigrants might define it differently than others. Also, there are different assumptions about what can be expected of the criminal justice system. We need to be careful about such assumptions, she said.

Ms. Menard said the group also discussed what the impact would be of shifting to victim-centered measures. These could be very different in dual arrest cases, for example. The group agreed that how the shift from quantitative to qualitative measures is made is important. We have to be concerned with victims’ views of safety and well-being, she said, but victims’ views and qualitative research do not seem as credible as quantitative data. The shift would require educating policy makers. The group also discussed system accountability and offender accountability, she said.

Dr. St. Onge, reporting for Group 2, noted the following issues discussed by the group:

• Are we holding offenders accountable and making women less safe?
• We need to look at context (e.g., welfare, immigration).
• Measures for quality of life are needed (not just PTSD).
• Protective orders may not be equal to greater safety.
• Judges in family law cases have been a negative influence for some of the programs developed.
• We need to be careful about using victimization statistics as a proxy for safety.

Dr. Bachman summarized the Group 3 discussion by noting the following points:

• Multiple measures must be used.
• "Safety" must be expanded to include well-being.
• Evaluations must illuminate barriers for racial and ethnic groups when asking, "has there been a reduction?"
• We need to measure victim safety over time.

Dr. Burt added that treating victims with respect is also important. She noted that Group 3 discussed offender accountability at length. We want to see offenders undergo a "total transformation," she said, but this is not a realistic expectation. However, we can look at changes in offenders’ "norm group" and changes in specific behavior.

Dr. Dugan noted that Group 4 discussed whether women’s voices are being heard and whether women’s options are being expanded. We can look at representative groups of women regarding this, she said, as well as at victim satisfaction. She added that the WESTAT/VOCA survey could be valuable in evaluating these outcomes. Group 4 encouraged evaluators to find ways to measure well-being using existing data. It is now an opportune time, she said, to get baseline information on court decisionmaking, using the court watch system to assess victim input; and on probation and parole with respect to both safety and accountability.

Back to Top

Question 3: What Current Measures of the Impact of VAWA I Can Be Used or Augmented?

Ms. Menard, reporting for Group 1, offered the following ideas:

• Take advantage of the geo codes used with the NCVS to target communities with and without VAWA funding; and more generally consider how the NCVS survey might aid in evaluating VAWA, paying attention to those sub-populations that the NCVS does not capture.
• With respect to STOP, require grantees to provide certain data. As this may not be fair to those who do not already collect these data, consider implementing a two-stage grant process.
• Implement a clearinghouse of instruments that researchers can use, with the aim of achieving greater consistency in the data collected.
• Consider using medical data (e.g., from emergency departments, large HMOs). Look to drug-related studies for ideas.
• School and housing data could be sources with respect to Native Americans.
• Use existing measures of economic well-being (however, it could be difficult tying these in to VAWA).
• Remember that victims’ sense of safety is not static. Look at this over time.

Dr. Bachman said Group 3 discussed the need to use control groups where possible, and that participants also noted problems with existing data. For example, surveys asking only about spouses provide an incomplete picture; and it is hard to determine who was arrested from available data on dual arrests. Other items discussed included these:

• There are very few studies about the causes of the violence; more are needed.
• There is a need to hold not only offenders but also the system accountable.
• It may be possible to develop a national gauge for victim safety and well-being.

Dr. Dugan, reporting for Group 4, said the basics needed for an evaluation are lacking. There is no database of programs funded in different communities and regions. The group recommended that this be compiled as soon as possible. Other observations and recommendations included these:

• Hotline data are a potentially valuable but under-used source.
• Add-ons to existing state surveys could be used. One of these is the BRFSS survey conducted by the Centers for Disease Control (CDC).
• The use of XML language holds potential for evaluators. It is the wave of the future and is being used in the PHAT study in Pennsylvania.
• There should be a core set of questions for all VAWA grantees to answer.
• Grantees should be required to have a logic model. This requirement should be kept simple, but grantees should have some sort of hypothesis before receiving funding.
• Expand the smaller, qualitative evaluations to get more contextual information.
• Look at NIBRS more closely to see how it could help.

Dr. Dugan noted that the NCVS does not ask about strangulation or whether the offender lived on the premises, and that the CDC is working on domestic and sexual violence modules.

Back to Top

Question 4: What Future Measures Should Be Collected to Measure the Impact of VAWA I and II?

Reporting for Group 1, Ms. Menard offered the following observations and recommendations:

• The concept of persistence is getting lost when victims are asked only about past six months and past year abuse.
• Broaden measures of justice outcomes to include social justice outcomes. This is needed but raises questions about how these attach to VAWA. HHS/DOJ research collaborations have potential to gain a clearer sense of this.
• Context and non-system factors (welfare, coordinated community response) are important to assess.
• Pay attention to demands on advocates’ available time when asking them to gather data.

Barbara Hart suggested funding not only court watch programs but also "police watch" and "victim service watch" and using them to gather data. She acknowledged that researchers might consider this approach to be too unscientific. Dr. Burt said there was a need to better understand what particular behaviors might mean in the context of various minority communities. She also suggested using focus groups to come up with measures of victim well-being and encouraged other flexible methods for developing concrete measures.

Dr. Dugan, reporting for Group 4, noted the following areas about which little is known:

• Low-level violence
• False imprisonment
• Psychological safety
• Gun confiscation.

She also said there was a need to assess what states already have in place that could inform an evaluation on a national level.

Other workshop participants made the following recommendations:

• Provide grantees with simple measures and consider giving them the software needed to collect data. The process and information should be useful for the grantees.
• Give states incentives to collect the data.
• Do pre- and post-tests to determine what participants are learning from the training provided.

Back to Top

Discussion

Bonnie Campbell, Director, Violence Against Women Office, asked whether we can measure, or whether we have already measured, what doesn’t happen and why—for example, a batterer no longer hits because his attitude has changed. An audience member said this can be determined only with comparison groups. Another participant said this has not been measured. She also said time series analyses have not been possible because of bad historical data, and that no one wants to be a comparison community.

Dr. Burt noted that with respect to STOP, the usefulness of coding communities (e.g., law enforcement funding, other funding, no STOP funding) is limited because the communities were different to begin with. We are trying, she said, but either we are never going to get comparison groups, or we are going to have to go about this differently. Currently, the money is not being given out in a way that makes random experiments possible, she said. Dr. Dutton commented that the evaluation could be built in at the beginning of the program. Dr. Burt said any requirement to do this would have to come from the national level, but that the effort cannot be national. It will not be possible to do an "everything, catch all" evaluation for VAWA II, she said; the states will never do it. The reality is that we cannot demand that the states send data or do evaluations.

Ms. Campbell noted that VAWA was passed because it could be tied to the justice system, "but it still doesn’t feel complete to only measure justice system impacts." Comments by various participants included these:

• Many cases are ignored by focusing only on justice system statistics like numbers of arrests.
• National data on victims’ use of various services could be valuable.
• Researchers’ use of the Conflict Tactics Scale (CTS) without providing context results in a "nightmare" for those who deal with policymakers and inevitably leads to the question, "Don’t women batter as much as men?"
• The use of focus groups is helpful, but it is hard to get policymakers to accept qualitative data.
• Despite the political ramifications, "it is foolish to say females don’t use violence." Female use of violence also needs to be addressed, particularly with respect to adolescents.

Ms. Campbell said that while violence by women is a reality, "it’s the exaggeration that’s so frustrating."

Back to Top

Closing Remarks

Jan Chaiken
Director, Bureau of Justice Statistics

Dr. Chaiken said Attorney General Reno cares deeply about violence against women, has been tracking evaluation efforts closely, and has been cooperating with DHHS on these issues. He noted that BJS/OJP have been coordinating with CDC, and that the Attorney General has asked that questions about intimate partner violence and sexual assault be included in the BRFSS. This is currently under consideration, he said, although some states have expressed concerns about the effects this might have on other BRFSS questions.

Dr. Chaiken noted that because of time limitations, the report-out session could not fully cover all of the breakout groups’ detailed recommendations and discussions. Referring to the handouts offered to guide those discussions, he said the answers to some questions (e.g., "are women safer?") could be negative, but that it is important to remind evaluators to also look for and discuss positive outcomes if VAWA is to continue.

With respect to the database of funded programs being developed, Dr. Chaiken said the value of linking it to other databases such as NCVS is clear. He said he would spend some time examining the specific types of data needed, costs, and other details.

Of the recommendations offered today, Dr. Chaiken said, those involving relatively "minor fixes" to NCVS are the ones most likely to see action in the near future. For example, asking a victim whether the offender lives in the home might be possible, although it does raise human subjects protection issues. With NIBRS, the arrest record is attached to the reported incident, and this can help determine details about dual arrests.

Dr. Chaiken said the breakout groups clearly agreed that "one size does not fit all" with respect to evaluations, and that not only victim safety but victim well-being should be considered. Well-being is more difficult than safety to operationalize, but "we hear you loud and clear," he said, and understand that measures of well-being for sexual assaults are different than those for other crimes.

Back to Top

Details from the Breakout Group Discussions

This section of the report provides additional detail on the discussions that took place in the workshop’s four breakout groups. The information is organized around the four key questions used by facilitators to guide these small group discussions.

Back to Top

Question 1: How Do We Define the "Impact" of VAWA I?

Various participants offered ideas for reframing the questions about VAWA I’s impact; discussed the Act’s inherent limitations (particularly its focus on criminal justice system responses); advocated for involving battered women in defining success; pointed to differences in state and local goals and pre-VAWA resources; noted political issues of concern; suggested areas where VAWA I has had a positive impact; and emphasized that the Act’s differential and negative impacts must be explored as well.

Reframe the Questions

Several participants called for greater clarity about what VAWA I can be expected to accomplish. Comments to that effect included these:

• Reframe the question as: How do we define the impact of VAWA I on the criminal justice system response to violence against women (that is, the particular set of programs and approaches supported by VAWA I)?
• The challenge is that the STOP legislation implies certain things will make a difference when this is not necessarily so. We need to be clearer about what we expect VAWA to do. What are the reasonable expectations? And will they enable us to measure whether violence against women is reduced?
• Define "impact" in terms of what the Act funded (not necessarily the goal of VAWA [for] reduction of violence against women); goals are not matching what has been funded (activities).
• Evaluation questions (similar to those for any longitudinal initiative) need definitions that can be measurable. VAWA initiatives have "resisted definition."
• The focus has been on women, but the violence is really a "system" involving men, women, and children.
• The term "victim safety" is not adequate for describing the outcomes desired for victims. We should be measuring "victim well-being," which includes safety but also other measures.

One group discussed the idea that there are "three social movements feeding in [women’s movement, victim rights movement, system effectiveness movement], and all three should be considered." It was suggested that all three movements are concerned with victim safety, but that their long-term goals differ. For example, for the women’s movement, long-term goals include "economic empowerment so women can make their own decisions; autonomy, or what some call ‘agency’"; victim movement goals emphasize "offender accountability and victim inclusion in the process"; and the system effectiveness movement "has very different goals and sees the agencies as a system, although they’re not."

Consider Limitations of Criminal Justice Responses

Many participants said that while the Act primarily supports criminal justice system responses, those responses may not make women safer. Moreover, use of criminal justice measures alone fails to capture information about victims who use non-system alternatives. Related comments included these:

• Looking at only the criminal justice system limits studies to a subsection of the community. What about victims who are "not in the system?"
• Funding was given to those agencies friendly to a criminal justice system response, and we are therefore missing a lot. The criminal justice system response has limits (e.g., dual arrests).
• The focus on the criminal justice system measures only behaviors, not the attitudes/treatment of women using the system. For example, how does training police affect how they do their work?
• [Consider victims’] use of criminal justice system elements vs. other common services (success may show in increased use of these other services).

Ask Battered Women How to Define Success

Several participants said battered women should have a voice in determining what constitutes success under VAWA. For example:

• How do we define success? Ask what a battered woman would want—arrests, numbers of offenders in counseling, or what?
• According to the National Clearinghouse, resources was the main thing 14 women’s organizations said battered women need, as opposed to criminal justice responses, etc.
• There are assumptions behind what the Act is funding; what has been allowed (paid for) does not necessarily match what victims would call aids to success.
• Outcomes must include social justice for women, economic independence, social equality.
• The criminal justice approach is only one piece of what women want.

Consider State and Local Goals and Pre-VAWA Status

Several participants expressed the view that local definitions of success were needed, or pointed out that states differed greatly in terms of what was already in place when VAWA was enacted.

• Definitions of success must be local.
• Communities set their standards differently ("zero tolerance" v. other standards); family courts require mediation.
• The states are so different, I’m not sure we can get [consistent data across the board]. Some states . . . were doing things before VAWA, whereas in [others], VAWA did chip away and make a difference.
• In the states, VAWA was most effective in states with active advocates.

Recognize Political and Funding Issues

Some participants commented on political and funding processes that they felt contributed to difficulties in evaluating VAWA’s impact. Comments to this effect were:

• Shotgun approach to funding what is out there will not necessarily reduce violence against women.
• Who defines "success?" Federal money is going to the same people all the time; Congress is the "mega-context."
• Legislators are primarily interested in their own states.
• VAWA [is a] consequence, not a cause of the change.
• What is meant by short-term, long-term? How can anything work in five years?
• [Efforts associated with victim rights and system effectiveness movements] will get funding, but resources for women’s economic viability are going to go away, [and] as women are more captive, things will get worse.

Emphasize Positive Impact of VAWA I in Selected Areas

Although many difficulties were noted, various participants in each group did suggest areas in which VAWA I might be shown to have had a positive impact. Comments regarding VAWA’s impact on public awareness, information diffusion and conferencing effects, system-level improvements, and strengthened coalitions and new opportunities are listed below.

• Public Awareness
• As a first hurdle, at least the country [will] know what violence against women is all about.
• VAWA contributed to changes in attitude—[helped lay the] groundwork.
• Just the announcement of money available had a system level impact [in terms of awareness of the issues]. Look at before and after the announcement, before and after funding; compare those with funding to those with no funding.

  Information Diffusion, Conferencing Effects

• Another measurable outcome: diffusion of information, of program approaches (this is both a community and a system effect).
• The conferencing effect is easily measurable. We’d have to [convince DOJ] that the "conference effect" [like training impact] is a real outcome.

  System-Level Improvements

• "Consumer friendliness" for victims.
• Making the criminal justice system consumer friendly is important. Count it!
• Valuable uses of funding have gone to setting up systems/service delivery that will work better, and toward helping known victims to get through the system better. This may have a secondary prevention effect also (but these are "standards" that the Act was not aimed at).
• Real cultural change in police departments.
• Findings that the quality of responses to victims has changed—for example, police doing more drive-bys, special units.
• A small measure might be more consistent documentation, for example, the "Kentucky experiment" in seven states in the region, which developed uniform protective order forms.

  Strengthened Coalitions, New Opportunities

• In states without a strong domestic violence/sexual assault coalition, there is now a stronger coalition (primarily sexual assault coalitions).
• [In reference to STOP purpose areas], at community-based levels, initiatives never before undertaken [were launched] with an infusion of money.
• The importance to battered women of the chain of helping people in the community.
• We need to say long term how the lives of women who have been raped have been changed, and what it is like in states without coalitions.
• One measure is jurisdictions’ leveraging of VAWA money to get more money from other sources.

In addition, one participant noted that the "BJS data (p. 3) showing a decrease by 20 percent in intimate partner violence against women is reason for some optimism," and another said the "rate of reporting to police has increased nationally, [although] looking at local projects, the rate of reporting hasn’t changed."

Measure Differential and Negative Impacts

Some participants said the literature now is showing problems with various criminal justice system approaches, or they said some women may be less safe because of criminal justice responses. Many also emphasized a need to "consider the context" (e.g., populations not served by or affected negatively by VAWA programs). Related comments included these:

• Racial and Ethnic Minority Groups; Immigrants
• We have to look at race, class, and region. Is our legal advocacy work making things worse for some African American women? Unmarried women? Women in certain age groups? In certain ethnic groups?
• Over-incarceration of minorities makes them not want to report.
• Differential impact on minority and lower income community—e.g., who qualifies for diversion, different types of sentencing. Emphasizes need for context in statistics.
• [Have to consider the] discriminatory effects of criminal justice solutions.
• When looking at mandated programs (i.e., arrest programs), research must take into account the ethnic groups involved and the context of the program.
• In Indian country, there is often a need to refer crimes federally (resulting in worse penalties). [This has a] chilling effect on women’s willingness to move forward. The history of racism/treatment of Indian men in the non-tribal justice system is part of the context.
• One of the negative impacts is women getting deported.

  Dual Arrests

• Women acting in self defense are being arrested for domestic violence.
• High numbers of women arrested, dual arrest; these women won’t use the system again (women sent to anger management, "obedience school").

  Child Custody

• Child custody issues; if the woman "gets the man arrested," are her kids going to be taken away from her?
• Look at child custody issues. Outcomes need to be focused on the most vulnerable and the most complex cases.
• It used to be that the victim was considered blameless, but now the focus is on helping the kids . . . this is fueling a backlash and having an adverse impact on women.

  Offenders

• What has VAWA done for women who are offenders? What is our notion of victim services [in that context]?
• 80-90 percent of women in prison are battered. This is a huge class, and it appears that VAWA has had no impact on them.
• How do we capture [the effects on] women who don’t get responses because of their history (e.g., convicted, substance abusers)? Use qualitative data.
• [VAWA] may not be reaching many women who are "on the margins."

  Timing of Interventions and Safety Risks

• Particular historical moments—for example, how men get arrested and for what (increased criminalization for "everything"). Timing.
• [Consider the] risks women are placed in when they start to use a criminal justice system response.

Back to Top

Question 2: Are Women Safer? Are Offenders Held Accountable?

Participants raised research questions about measuring safety, including the need to measure safety over time, take into account low-level violence, and consider the safety of women who do not use the criminal justice system. They also discussed other measures of well-being. In addition, participants raised research questions regarding interventions with offenders, with some commenting on batterer treatment programs and some recommending further exploration of restorative justice and other non-traditional approaches to holding offenders accountable. These themes are listed below, along with questions and comments related to each.

Measuring Safety: Research Questions and Issues

• The question is, "Are women in less immediate danger?" not, "Are women safer?"
• What is the relationship between issuance of a protective order and risk? Lethality? What is the role of safety planning?
• Police decisions about whom to arrest and ticket do not protect women.
• There must be subjective appraisals of danger, collected from both parties at an incident.
• We need to sample more than just the women who are easy to get.

• Measure Safety Over Time; Include Low-level Violence, Coercion
• [Consider] prevalence vs. incidence (do not want to miss the impact and persistence of violence against women).
• Is leaving "protective" (eventually) despite the year(s) of risk?
• A perception that services and safety are static is not true; [safety] depends on where [interventions occur] on a timeline.
• Even when the physical violence has stopped, the control issues or coercion may still have serious effects.
• To measure control and coercion, research needs to acknowledge more effects than simple measurements of incidents of violence. The "Davies model" of risks takes into account loss of custody and other issues.
• There are different long-term recovery rates for different violent events (i.e., sexual assault can be long term) and "feelings of danger" in the victim are much greater in domestic violence cases where there is ongoing occurrence (of threats, coercion etc.).
• We have to get the in-depth story—not just, some say yes, they are safer and some say no.

  Consider Safety of Victims Who Do Not Use Traditional Systems, Services

• [VAWA helped] put advocates in places [where some] victims tend to go, but who isn’t using shelters? Who isn’t calling police? Who is really seeing victims?
• Are there other support systems for [some of] these victims that work better? And is that why they don’t use the system?
• Do we feel we have enough data about why certain victims don’t use the system?
• Does increased knowledge of criminal justice system personnel make women safer, and are personnel rewarded for their response. Again, we need true coordinated community response to include more than the criminal justice system and advocacy community—other parts of the community need to come along.
• We should study the kinds of services already existing and how they are being used by women victims.
• If the goal is "serve the victimized women," then the research could measure things like protective orders received, counselors visited (i.e., within specific communities). [But] this does not address whether the funding is "reaching the right thing."

  Other Measures of Well-Being

• Increased well-being may relate more to "being heard," "respected," or given "empowering resources."
• To get a more complete picture of change, perceptions of change in well-being should be measured, rather than just safety. It would need multiple time frames and longitudinal study. How can results be attributed, practically, to interventions under VAWA?

  Knowledge and Use of Options; Agency, Autonomy

• Has women’s capacity/ability to make decisions independently changed (shall I stay or go?)—we call this "agency," one of three measures we use, along with safety and restoration (of health, etc.).
• Do women know their options? Do they exercise their options? Are the options useful?
• What do women know about their system alternatives?—for example, use of the prosecutor’s victim advocate versus other victim advocates. Are women making decisions about what happens at the system level?
• Do others (for example, doctors) know the options?
• This would be hard to measure, but over the long term, has she had a paradigm shift? Does she apply what she has learned in other settings?
• You could ask, what support systems have you created for yourself? Do you have them now? Measure whether women can predict and plan for avoidance of the batterer—that is, identify the risk, make an assessment of the risk, figure out strategies, etc.
• We have to have more than one measure of success for restoration, agency, safety planning, etc., given womens’ various cultures: For example,
• Are services more accessible—to whom?
• Are they the appropriate services?
• Do victims know what’s going to happen next?
• Do [the next steps?] give victims a better sense of safety?

  Satisfaction with Services

• Women are often satisfied if they are treated with respect.
• We have found high levels of victim satisfaction even when there is no increase in safety. Women may want different things from the system.
• Regarding use of services, Gondolf found that victims usually picked just one service. He was concerned about this, but one stop shopping (vs. use of many services) can have advantages.
• Although we have many surveys that address accessibility, same-sex relationships have not been counted in the past.

  Other Measures of Well-Being

• Consider whether women are better off in terms of being a part of their community; collective efficacy.
• How do we measure [well-being] at the individual level, in terms of psychological, social, and economic changes?
• We should be careful about not holding up battered women who leave to higher standards than we do the average woman who leaves an intimate partner but has not been a victim of domestic violence.

Measuring Offender Accountability: Research Questions and Issues

Many participants cautioned against thinking of offender accountability solely in terms of criminal justice responses (arrest, incarceration). For example: "too much emphasis on the criminal justice system"; "we need different concepts for accountability (not just ‘punishment’ as in present VAWA)"; "we need system changes, not just counts of how many are going to jail"; and "think beyond the penal code to the risk for women." Along these lines, many participants recommended research questions and issues to consider.

With respect to the impact of VAWA I on offender behavior, a few participants noted that "specialized probation officers may help the system hold batterers accountable and reduce recidivism," and several mentioned the importance of "the swiftness of response to batterers." Only a few participants commented on batterer treatment. However, many suggestions were offered regarding interventions that involved community-based groups, the faith community, and other non-system resources.

Offender Accountability: Research Questions and Issues

• Defining Accountability
• What is meant by accountable? Accountable to whom? Expand the definition; we need different models for different communities.
• Ask the women how to measure offender accountability.

  Criminal Justice System Interventions

• [There is a need to] talk about who needs to be locked up . . . consider developing batterer typologies, conducting actuarial research.
• How do measures like reduced recidivism and protective order enforcement tie in to impact on victims?
• How does offender accountability relate to victim safety? Traditional measures of recidivism aren’t working.
• Is accountability going across the civil and criminal courts?
• Are misdemeanor offenders being held more accountable?
• What happens to the man in the justice system (length of probation, etc.)? How does that relate to the relationship (ending or resuming), employment/economic support situation, increased anger/anger counseling, and context for the woman and children?
• How quickly are offenders released?
• If you scrutinize judges a lot, is there any effect?
• Is there any reason to believe domestic violence courts will work?
• [Consider the negative] impact of offender interventions on women’s well-being and kids (e.g., custody cases, economic losses). What are the unintended consequences of over-investing in criminal justice system [responses]?

  Broader View of Justice

• Need a broader view of justice (not just criminal justice but equality)
  • Economic autonomy; race issues; tribal sovereignty; more social justice for survivors; and for perpetrators?
  • Respect for victims
  • Fair treatment of all perpetrators in the system.
• One of the important things we learned is that we cannot separate community and criminal justice system. We have learned that one size does not fit all for communities.

  Batterer Treatment Programs

• Batterer interventions are weak, no real accountability for offenders.
• The nature of batterer intervention programs has changed with court referrals. The term "battering" has become problematic.
• Need better court monitoring for batterer’s programs. The men who do come and who do complete the program tend not to come back to jail. There should be some means to take into account men who do not get "into the system."
• Who gets notified if perpetrator doesn’t attend batterer intervention program?
• Eliminate need for CJS; [batterer interventions/treatment should include] stages of change; change in reference group; changes in attitude; developing empathy.
• There is still a big story that hasn’t been told—take a big view of factors contributing to violence.

  Other Interventions with Offenders

• What other ways can we hold offenders accountable? Are victims safer using the criminal justice system or not?
• Restorative justice should be integrated into violence against women interventions.
• [Consider] the concept of restorative justice (non-jail alternatives). More players are involved now (e.g., medical, victim services, law enforcement).
• Family courts require mediation.
• Family violence problems point to a need to "think in ways other than adversarial" and to change community norms.
• In the last decade or so the face of community has really changed. The society is much more diverse. For some norms, it is necessary to enlist the cooperation of religious leaders (such as priests in Latin American communities). Only this would start to create change.
• Other communities/systems (community, faith-based, especially other men) [should become more] involved in holding the offender accountable (changing attitudes/perceptions about violence).
• In a STOP project in Alaska, the staff realized they could not just "arrest all the men." So they got male relatives together to talk one-on-one and "lean on" the batterers. This has succeeded in changing community norms.
• Interdenominational or EAP (Employee Assistance Programs) can be useful parts for the education component of programs. A program has to "get to the cultural leaders" to secure real change.
• Circle sentencing and other alternatives (like some Islamic practices) are good for some communities.

Back to Top

Question 3: What Current Measures of the Impact of VAWA I Should Be Used?

This question was directed at eliciting ideas about existing measures that have not been taken advantage of in determining the impact of VAWA I. Participants were asked to consider valid evaluation measures of VAWA I programs, information that could be added to existing data sources or studies, and ways to augment current studies. One breakout group emphasized two measures that "cannot be used to measure the efficacy of VAWA interventions:"

• National level statistics that are not race or gender specific.
• Aggregate statistics from jurisdictions that have or have not received funds.

Ideas on what can be used and related comments are listed below.

VAWO Database

With respect to existing data sources, several participants noted a need to complete the Violence Against Women Office (VAWO) database to facilitate data collection on "the big picture" and make comparisons among jurisdictions—that is, which jurisdictions or regions have STOP and other VAWA funding (how much, for what types of programs, etc.).

National Crime Victimization Survey (NCVS)

A number of participants suggested using the NCVS, for example, "in combination with other data to look at VAWA sites and specific communities." Specific changes to the NCVS were also suggested (generally with VAWA II in mind):

• The numbers are small, but the NCVS does ask whether the response made things better . . . they may have said, "worse," because the offender got angry.
• NCVS does not consider "hands" a weapon, yet strangulation in domestic violence is a predictor of lethality.
• NCVS data on offenders’ race/ethnicity has only black, white, and other. There is a need for oversampling for various ethnic groups.
• Revise NCVS to include context codes (community) allowing researchers to control for other factors affecting rates.
• Try random-digit dialing method with smaller sample, rolling basis, 1000 every 6 months (misses women without phones and who don’t speak English/Spanish)
• Add questions about sub-populations (have you gone to a shelter? the police?); target geographic subsections.
• Target NCVS to participating communities with and without VAWA funding.

Other National Surveys

• Consider [adding onto] the CDC behavioral health risks survey.
• Existing surveys should add questions about violence against women (e.g., periodic youth survey).

National Incident Based Reporting System (NIBRS)

• NIBRS data still does not have a category for "ex-boyfriends" (and thus approximately 20 percent of femicides are not counted in this context).
• NIBRS can capture data on victim/offender relationships (living together, types of ex-relationships); however, "it’s not in there right now."
• NIBRS may have to change its race/ethnicity classification.

Interviews, Safety Audits

Many participants emphasized the "need to be sure we are hearing from the people we are supposed to be helping (victimized women)." However, they expressed doubts about whether there was sufficient funding to add victim interviews to existing VAWA I program evaluations.

It was also suggested that offender interviews could be fruitful despite issues of offender credibility; for example, one participant said that in the Gondolf study, such interviews pointed to the swiftness of the response as an important factor. Several participants also asked "whether safety audits could be incorporated into outcome measures somehow?"

Existing State and Local Data

• Look at prior domestic violence and gunshot wounds and intimate partner assaults (however, few state databases capture this).
• Consider using hotline data. One study [directed by Dr. Marcia Chaiken] is now looking at hotline data for 2-3 states. The hotline data does identify repeat calls…but the impetus for calling a hotline could have been a highly publicized homicide, rather than VAWA. Researchers might also review what data are available from the National Hotline.
• Data on "local community counter-violence efforts"—is this captured in existing national data?
• Is it too late for current evaluation projects to capture impact information? (For example, for training projects, what are we counting and what has happened?)
• Dispatch data from police may be more useful than arrest data or police reports; however 911 codes for domestic assault may be inconsistent, not used.
• Are any of the 3-1-1 systems getting domestic violence calls?
• Use data kept by community based organizations (e.g., for smaller ethnic groups).
• [Consider] reductions in calls to places that would have gotten calls without VAWA (e.g., emergency rooms); substitution; displacement; however, a number of factors may confound this.
• Look to the Williams and Brown [study] on intimate partner homicide and studies on sexual assault that measure, for example, access to services and the impact of protective orders.
• Shelters: number seeking help, number turned away from shelter, number kicked out after 30 days.
• Increases in SANE units? Increases in sexual assault response teams (SARTs)? Some jurisdictions have merged the two. Also, see if existing SANE/SART data could be used to evaluate the existing VAWA.

Other

One group suggested reviewing the Family Violence Intervention Panel recommendations regarding "VAWA 2000/domestic violence findings/research."

Back to Top

Question 4: What Future Measures Should Be Collected to Measure the Impact of VAWA I and II?

Various participants noted that VAWA II presents opportunities to collect baseline data that were not captured with VAWA I grants. Some said that "sampling, instrumentation, and methodology has to be adjusted to be more specific," and there was a call for more complex methodologies and longer-term studies. A number of advantages to using qualitative studies were also noted. Comments on future measures are grouped in terms of what should be measured, recommended methodologies, and data sources.

What Should Be Measured?

In one group, participants recommended involving victims in defining what should be measured, as indicated in this comment:

• What if women got to define their own increased safety and well-being [rather than focus on arrests and convictions]? This could be done through focus groups, qualitative interviews with women, NCVS data on women’s increased reporting. Ask women. But [expect a] critique [of qualitative methods and small samples]—small numbers aren’t respected (bar graphs are).

Other research areas and related comments from the discussions on Question 4 are listed below.

• Differential Impact

• Record differences in impact relating to community/racial aspects, count response rates.
• Include codes for context (e.g., economic, other factors).
• Measure impact on different racial and ethnic groups over the next five years with the new VAWA.
• Differences by race are seen in the ILJ arrest project evaluation.
• How many women lose custody (of their children)?

  Safety, Well-Being

• Aspects of economic well-being [could be] combined into studies (eviction, homelessness, food supply, utility shut-off etc. – is this linked to criminal justice events?)
• Ask safety/well-being questions longitudinally.
• Measure existence of various forms of power and control.
• Need to distinguish components involved in feelings of safety (why are women feeling safer, etc.?).
• [Consider the] different recovery structure and response for one-time incidents, such as types of sexual assault.
• Consider multiple offenses by the same perpetrator.

  Use of Services

• Use of criminal justice system elements vs. other common services (success may show in increased use of these other services). Study with matched pairs? Good comparison groups are hard to set up/find. Would have to use women who are not "identified" by the criminal justice system.
• We also need to consider impact at the individual level. How many people must victims have contact with as they move through the system? How many effective referrals were made?
• What are the differences between women who use services and those who do not?
• Measure how the tribal community is creating a way to deal with domestic violence.
• [Because of] tribal sovereignty, [we need to determine] how to measure what’s happening in tribes not receiving funds.

  Sentencing and Judicial Response

• Track allocation of court money.
• Regarding offender accountability, look at:
  • Sentencing
  • The same triad used in considering sex offenders, i.e., the presentencing report, sentencing guidelines, and victim impact, for VAWA II.
• Determine who ends up in court-mandated programs (appropriate? ethnic group?).
• Maybe think of success as, "when he has to leave and she doesn’t," as some have said; [but] judges would have to agree with that.
• Have there been changes in sentencing practices and have these been useful? To her? To the goals of the [women’s, victim, and system effectiveness] movements? . . .This should include questions about presentencing evaluations. Are they using them? Do they include prior offenses (i.e., as part of the push for accountability)? Are courts using sanctions? Asking the victim what she thinks might work? Does she have representation?
• A study on judicial demeanor [would be] a very important piece of research.
• [Consider (judiciary)] backlash.

  Femicides, Homicides, Suicides

• Conduct femicide studies.
• Look more carefully at why fewer men are being killed (gun availability? Incarceration? Or availability of services?).
• We haven’t even looked at suicides (ideation or action) among battered women.
• We could look at before and after VAWA regarding women in prison for killing an intimate partner, although this involves very small number of women; also look at what happens (e.g., to her children) when a woman kills.

  Low-level Violence, Coercion

• Examples of measures include entrapment/imprisonment in the home, having keys taken away.
• Some measures relate to psychological safety (e.g., husband always brings the children home late from visits with him).

  Domestic Violence in Rural Areas

• [Some communities need to] better define the problem of domestic violence. In some rural areas, "ownership of women is a big issue." Simply assessing rates of violence will not give the right measures.
• Some funding for rural programs has gone into a greater variety of services (e.g., school programs, community education).

  Community Ownership of Programs

• If we believe "community ownership" is a significant factor, then measures for this are needed.

Recommended Methodologies

• Qualitative Studies
• Qualitative data are better for identifying gaps, [however, it is] difficult to access more marginalized victims (no phone, relocation, immigrants).
• Use focus groups to get the perspectives of women involved in the criminal justice system.
• Qualitative data are required. Have to stop doing "status quo" research, not acceptable.
• Many immigrant communities have high levels of violence and little reporting. Random sampling will not give a true image in an immigrant community where most are not answering the questions.

  Community-level Victim Surveys

• Could communities be doing their own victimization surveys?
• Refocus attention to local surveys (neighborhood, community responses to intimate partner violence).
• States can use those surveys to gather better data and create ongoing tracking and survey systems.

  Local Case Tracking

• Identify a few measures for every entry point ([consider providing] laptop and software).
• For evaluating community effects, there is a need to identify a random group of women and track changes in expectations. Has the experience of support (define?) changed over time? Contacts with particular kinds of service?
• There should be two kinds of case-tracking over time (before and after VAWA intervention), with cross-sectional sample that has interviews of victims from different areas.

  Advocate Participation in Data Collection

  A number of participants recommended greater involvement of victim advocates in data collection; however, they also noted several related challenges, including the credibility of the data, confidentiality issues, and the change this represents in advocates’ role. Other comments were:

• Use victim advocates to create database (e.g., work with police, relationship changes. [They] also know women’s experience, can look at context like welfare, child protective services, non-criminal justice factors, and can also comment on CCR).
• Researchers need to be part of the community, develop links, and reject traditional models.
• The field will collect some data but doesn’t want to collect [an excessive amount]. For example, they might collect data for 4-5 questions regarding safety and 4-5 questions regarding accountability.

  Multiple Methods over Longer Time Periods

• Multiple methods over a longer time period and participation of victim advocacy are needed; multiple methods, including subjective measures of danger, quantitative/qualitative.
• Use cross-section for long-term design (must have more funding for this).
• Need longitudinal measures and representative measures; need to look at more than past year and lifetime (e.g., previous 2 years, 3 years).
• Calendaring.
• Conduct comparison studies (those [communities] with and without VAWA funding); need comparison/control groups.
• Consider different states of readiness in impact study and define outcomes.
• Pick communities at various stages on the continuum to track over time—what have we learned about what to do?
• Random community surveys that ask about effects.
• Purposive surveys/responses.
• Answer questions at the community level despite national scope of VAWA.
• Face-to-face interviews in the community.
• Multi-stage cluster sample.
• Acceptance testing, use most valid responses.
• Explore design options; staggered multiple baseline measures.
• Include measures of offender accountability in research design.
• Quality of data: efficiency vs. reliability.

Data Sources

Participants did not attempt to list all potential data sources; rather, they mentioned several that have not been used extensively in the past, as noted below.

• Hospitals, HMOs
• 80 percent to medical, 29 percent called police; HMOs and the medical system may be rich data sources.
• Hospital/ER data (more women may access. Could do random sample and then extrapolate. But problem identifying persons as domestic violence victims).
• Large HMO systems.
• ICD-9 codes at hospitals.
• Could take random sample from medical data and interview women in depth.
• Health data are not difficult, but school/housing authority data may not be dependable.

  Other Data Sources

• Indian housing service, schools run by tribes, housing run by tribes.
• Nonprofit victim services organizations.
• To really focus on safety of the women, information should be collected from community centers, clinics, and religious outreach organizations.
• The next thing is XML (extensible markup language) instead of HTML. It has filters and levels of security. It imbeds actual data, for example in a petition or order. XML and electronic filing are the big things with respect to courts.

Back to Top

Recommendations on Federal Roles

At various points in the breakout group discussions, participants made recommendations which would either require federal action (e.g., VAWA II grant application requirements, special conditions), or which could benefit from federal funding or a federal leadership role. Some of these were discussed earlier (e.g., completing the VAWO database, making enhancements to NCVS and NIBRS, encouraging inter-agency collaboration on surveys). Other recommendations regarding a federal role are listed below.

• Require tracking of data/numbers before the federal grant is given; start baseline data gathering early; ask certain core questions of all VAWA grantees.
• Implement a two-stage grant process (delay initiative for 1 year, first year involves data gathering (e.g., startup grants getting coordination started).
• [Have] stricter stipulations about who gets the money. What should grant recipients have in place for data collection to receive the money? Certainly, we want to have better data tracking.
• We need some way to standardize the terms we use. Also, require that grantees have a logic model: this is what we have, this is what we want to do, these our our objectives, these are the outcomes we want.
• Require communities to report basic data (e.g., number of domestic violence arrests).
• Require that grantees be testing some hypotheses.
• More education about the complexity of domestic violence programs is needed, and the evaluations need to be as complex as the programs.
• Challenge to DOJ: How to make it practical, feasible for states to collect data (training, resources, identification of variables).
• Provide victim advocates with the ability to collect data on subsets of women; funding is needed, advocates are overworked.
• Make data collection exciting and communicate importance. Develop a prototype of this and standardize data being collected now (uniform categories, software package with common data elements).
• What about requiring a community resource coordinator for some grants (e.g., rural, arrest)?
• Don’t forget to go into communities and talk to people as well.
• Provide more funding for training development; access, time [are also training issues].
• Provide funding support for qualitative studies.
• Provide data on how all DOJ money has been spent on violence against women, not just VAWA money.
• NIJ should continue publishing information on smaller, qualitative studies; the NIJ "RIBs" are valuable, but there is a need to contextualize the findings. Either the researcher or a practitioner needs to discuss the findings in lay terms.

Back to Top