Q&A: Alicia Wilson-Ahlstrom on Choosing the Right Outcome-Measurement Tool

tags:

report cover reads: Fron Soft Skills to Hard DataYouth workers believe that what they do every day makes a difference in the lives of young people. But policy makers, donors and the public want to see hard data that shows youth programs work. To help youth-serving agencies measure their success, the Forum for Youth Investment recently released “From Soft Skills to Hard Data,” which reviews eight outcome-measurement tools that can be used to evaluate afterschool programs and other settings.

NCFY spoke with Alicia Wilson-Ahlstrom, a lead author of the report, to learn more about how to choose the right outcome-measurement tool for a youth program.

NCFY: What keeps youth programs from measuring outcomes?

Wilson-Ahlstrom: Programs think they lack the time and staff expertise to devote to finding tools that are a good fit for their program’s outcomes, to administer the tool, and to communicate the information within the program and back to community stakeholders. But evaluation doesn’t have to take a lot of time if you choose a good measure for your program. There are also ways to save time by using slightly more expensive programs with computer-based elements, which we indicate in the report. Our hope is that this resource will help youth workers understand what tools are out there and navigate them more quickly and successfully.

NCFY: How should programs decide which tool to use?

Wilson-Ahlstrom: Programs should choose a tool that does a good job of reflecting the outcomes that they hold themselves accountable for or that the community is holding them accountable for. For some programs they’ll have to take a step back to figure out what the priorities of the program are before they can measure them. If you have three areas that you’d like to measure and find a tool that evaluates them, don’t shy away from it because it has none-to-limited evidence. All of the tools selected in this report are strong in different ways. Programs should also make practical considerations around cost and format. If your program doesn’t have a lot of access to computers, it wouldn’t make sense to have an online measurement system. There are also advantages to choosing a measure which is more customizable. We indicate which programs you can do that to in the report.

NCFY: What would make a tool a good fit or a bad fit for a program?

Wilson-Ahlstrom: Your measure should evaluate what you’re good at—the x, y, and z you contribute to young people instead of the whole alphabet. We’re in an environment where we’re expected to “over promise,” but programs shouldn’t be held accountable for outcomes that they were never required to meet. We’re part of a larger community that provides a whole range of services and support to young people, and we need to situate ourselves in that network. This guide was put forth in part to say let’s let programs do what they do well.

NCFY: What do you hope a youth worker would take away from this report?

Wilson-Ahlstrom: I hope that we’ve helped some portion of the professional workforce cut through some of the decision making. I hope we’ve answered some of the big questions around outcomes and help programs decide how to measure their outcomes, whether they’re taking a tool off the shelf or developing their own. We also want to give programs permission to focus on what they do well and the outcomes that flow from that, to help programs situate themselves in a larger environment of supports and opportunities that all young people need and deserve.

National Clearinghouse on Families & Youth | 5515 Security Lane, Suite 800 | North Bethesda, MD 20852 | (301) 608-8098 | ncfy@acf.hhs.gov