The Evaluation of NIJ by the National Research Council of the National Academy of Sciences: NIJ's Response

Moderator:Thomas Feucht, Executive Senior Science Advisor, National Institute of Justice
Panelists:

• John H. Laub, Director, National Institute of Justice
• Daniel Nagin, Professor of Public Policy and Statistics, Teresa and H. John Heinz III University, Carnegie Mellon University

Thomas Feucht: Good afternoon. Welcome to this session of the NIJ conference. I'm Tom Feucht from NIJ. I'll be the moderator for this session.

The National Research Council conducted a study of the National Institute of Justice, completed last year, entitled "Strengthening the National Institute of Justice." This was a study that was done at the request of NIJ. It was commissioned in the spirit consistent with the Government Performance Results Act, ideas of accountability, evaluation of programs and agencies, and the need for independent evaluation. This study was completed and published in July of 2010.

The study centered around five broad recommendations, and they are here. Since this session presumes a certain amount of familiarity with the recommendations in the study themselves, I'll give you a second to look them over. This session is focused primarily on NIJ's response to the NRC study and the recommendations of that study.

Two recommendations, number one and two, ensuring independence and improving governance; number two, strengthening the science mission — .

[Inaudible]

Feucht: Number three, bolstering the research infrastructure.

Feucht: Number four, enhance the scientific integrity and transparency of research operations. And number five, establish a culture of self assessment. In this panel, we'll hear first from NIJ's Director John Laub, who is releasing today the NIJ response here at the NIJ conference and also on the NIJ website, www.NIJ.gov. The response that John will refer to is front and center on that page, so after this session, if you want to see a full-length text copy of that, you'll be able to get that on the website. We'll also hear from Dr. Daniel Nagin, who was a member of the NRC Study Committee that conducted the initial assessment of NIJ.

Now, it's not every day that you get to introduce a Stockholm prizewinning criminologist, and it's rarer still that you get to introduce a Stockholm criminology prizewinner twice in one day. For the second time around, John Laub is Director of the National Institute of Justice. He was nominated to that office by President Obama and confirmed last summer by the U.S. Senate. He is Distinguished University Professor of Criminology and Criminal Justice on leave from the University of Maryland. He hails from SUNY Albany and also from Chicago.

He was elected President of the American Society of Criminology. He and his co-panelist, Dr. Nagin, are both Southerland Award winners from the American Society of Criminology.

Daniel Nagin is Teresa and John Heinz III University Professor of Public Policy and Statistics at the Heinz College, Carnegie Mellon University in Pittsburgh. He, as John is, is an Elected Fellow in the American Society of Criminology. He is the 2006 award recipient of the Southerland Award from ASC. His research focuses on evolution of criminal antisocial behaviors over the life course and, in particular, deterrence.

So we have, I think, two very interesting folks to hear from today on a very interesting issue. If you care about the direction of research and you care about NIJ's role in research on crime and justice, I think you'll find not only the report, if you've read it, the report from the NRC, interesting, but perhaps even more compelling, NIJ's response to that. And without further delay, I give you John Laub.

John Laub: Thanks. Thanks, Tom. Tom said that Dan and I both had to stand at the podium, but we said no.

Feucht: That's right. [Laughter]

Laub: So we're going to sit here, and if people have trouble seeing me or hearing me, just yell and I will stand, but I'd like to just make this as informal as possible, and just want to thank Dan for his willingness to come and comment on our response.

As Tom mentioned, the response to the NRC report is now public, and I can't stress enough how important the NRC evaluation is for the National Institute of Justice. As some of you know, I sat on the Crime, Law and Justice Committee, a part of the National Academy of Sciences, where this evaluation was first discussed a number of years ago. I believe the work of the National Research Council is very important in terms of shaping the future direction, not only for science agencies like NIJ but also for the field at large, be it through deterrence, incapacitation, juvenile justice, criminal careers, so on and so forth. So this is an extremely important document.

And I also think, although I have to tell you that I kept on, as anyone knows the National Research Council, there's a lot of secrecy, and even though Charles Wellford, the Chair of the Committee Office at the University of Maryland, was two doors down from me, I felt like I knew nothing about what was going on. All I kept saying to Charles is, let me get to NIJ three months before it hits the street, please.

Well, the report was released July 22^nd, excuse me, July 2^nd; I arrived at NIJ July 22^nd. So up in a beautiful house in Maine, I was reading the National Research Council report and said, you know, I could just stay here and not come back. [Laughter] So it was a critical report if you've read it. But I also think it was a report that was much needed, and I do think all academics are sensitive to criticism. All our work is perfect. I do think it provides a blueprint for the future as we move ahead. And that's the spirit in which I view this document, and that's the spirit in which I felt it was important at the National Institute of Justice to embrace it.

And so, what I did basically, I want to just tell you a little bit about the process that NIJ took to react to, discuss this report, going to talk about some of our key accomplishments to date in light of the five recommendations that are now bulleted up on the screen, and then identify three remaining issues that I feel NIJ has not resolved and need more discussion of as we move forward. So that's the plan for my presentation.

So first with respect to process, because I thought that this document was so important, and I do think it's going to provide the future blueprint for the National Institute of Justice, I believe that it was vitally important for everybody at the Institute, and I mean everybody at the Institute, to read the report, be involved in the discussions and talk about what it means for the Institute as we move forward.

In fact, this became an opportunity for us. In some ways, it provided me a perfect leverage to say let's talk about what our mission is; let's talk about what we should be doing. Yes, the NRC report is providing us the opportunity to do so, but let's really take advantage of this and have a broad-based discussion not only of the five recommendations, but really other things as well that are related to the Institute.

So upon my second week, I was told that the executive staff, as they're called, meets every Tuesday for 90 minutes. These are the supervisors and managers at NIJ, and I said, okay, we're going to dispense whatever business you normally do, thinking that they're a lot like faculty meetings, probably not terribly important, and what we're going to do for the Tuesday time period until further notice is we're going to discuss the recommendations, each in turn.

So we started with recommendation one, and we did this as the executive staff, I'd say there were about 15 of us, we did this for roughly 10 weeks, so August, September, October and into November. And for me, knowing that I was going to face Charles Wellford, the Chair of the panel at the American Society of Criminology meetings in November, was the kind of benchmark. We needed to have this discussion before I get to San Francisco because I don't want to embarrass myself or NIJ.

However, it couldn't just be owned by the executive staff. So what I did was I identified the 10 subunits at NIJ, as you know, we have the Office of Science and Technology, we have the Office of Investigative Forensic Science. We also have the Office of Research and Evaluation. We also have a Communications Office, our Publications Division and we have our Operations Division, the real power, the budget.

And what I did was I identified, through help, but identified one person in each of those units who was not an executive staff member, and I said, we are going to meet every Wednesday for 90 minutes for the next 10 weeks and we're going to discuss the recommendations. So my hope was that I would have at least 10 people in the room with me and we would do this, but I also said that the meeting was open to everybody at NIJ except for the executive staff.

And I have to tell you, the first meeting we had, there were 23 people there, which doesn't sound like a lot, but NIJ has 88 employees and it was roughly 40 percent of the non, when you eliminate the executive staff, 40 percent of the agency was at the meeting. And I would say we averaged between, I think the lowest number we ever had was 18, the high was about 26.

And so, what we basically did, the idea was to have these two parallel discussions about the recommendations in turn, and I have to say we spent probably three weeks on number one, another two to three weeks on two, and then recommendations three, four and five went fairly quickly, relatively speaking. But we had these two parallel meetings in synch, which I chaired both of them; we had really good discussions, frank discussions, honest discussions. I learned a lot about the agency. Somebody said to me, "What a brilliant idea to include the working staff." It wasn't a brilliant idea; I was pretty much going by the seat of my pants. But it worked out well; I learned a lot about the agency and I learned a lot about how this report was viewed and how important it was as we move forward.

After that process, we went off and did our ASC thing where we talked about this and so forth, and then we came back together and we had two all-staff meetings where we talked about, with the exec staff and non-exec staff, the two documents that each group produced in light of the recommendations. And we had extensive discussions of that, and then come spring, I decided, okay, I have heard from folks; the process seemed to work. We actually had surprising, in my view, a fairly broad-based consensus on how we should respond to the recommendations, and I would say the vast majority of the recommendations were accepted, if you will, by the Institute.

However, there were differences with respect to some of the issues that I'll talk about at the end, and there were some slight differences between exec staff and working staff, as they're called, but nothing really major. It was more nuance and complexity and so on and so forth.

So basically, what we did is we have these two internal documents that provide summaries of our meetings, and then I took about a month and I wrote the response, which you see posted on the website. So it's basically John Laub's response as the Director of NIJ to the NRC.

So what have we done with respect to these recommendations? And I want to talk about the accomplishments to date going each in turn. With respect to recommendation one, ensuring independence and improving governance, we have begun discussions, and now that the report is made public we will begin more extensive discussions, about drafting proposed language for changes in the statute to establish, number one, clear, necessary qualifications for the NIJ Director in terms of experience with science and research, which was one of the recommendations of the report; to modify the appointment of the NIJ Director to be a term of six years, much like other science agencies, a fixed term; and to clarify the independence of NIJ in all aspects of its key work, particularly in commissioning research and in publishing and disseminating research findings. Okay? So we are beginning those discussions, and we are taking those recommendations that were made by the NRC panel to heart.

With respect to the independence issue, I want to point out that as some of you in the audience know, one of the issues that's been debated over the last few years is whether or not NIJ should remain in the Office of Justice Programs or whether it should be an independent agency outside of OJP as it's known. The NRC panel recommended that NIJ stay within the Office of Justice Programs; they saw great benefit from that with respect to particularly having program folks like the Bureau of Justice Assistance being able to work with the research and evaluation arm.

I could tell you both professionally and personally, it's really nice to have a friend as the Director of the Bureau of Justice Statistics because I think the statistical agencies and the research and evaluation agencies should work hand-in-hand, and it would be really hard to be able to run down to Jim Lynch's office if I weren't in the same building as him and say, "I'm going crazy; help."

So I do think there is really good reason to keep the agencies within OJP. The reality though is that that does suggest things for our independence, and for me, the most important pieces of how we define independence is whether or not we can control the research that we want to put out on the street; secondly, that we are able to publish and disseminate research findings; and thirdly, we have control over the peer review process. Those are the critical aspects of independence, and we're going to talk about peer review in a little bit because we are examining our peer review process, and I believe there is going to be the independence we need to establish the peer review process.

Now, there are a whole lot of other things that go into what it means to be independent. Control of your budget, and I will not tell you who told me this when I said, "I want to control my budget," somebody who is much more smarter than me, much more politically astute than me, said, "John, President Obama doesn't control the budget; good luck." [Laughter] And I think there are realities remaining in OJP that we have to face, and that's one of them.

I also don't think we're going to have the power to hire and fire. I don't think anyone has that in federal government. And so, there are going to be tensions regarding certain aspects of independence, but it seems to me we have to privilege the commission of research, the publications, dissemination of findings and peer review; those are absolutely crucial if we're going to be an independent science agency.

Now, with respect to number two, and I think this recommendation is the most important: here, there's been a number of things going on. Even before I arrived, some of these things were going on. So each of the three science units at NIJ, forensic science, science and technology, and research and evaluation, had begun strategic planning regarding research priorities and research agendas and particularly talking about a multiyear plan.

I've been very critical of the research portfolios of the National Institute of Justice. They appeared to me as having a cafeteria-style buffet, a little bit of everything and sometimes things don't go together very well.

One of the things that the National Research Council pointed out was the need for NIJ to begin to build, and contribute to, a cumulative base of knowledge. The way that you do that, I believe, is first off, make some tough priority decisions. We cannot evaluate everything; we need to evaluate what's most important for the field and that has to be the starting point.

Secondly, I think what we have to do is we have to begin to think about a multiyear planning stage. We have to ask ourselves, and I started to do this when I had an experience with Redbooks at the end of July, okay, if we fund this and it's successful, what do we do next? What does the field do next? And that has to become second nature to how we think about things with this eye of building this cumulative base of knowledge.

So the Office of Research and Evaluation, the Office of Forensic Science, the Office of Science and Technology, all have engaged in a strategic planning process to begin to focus on building these research priorities.

To give you an example of one of the things that the Office of Research and Evaluation is doing, is they are convening topical working groups, and over the last year we've convened three topical working groups, which involved bringing in leading experts to help us discuss the existing research, to identify the emerging gaps, the emerging issues in the research areas, and the gaps in the subject matter under discussion.

The three substantive areas that we've looked at to date: the first was crime prevention; that group met in October of 2010. The second focused on gangs, and that was in February 2011. And most recently in April 2011, we had a third focused on neighborhoods and crime. And there are plans for doing more of these topical working groups and we're in the midst of doing the topical working groups this fall looking at firearms, drawing on the National Research Council report on that topic.

The Office of Investigative and Forensic Science has basically engaged in a strategic planning process for the research development process and portfolio, and as a result, they made a number of changes for this year's solicitations that I just want to share with you.

There is a basic research solicitation that will supplement our applied research programs. Since I'm talking about the NRC report, somebody sometimes will ask me, which report are you talking about? There was a report that the National Research Council did looking at the state of forensic science in the United States, and as you know, that report is very critical about the lack of science in many forensic sciences. And we're very sensitive to that and are trying to develop basic research to respond to that need.

We also added a new investigator qualification to solicit proposals from those working in the life and physical sciences who are not doing research in forensic science. And finally, we are attempting, we are, not attempting, and hopefully this will bear fruit, we are also incorporating evaluations of all of the training programs we're doing. We do a fair amount of training in the area of forensic science, but we don't do much evaluation. And so, we want to look at that as well.

With respect to the Office of Science and Technology, there have been several steps taken there to strengthen the science mission. In particular, we have looked at the structure of what are referred to as the National Law Enforcement and Corrections Technology Centers, and the idea is that these centers are being redesigned to be more consistent with the science mission and in the process of addressing many of the concerns listed in the report, and we now have, and we have done some reorganization, and right now, we support four technology centers of excellence, as they're called, in the area of communications technologies, electronic crime technologies, information and sensor systems technologies, and weapon and protective systems technologies. And again, I would say I am not satisfied where those centers are with respect to supporting the NIJ science mission, but I do feel progress is being made and we're moving in the right direction.

Another thing that happened with respect to strengthening the science mission is the Office of Justice Programs in January 2011 convened the Science Advisory Board, and this is made up of a distinguished group of researchers and practitioners who will provide direction for the Office of Justice Programs. And from that Science Advisory Board, there is a National Institute of Justice subcommittee that has been created, and the subcommittee will provide input to NIJ particularly regarding the strengthening of its science mission.

I also don't think that part of strengthening the science mission at NIJ has to be substantive. It's not just about process; it's not just about bringing science into our discussions.

And so, one of the goals that I had as a way of strengthening the science mission at NIJ was to begin to work with, first, other units within the Office of Justice Programs on what I consider to be cutting edge areas of research. And the first that I want to talk about is a joint project that's going to be launched this summer with Jim Lynch in the Bureau of Justice Statistics, and the title of this project is "The Mining of Police Data for Statistical and Research Purposes."

For those of us that have been in the crime business for a long time, we've had lots of complaints about how slow it is for the FBI to get information out and kind of the cumbersome nature of those data. Well, as we all know, lots of police data are being posted on the Web, and what we're going to do as an initial piece is systematically assess for the 100 largest cities in America what is, in fact, available on their websites, what's the quality of the data, what's the range of the data, how often are those data updated, and the idea eventually is to combine what we'll do from that Web-based survey assessment with the ongoing LEMUS data that the Bureau of Justice Statistics does to begin to tie together whether or not there are hotspots of innovation. Are certain police departments that have certain characteristics more likely to have better quality data? And we see this as the beginning of a way to really provide much richer data in a quicker fashion both for statistical purposes and ultimately for research and evaluation.

We also, just as a way of really kind of jumping to some of the other recommendations, but I do think it's part of the science mission, is BJS and NIJ has launched a monthly brownbag seminar series where we will have a member of the BJS staff present research. We had the first one. Janet Lauritsen was the Visiting Fellow and she presented her work on chronic repeat victimization, and NIJ staff presented their work on sexual assault kit backlogs at the most recent one. They are suspended until the fall, but we were able to launch that this year.

Another area of new substantive interest, the Bureau of Justice Assistance and the National Institute of Justice just released the solicitation for a demonstration field experiment evaluating Project HOPE. This is the Hawaii, and let me get this right because I always mess it up, the Hawaii Opportunity Probation with Enforcement Program. And for those of you that don't know, it's an innovative probation initiative designed to reduce recidivism. It's based in Hawaii. Judge Steven Alm is the person that's really behind this idea. And I think that from the standpoint of thinking of my colleague's work here, Daniel Nagin, focusing on certain sanctions as opposed to severe sanctions, quick response to probation violations, I think this has enormous potential for providing enormous understanding with respect to looking at innovative ways to reduce recidivism that don't involve lengthy jail sentences.

So we are going to be funding randomized controlled experiments. First off will be BJA will be funding the programs, and NIJ will be funding the research and evaluation. We will have four sites that we will be selecting for this study, and they will be using the gold standard, if you will, of evaluation research: randomized controlled trials. And so, that's going to be happening this summer; the solicitations are on the street right now.

BJA and NIJ are also doing extensive work looking at the Second Chance Act in offender re-entry courts. That's ongoing.

And then finally, we've had a long history at NIJ of working with the Office for Victims of Crime and the Office on Violence Against Women, and this year, we have an Action Research Program which is examining the untested sexual assault kit backlog, trying to identify what are the reasons for the backlog, and what can be developed with respect to best practices.

But that's not all. One of the things that I've tried to do is connect NIJ's science mission to other federal science agencies. So I spent the last, I'd say, since I arrived in August, first six to seven months meeting with different science agency heads, Education, National Institute of Drug Abuse, National Science Foundation, National Institutes of Standards and Technology and so forth, because again, for those of you that know the political scene, science and the Justice Department, science and lawyers, sometimes all doesn't fit together very well, so we have to remind people sometimes that there is, in fact, a science agency in the Department of Justice. And that has been quite successful, I think, in terms of having good meetings, and I am hopeful that there will be some collaborative efforts that come out of that, particularly between the National Science Foundation, National Institute of Drug Abuse and the National Institutes of Standards and Technology, that I think will lead to some collaborative efforts down the road.

And then finally, I've also reached out to private foundations, and it has, I mean, Laurie Robinson, my boss, is terrific, but one of Laurie's mantras, which I appreciate a lot more now than I did 11 months ago, is the importance of partnerships, and one of the important ways in which NIJ could move forward, I believe, is partnering not only with other science agencies, but also private foundations and folks outside of government. As some of you know, you've heard me speak about this, I'm very interested in how research evidence is viewed and used by practitioners in the field. I feel we, I've always said that we ask our researchers to use rigorous science to produce knowledge, but when we have discussions about research dissemination and research use, we rely on antidotes: well, I think it goes like this.

And what the W.T. Grant Foundation has launched in the last year and a half, I believe, is a very exciting effort in the field of education regarding kids, and what they're doing is systematically looking at how educational practitioners and educational policymakers define research, acquire research and use research in their work. And I basically, and I'm not bashful about this, I literally want to steal everything that they have and apply it to the field of criminal justice because I think we can learn a lot from that.

And so, we've been talking about the idea of doing a joint effort in this area, and hopefully, the United States Congress will be cooperative and will allow us to do that.

Let's see. A couple other things. And again, in terms of establishing NIJ as a science agency, I felt that it's really important for NIJ not only to reconnect with other science agencies, to reconnect with the research community which we've done with the respect to the American Society of Criminology and the Academy of Criminal Justice Sciences, I also think it's vitally important for NIJ to reconnect with Congress, and particularly congressional staffers, and to talk about the New Day at NIJ, the importance of science, how we're responding to the NRC report. And so, I have spent as much time as I could meeting with congressional staffers about the New Day at NIJ and some of our efforts in this area.

Now, at the core of a strong science agency, I believe, is a rigorous and fair peer review process. And when I first arrived at NIJ, we go through a period called Redbooks, grant awards are made, and suddenly, my phone started ringing. I never get phone calls, but I was getting phone calls from friends and colleagues and having to hold the phone sometimes like this: you know, what's wrong with your peer review process? It stinks. Who reviewed my [trails off, harrumphing]--? [Laughter] Terrible. And then there were some people who wanted to meet for lunch and I got yelled at at lunch, and so forth.

Now, I have no idea, I mean, the joke at our holiday skit party was John Laub was going to participate in all the peer reviews because he thought himself as a researcher and not somebody who manages research. I would love to participate in all the peer review so I could understand what goes on. However, I can't, and quite frankly, I don't know if the peer review system is broken or not at NIJ. However, I believe that perceptions matter, and there are perceptions that the peer review process at NIJ, for a variety of reasons, is not working the way it should.

And I take that very seriously because as a former journal editor, if you do not have a good peer review process, you are not a science journal, and you cannot have a science agency without a rigorous fair open competitive peer review process.

And so, I am excited that Tom Feucht and Phyllis Newton and others really met the challenge of coming up with alternative ways of doing peer review at NIJ, and at this conference, we're announcing that we are exploring a pilot study for the fall, next year's fiscal year, where we will be using standing peer review panels. We are going to pilot this both in the social sciences as well as in the forensic science, as well as in the science and technology, and again, programs within these. We're not going to do it wholesale. Got to go slow with change in government, folks; can't be too bold. But we're going to start small and we're going to learn a lot, I think, from this.

But it is, I believe, going to be the steppingstone to standing peer review panels at NIJ. That's the way that other science agencies do it. It's more transparent and I believe will provide more continuity year to year with respect to the peer reviews, which is vitally important. And we're going to be talking more about this tomorrow at a session that Tom is chairing and Phyllis will be at, and we'll be actually involved in, so we'll be doing that tomorrow.

Just a couple other things, I guess; I'm not sure, I had three hours you said for this? [Laughter]

Feucht: Just like this morning. [Laughter]

Laub: Yes, right. Another 10 minutes?

Feucht: Sure.

Laub: Okay.

Feucht: No problem.

Laub: Yes, we're trying to build a culture of science, and again, symbols are important. I think NIJ should be a culture of science. We've invigorated the Research in the Real World series, brought some of the best researchers in for talks. We're doing an internal seminar series at NIJ. We're also looking into creating, recreating, if you will, an intramural research program. And so, there are a number of things going on there.

With respect to recommendations three, four and five and I'm just going to give a couple examples here, with respect to three, bolstering the research infrastructure, the NRC report actually had a number of complimentary things to say about our Graduate Research Fellows Program. And we are expanding that to include forensic science and physical and technical science and I think that's going to be very exciting.

We also are reestablishing the Visiting Fellows Program, which has been somewhat dormant, and we're starting with, I mentioned this morning but I'll repeat here, we're having our first Senior Executive Fellow, Jim Bueerman, retired Police Chief from Redlands, California, is going to come for the month of July to help us on translational criminology. And we're in the midst of looking for folks to come for a yearlong stay in a more traditional research fellowship. So that's ongoing.

With respect to enhancing the scientific integrity and transparency of research operations, again, for me, this all falls to peer review. Peer review has to be at the top of that list, and I think that if we could establish these standing panels, that will help us go a long way in enhancing our scientific integrity along those lines.

And also, I think with respect to transparency and scientific integrity, we are doing a number of things on the Web. One of the things that I said I thought was very important was to keep the community abreast of how we are responding to the NRC report. So we've had periodic updates regarding that.

And one of the things I would like to do is begin to have periodic updates regarding our active grants so people don't have to wait until the final report but they could see what's happening, and this is something that the Bill and Melinda Gates Foundation has used with respect to their work in education, as well as their work around the world with respect to child health.

Establishing a culture of self-assessment, recommendation five: this is absolutely crucial. I had the good fortune, although I have to tell you I was scared out of my bejesus, to be part of a 15-person senior agency official group that met in the situation room in the West Wing of the White House. The meeting was chaired by Bill Daley, and 15 minutes into the meeting, President Obama came in and met with us for about 15 minutes.

And I came away from that meeting thinking about how important it is for us to demonstrate that the investment in NIJ is worth the taxpayers' money. I think that this is absolutely crucial. And I actually, at our last NIJ all-staff meeting, posed a very scary counterfactual. I said, we have to think about the counterfactual as what if NIJ didn't exist? What would the world look like? What would happen to crime research? What would happen to criminal justice research, because I think answering that counterfactual question in a systematic way is going to help us establishing that our programs are vital, they're necessary to the nation, which I believe they are, but we need to demonstrate that.

And so, we have begun looking at this self-assessment with respect to measuring the investment in research dollars. I also think this is where the NIJ subcommittee of the OJP Science Advisory Board can play a particularly important role in both identifying research priorities but also help us assess how those portfolios go further.

Now, I said that there were controversies, so let me touch on those three quickly before I turn things over to Daniel Nagin. Independence: we had long, hard debates, and I will say without, and I don't think this is talking out of school, there are some people at NIJ that feel NIJ should pack up their bags, leave the seventh floor, are we on the seventh, yes, seventh floor of the building and go somewhere else. Where that somewhere else is, I don't know, but that's what they'd like to do. They really feel strongly that for us to be totally independent, we need to be removed from OJP.

I believe that with the current Attorney General, and more importantly, the current Assistant Attorney General, the necessary requisite independence for NIJ can be established. However, I think it has to be institutionalized; it cannot be dependent upon the personalities of Laurie Robinson and John Laub. We all have an expiration date on our forehead and Bob Gross says we all have an expiration date, John, don't worry.

The point is that we need to establish an institutionalized and requisite independence, and as I said for me, it's about what research gets commissioned, what publications get out and peer review. Those are the three.

And so, my hope is that we can establish that independence, and I put the target of we need to do that and lay the groundwork for that within the next six months. I think the clock is ticking, and I'm hopeful that we could establish the necessary and sufficient independent authority that's practical. I mean, we're not going to leave the Department of Justice, we're not going to leave OJP, we're not going to get control over the budget, we're not going to have our own general research, excuse me, Office of General Counsel and so forth.

So what's practical? What could we do within the timeframe? And I would say, much like the NRC report recommends, that if the necessary independence is not established, I think this issue needs to be revisited two years from now, and perhaps this could be the task for the 2013 Scientific Advisory Board, to look at that.

The second issue concerns the Advisory Board. The NRC recommended that NIJ have its own Scientific Advisory Board. Well, Laurie Robinson did not know that the NRC report was going to say that, and she established an OJP Advisory Board. And it seems to me that given that board, and particularly for us, given the NIJ subcommittee, because on the NIJ subcommittee, we were able to add members that did not, that were not represented on the larger Scientific Advisory Board, which was largely criminal justice practitioners and social scientists, Al, no disrespect to operations researchers, [Laughter] but you are a social scientist now.

So the point is that now we have a subcommittee that represents the three bedrock sciences at NIJ. We have a forensic scientist on the subcommittee, we have physical and technical science, and we have somebody that actually cuts across both areas, as well as a subgroup of really strong social scientists.

So my take is let's work with this NIJ subcommittee and the larger Advisory Board and see what happens. I think the reality of NIJ creating its own Advisory Board at this juncture is not very likely, and I don't think, quite frankly, it's worth our efforts. I would rather work with the existing structure and see if it works out. But again, NIJ reserves the right within a two-year period to, I guess the same way that Governor Perry wants to secede Texas from the Union, we would secede from the OJP Advisory Board, NIJ subcommittee, so on and so forth. So again, I hope we don't have to revisit that issue, but I think that is an option for us down the road.

And then finally, what's not up there in recommendation two, was strengthening the science mission and removing capacity building from the NIJ portfolio. And for those of you that don't know, capacity building is large, large sums of money that Congress gives to NIJ to fund crime labs, to address the DNA backlog and so forth. And the argument that the NRC group made was that by having that located within NIJ, you weaken the science mission of NIJ.

And our point, I think on this, is twofold. One is we believe that you could remove the capacity building from the National Institute of Justice and not strengthen the science mission; we actually see them as independent. And we could envision keeping capacity building and, in turn, strengthening the science mission. So this was something that we have debated within the agency.

Now, we have decided, and since I see Allan [phonetic] there, I probably should say that I have decided, we have decided to remove the Paul Coverdell Forensic Science Improvement Grant Program from NIJ, and as soon as the report is released, and Tom tells me it was released at 3:45 p.m. this afternoon, I will begin discussions with BJA about them taking over that particular program, the Paul Coverdell Forensic Science.

However, with respect to the DNA backlog money and so forth, it is an open question from NIJ's standpoint of whether or not capacity building is consistent with the science agency's mission. And one of the things I think that's important is to look at whether or not NIJ's role in the forensic science community, particularly regarding DNA backlog but also writ large, whether or not NIJ's role should be to enhance how capacity is done and create really a synergy between capacity building and research. And there are other arenas, environmental studies, public health, where capacity building and science work hand-in-hand and they're not separate; they're actually integrated.

So basically, NIJ is going to be looking at this issue, and hopefully we'll be resolving this issue over the next six months because I believe that it is an extremely complicated issue and it is not easy to simply draw lines as to what's capacity building that's not linked to research, what's capacity building that's linked to research, because they are often one-in-one.

And so, I think it's important for us to really look at this issue and look at other avenues where there might be models that we could use for keeping the two together as we strengthen the science mission.

So that's where we are. And let me say that again, I can't thank the NRC enough for doing this evaluation. I sure wish it would have, I wish the U.S. Senate had confirmed me so I could have gotten to NIJ before the report, but in some ways, the timing was really perfect in terms of coming hand-in-hand.

And the NRC report does not talk about enhancing the NIJ budget. Notice I did not talk about enhancing the NIJ budget. However, I don't think it's any secret for us to do the kind of science that I envision NIJ doing, we need more funds. But that's down the road; we need to do a number of other things first.

So I think it's with a renewed sense of purpose that we are bringing together three bedrock sciences at NIJ to enhance criminal justice policy and practice, and we look forward to working with all of you as we move forward. So we'll stop there.

Daniel Nagin: Well, thank you. First of all, I want to point out that I am a very poor substitute for Charles Wellford. Charles Wellford was the Chair of this committee, and I can't express too much admiration for the hard work he did in writing and organizing this effort. So he really deserves an enormous amount of credit. He is with, I think this is his 50^th anniversary and he's with his wife in Martha's Vineyard at the moment.

And so, in that regard, I also want to acknowledge Betty Chemers and Julie Schuck, who are in the audience, who also worked very closely with Charles on developing this report.

So I also want to indicate that I'm speaking for myself now; I'm not speaking for the NRC and not for Charles, although I think Charles would agree with mostly, with some of my comments.

So as a prelude to my comments, I, first of all, want to express my admiration for what John has already done. I think that it's quite remarkable in several dimensions. One is that some of you may know that more years ago than now than I care to think about, I spent six years working in government for a former Attorney General, but when he was Governor of Pennsylvania, Dick Thornburgh, and during that period, it took me six years to figure out how to manage and run an agency in the way that John seemed to have figured out in two weeks. [Laughter] So I express, well, to say you're doing a terrific job.

So let me make some comments. I have a couple of comments, and I would just call them residual concerns or uncertainties about this because again, let me reemphasize, I think John already has and is moving on 95 percent of what we recommended, and for that to happen so quickly is really quite gratifying.

So in raising these, in some cases, they're just simply observations. In other cases, I'm presenting an alternative view on some things, and I'm not even sure that they're even mine, but they are an alternative view. And specifically, I want to comment on the last three items that John mentioned, actually in reverse order because I want to go from what I think is the least important to the most important.

So I first want to comment on the issue of the capacity building function. The second is the Advisory Board function, and finally, the independence issue.

Concerning the capacity building issue is, John pointed out that they've already or are going to be spinning off the Coverdell program, but they are holding on the DNA program. And in a conversation John and I had about this, he made the argument that you just heard, that he felt that there were research benefits that attended perhaps holding onto this program.

But let me sort of give, present an alternative point of view, and again, I'm not sure this is my point of view, but I think it's a legitimate argument, that, first of all, one of the operative words that John used in describing these capacity building programs is that there is a lot of money involved in them, at least, and so, that the capacity building programs may conceal the fact that is to most legislators, that the research budget at NIJ is really quite meager, that they might not see that.

And then also, just as importantly, and this I do remember from my time in working in Harrisburg, and I don't think that it's any different in Congress, is that as soon as you set foot in a legislative body, you set foot in a political institution, I mean, it's always about politics; and that the effect from a legislative point of view of keeping these capacity building programs here is that it might affect the legislative perception of the NIJ. In particular, it may create their perception that NIJ is a distributor of funds to local constituencies versus being a scientific organization. And so, the distributions of these kinds of funds can often inherently be political, and that can come back again and affect the view of NIJ as a scientific organization.

Concerning the Advisory Board function, just to sketch out the NRC recommendation, we recommended that there be 15 people on the Advisory Board, that two-thirds of them be researchers, and that they advise NIJ on policies related to grant selection and administration, research priorities, dissemination strategy, and also that they review large grants and contracts of more than $1 million.

The idea of the Advisory Board was twofold. One was to create independence from OJP, and also to provide kind of a sounding board for priority setting and performance and research dissemination, not unlike, I mean, for those of us who are in academic institutions, we often have visiting committees, and the virtue of the visiting committees, they just come and go, but they force you to think about what you're doing and why you're doing it and explain it to somebody. And so, I think that that's what we had in mind.

So again, if I were in John's shoes, it probably doesn't make sense to have this kind of Advisory Board or to press for having this kind of Advisory Board with the current Board that Al is chairing, and that is with the addition of the several people that John mentioned to deal with forensic science and some of the technology issues, and I know the technology issues that a CMU colleague of mine, you really got yourself, Al recruited some first class people, that this subcommittee will do a good job in providing the kind of advice that advisory boards do.

But I think it institutionally does not create the independence, or an argument can be made in that indeed, it just has, in fact, the opposite effect. The opposite effect in the following sense is that this is an advisory board to Laurie Robinson, the Assistant Attorney General for Justice Programs, not to John Laub, the Director of NIJ.

And this comment brings me to the broader issue of independence. I deliberately in the last comment chose to describe, name the current Assistant Attorney General and the current NIJ Director to come to, as opposed to just the roles, to make some comments about this independence issue. And first of all, let me begin by saying that there's a lot of discussion in the report about this issue of should NIJ or shouldn't NIJ stay in OJP? And I saw Jeremy Travis just walk in, and when he made a proposal that would have taken NIJ out of OJP, and I was impressed by that proposal, but ultimately, me and, more importantly, the committee, was convinced that the problem ultimately was not the location of NIJ but it was that some efforts had to be made to ensure its independence.

And so, what do I mean by the independence in ways that John just described? That is, NIJ, it's important that NIJ be able to direct and ultimately set research priorities and design solicitations; that they have control of the review process, particularly with the selection of reviewers; that John, the Director, has grant signoff authority; and that they control the dissemination process.

What are my concerns that I have? One is that as John explained to me in a phone conversation, NIJ still does not have control of all aspects of the process having to do with issues like the timing of solicitations and that an outside contractor handles the various kinds of administrative matters. And it's a little bit hard for me to judge from a distance how important those are and, well, how important those are.

But the most important concern I have and actually the most important, and this brings me to what I think is the most important thing that John said in his presentation, was the thing, the most important thing I think he said in this presentation was what he said at the very beginning, and that is that they're beginning the process of drafting legislation to create a statutory authority for the kinds of independence that John was describing.

So I began by expressing my admiration for what, genuine admiration for what John has been doing, and I would similarly express admiration for what Laurie is doing in her function, and in going back for people in this room, I similarly admire what Jeremy Travis did during his tenure as the Director and "Chips" Stewart when he was the Director. But we need, and as John pointed out, we need to move beyond effective individuals. NIJ is a small organization, and so, unlike larger organizations like NSF and NIH, it's probably the case that for the foreseeable future, NIJ will be more dependent on inspired leadership than these larger research organizations, which have their own kind of momentum to them. I'm sure, although I don't know by name, that there have been less-than-stellar NSF Directors and NIH Directors, but they have, but they're large organizations.

But it's important, I think, that the process begin of institutionalizing the kinds of things that John was talking about. And I think a necessary condition for that to happen, not a sufficient condition but a necessary condition for that to happen is that NIJ's independence within the Department of Justice in the kinds of ways that John was describing in recognizing that there are limits to what that can mean for various kinds of organizational reasons like having a Chief Counsel or not, I mean, there are a variety, is that that independence be statutorily created, and while that won't guarantee over the long haul the kind of NIJ that everybody in this room would like to see, I think it's important, it's an important step forward to making NIJ effective in a way that's independent of the strengths of the current, the people who were holding, who was the Director and the person who is the Assistant Secretary of, I mean, the Assistant Attorney General of Justice Programs.

Feucht: Q&A?

Feucht: So here's where you have a chance to participate in this very important discussion with your thoughts, reactions and more particularly, with your questions either for John or for Daniel at this point. So we'll be happy to take your questions. Yes, please?

Audience Member 1: [Inaudible]

Laub: I think we should begin the discussion with the DuPont industry; oh, I'm sorry. [Laughter] Yes, again, it was a matter of I know that we have had meetings, so I would include them as well. It's a realm that I know less about, so I go to what's comfortable and I think of academia and I think of private foundations. But again, I think that as we move forward particularly with tighter budgets, anything we can do to work together, and I think some of our work in the area of Office of Science and Technology would lend itself to potential partnerships regarding research and evaluation with industry.

Feucht: Yes, please?

Audience Member 2: [Inaudible]

Laub: Well, I'm not sure I fully understand the question. I mean, we had a long wait for our budget this year which I have to say is probably the hardest thing that I've had to deal with. In fact, I've said to several people I don't have enough brain cells to figure out how to move ahead with all the things I want to do with the uncertainty regarding the budget. And it came very late this year; people said it was the worst they've ever seen it.

So what we have to do I think is, and I would love to be able to figure this out, is to find some way to get more certainty into the budget cycle so we could plan because in the same way, I mean, Jill is at DuPont, I'm talking to Vivian at W.T. Grant Foundation, these are wonderful discussions, but ultimately, they come down to, well, what can you contribute?

And if you can't begin to put some dollar figures on that and with some certainty, I mean, I could tell you October 2^nd, you and I could begin to exchange money, but the reality is I don't think we're going to have a budget October 2^nd, and everyone is saying next year could be worse. I think that's the real challenge to how do you keep, particularly for somebody like myself that came with not quite the same change mandate as the President, but clearly, I felt there was a mandate to change, the inability to plan with certainty is a real challenge.

And we don't know what's going to happen next year, and because of our situation, we don't have any ability to carry over funds year to year. So basically, we have a short window in which to expend those funds. I wish that could be changed, but that's something that's probably not going to change in the near term, given concerns about the deficit.

So I don't really have a good answer to your question. I think this has been the hardest thing for me to deal with, as Dan knows, a 30-year academic that decided to move to government. Mark, maybe you have an answer for me? [Laughter]

Audience Member 3 : [Inaudible]

Laub: And that I think means, I think that just underscores the importance of NIJ being viewed as a strong science agency in Congress.

Feucht: Question?

Audience Member 4: [Inaudible]

Nagin: — well, I think our recommendation was that the capacity building, all the capacity building funds be —

Laub: Removed.

Nagin: — removed from NIJ to some —

Laub: Right.

Nagin: — and we didn't specify to what agency, and I was repeating the arguments for, I was responding, giving a different view to John. John argued, well, why, I'm not saying, I and the NIJ wasn't convinced of it, and I was giving, providing the argument, the counterargument for why it should be removed. That was what [inaudible].

Audience Member 5: [Inaudible]

Laub: Well, I think that with respect to the National Science Foundation, they are, first off, the discussions are right now largely around avoiding duplication so that grants are not submitted to NIJ and NSF for the same proposal, and just to make sure we're communicating. Also, the mission of NSF is a little, it's actually different from the mission of NIJ. They get things that really are more appropriate for NIJ because of our focus on practice, NIJ viewed as more basic science and so forth.

And so there aren't any substantive areas that I could point to that NIJ is engaged in discussions, although I have a strong interest in a topic that Daniel is very interested in, and that's decision-making and risk analysis, particularly by offenders, and kind of taking advantage of the whole revolution in behavioral economics. NSS has a very strong program looking at decision-making, and I could envision down the road that being the substantive area where we would partner.

It would seem to me that with respect to NIST, there are several areas that we share in terms of work on body armor, I think an interest in forensic science and so forth. So I think there is again a number of possibilities there of joint partnership. And I think what has to, my approach to this both internal to OJP and external to the Justice Department, is we have to look at these partnerships as asking the question, what's really best for the field? And I think if we come in with that kind of approach, I think that the collaborations can be authentic and will advance the science, and that's what it's really all about for me.

Nagin: Let me take a crack at the way you, what that might be approached. Over the years, I have received quite a bit of funding from NSF from the Law and Social Science Program and the Measurement and Methods and Statistics Program, and a lot but not at all of it has related to crime. I, for example, have an active NSF grant now looking at the effect of imprisoned, the experience of imprisonment on recidivism.

So I mean, there are clear overlaps between what NIJ does and what NSF does. NSF also, as a larger organization, funds networks of scholars to come together to rethink any kind of range of problems, and so, sometimes they're in the physical sciences but they can be in the social sciences as well. And so, I think that there, and then they could have to do obviously, therefore, with criminal justice matters that are directly relevant to NIJ.

And so, I do think that there are a lot, actually quite a few opportunities for collaborative funding where, with the idea being that if a project has both very much a basic science kind of dimension to it, but potentially has some more direct policy implications to it, or could benefit from having more practitioner involvement in various ways, that I do think that there actually are, there is fertile ground for connections between NSF and NIJ and also different parts of the NIHes as well.

Audience Member 6: [Inaudible]

Laub: Well, I think one thing that, I think you make a number of really important points, and what I think is vitally important is that folks at NIJ care about the independence as much as the Director does. So in a sense, what happens at NIJ has less to do with the leadership and the culture of the people who are at NIJ, and it's one of the reasons why I thought it was extremely important to involve so many people in the response to the NRC report, and not in a political way to co-opt them and get them to buy in, but it's their agency, not mine; it's their future, not mine. And so, I think that that's one way to create the culture, the true culture of science at the Institute which doesn't mean that independence will happen, but there will be more, there will be a collective voice about independence as opposed to the single voice.

I think this was a group that started after I was involved in the ASC Executive Board, but my dealings with the ASC, ACJS Criminal Justice Coalition I think is another potential force here that could be brought to bear on this issue in many ways like the ABA becomes part of a vetting process for potential judicial nominees. I would think that the American Society of Criminology, Academy of Criminal Justice Science, Criminal Justice Coalition can be active, and I also think that this is a role for the Advisory Board. That, coupled with the fixed term, which should buffer a little bit the kind of swings of the political changes.

But you raise an excellent point as to what we could do, and I think that those are the forces, at least from talking to other science agencies, the advisory board becomes a real way of establishing that the priorities don't swing with the change of the Administration.

Nagin: Let me comment on that. I mean, again, your point that law is just having a law, putting these recommendations and making it statutory is not a guarantee that any of this will happen. But I would point out, Al, that there are places in the Justice Department where social science is taken seriously, for example, in the antitrust division. And I think that that's the case because it's proven to be, they found it to be useful to, well, to the business of dealing with antitrust.

And so, another part of what John, a step in the direction of having, of insulating NIJ from changes in the problems that you're describing, is another important thing that John said was trying to come up with a way of having a cumulative research agenda which answers important questions about two things: the fair administration of justice and the prevention of crime, what the criminal justice system can do about that. The more NIJ is successful in coming up with useful answers on both of those dimensions, there are steps taken in the direction of not having NIJ be simply just at the mercy of what the current whims of the Administration are, but it's not, obviously not a guarantee.

Feucht: Yes — ?

Audience Member 7: [Inaudible]

Laub: I believe we are because what you described is actually happening for the last 11 months at NIJ. Even though we are in OJP, we've been working on a variety of areas that go beyond the state and local focus, our work in the area with response to the tribal community being one example. I have a big thick report from the Civil Rights Division upstairs in my hotel room that they're asking advice on, a scientific read on with respect to discrimination in juvenile courts in the State of Tennessee. We have worked with them with respect to New Orleans Police Department. We partnered with them on a pattern and practice roundtable. Our International Center is working with the State Department, with the Department of Homeland Security.

So again, it may be that those things could be easier done outside of OJP, but, in fact, Jeremy, they're happening now every day, and I feel that we're able to make those kinds of linkages. It's a little bit hard in the sense of trying to think about time management in this position to be involved in all those things, and perhaps this is something Jim Lynch and I have actually talked about is whether or not we need to have a research capacity that within NIJ, BJS, to do some work for the federal government, for the Department of Justice, but again, I think right now, it's the fact is it's happening. The reality is it's going on.

Feucht: All right, well, will you please join me in thanking John and Daniel?

[Applause]