Skip Navigation LinksCSR Home > About CSR > News and Publications > Peer Review Notes > Peer Review Notes September 2012

Peer Review Notes September 2012

Archives    Subscribe    Contact Us

How Well Is NIH Identifying and Advancing Innovative Research?

An image of the light bulb of innovation with the new NIH logo inside it
Dr. Sally Rockey, Director for the NIH Office of Extramural Research, recently shared her insights on advancing innovative research in a lively interview for readers of CSR’s Peer Review Notes. Her office oversees trans-NIH grant and review policies and practices.
How well does the NIH peer review system work in identifying innovative research?
Since we restructured our critiques and aligned them with our
applications, both reviewers and applicants are now more focused on “innovation,” and I believe reviewers find it a critical aspect of the job NIH asks them to do.
 
Identifying innovative research is still challenging. Everyone wants NIH research to pay off, but highly innovative research is often high-risk research. If everything you fund pays off, you’re probably not funding enough innovative research. So review panels need to take a balanced approach about enthusiasm for highly innovative vs. more incremental science, and NIH similarly needs to take a balanced approach when making funding decisions. For both reviewers and NIH, we must see that spark of innovation and get fired up about an application that might dramatically propel science forward.
 
Another tricky thing about innovation is that it often comes from a compilation of projects that together have a great impact. Any single project might seem derivative or incremental but when you put them all together they make science leap forward. So sometimes it is hard to tease out the innovation that will come.
 
Despite these challenges, I believe reviewers are thinking more about innovation and thus giving us even more input about this, which in turn increases our confidence that the research NIH supports will advance biomedical science, technology and health.
 
What do you think about data showing preliminary “approach” scores are the number one driver for overall impact scores? 
Sally Image

Reviewers may concentrate most on approach because it often will determine for them whether or not a research project is feasible -- an important consideration. Of course, we have a long history of applications with extensive approach and methodology sections, and some reviewers can find it easier to assign an overall impact score by honing in on the approach and picking it apart.

It’s important to know that the second and third factors that drive

overall impact scores are “significance” and “innovation,” and they come close together. So the emphasis on approach is probably not as dramatic as what we saw in the old days with 25-page applications. I think reviewers are looking at projects in their totality, and this is very helpful in determining the potential overall impact of the projects.
 
Have increased competition and lower paylines made it more difficult for NIH to identify and fund innovative research?
 
I know this may be the thinking out there -- that it is easy to become conservative when funding is tight -- so I believe one of the best things reviewers can do is to stay focused on our goal, which is to identify innovative and high-impact science. Without their input, it will be very difficult for NIH to make funding decisions that support innovation. We also can’t lose sight of the fact that innovative research includes basic research, and NIH continues to strongly support the fundamental sciences because they will lead to the innovative solutions down the road.
 
NIH does have difficult choices to make in deciding how to fund research. I think sustained funding for new and otherwise promising investigators will foster innovative research and keep the pipeline of talented next generation scientists strong. Often if funding for your lab is covered, you have the freedom to do even more things outside the box. This also sets up the question, does NIH fund totally as a meritocracy and continue to add more to well-funded investigators or do we award funds more broadly to a larger cadre of investigators, and how will that impact innovation? As we do this balancing act, I think we are doing the best we can given our circumstances. I believe we have elevated the conversation enough to have innovation at the forefront of everything we do.
 
Is there anything more NIH can do to improve the way it funds and advances innovative research?
 
We certainly can always do more. This is a time never before seen in our history where technological advances and our understanding of biology have come together with such a force that we can advance science more rapidly than we ever imagined.
 
We have a number of programs that target innovative research, such as the NIH Director's Transformative Research Awards program, and the Pioneer and New Innovator award programs. There are other exciting efforts underway and planned at the National Center for Advancing Translational Sciences and at other NIH Institutes and Centers.
 
Are there ways to improve the review process to advance innovative research?
 
When we looked at the data, we found that individuals who got discussed but didn’t get funded tended to do better in the long term than those who didn’t get discussed at all. One reason for this may be the feedback they get when their applications are discussed in review meetings helps them submit more competitive applications in the future. However, we can’t practically discuss every application. So we continue to look for ways to provide more constructive criticism without increasing burdens. We’d like to see scientists spend less time writing and reviewing applications and more time doing innovative, high-impact research.
 
What do you say to new applicants who tell you their mentors advised them to hold back their innovative ideas and only send NIH their most solid research applications?
 
The most important thing for a new investigator is a slam-bang, creative, innovative idea. We want a review committee to feel that not being enthusiastic about such an application would be a travesty. And young investigators are probably at the most creative points in their lives, so they should put forth their great ideas! However, new investigators shouldn’t be unrealistic and load their applications with too many objectives. They should have a manageable number of objectives and concentrate on explaining how important they are and how they can be accomplished.
 
Making their cases may be difficult, particularly if they are proposing something innovative where they don’t have a lot of experience. So I would advise new investigators to find collaborators that can bolster their application’s gravitas. And importantly, new investigators should ask others at their institutions to pre-review their applications well before submitting them to NIH. This is probably the most beneficial thing applicants can do to make sure they are conveying their ideas in the way they intend.
 
If you attended a study section meeting, what would be your final words to the reviewers on innovation?
 
Don’t lose sight of it. You need to balance the different types of applications you have in front of you. But be open to that really innovative idea that captures your fancy. You’ll know it when you see it, and then make the case to your fellow reviewers about why the application is so compelling.
 
Finally, I would say thank you. You are the ones who keep this big ship afloat. I really admire you and all NIH reviewers for what you do. You are the foundation of the good we do at NIH.
 
Learn more and share your thoughts by visiting Dr. Rockey’s blog: Rock Talk.