Public FAQs
		NIH Staff FAQs

Related NIH Staff FAQs

On This Page:

A. Summary of Major Changes B. Reduced Page Limits C. Application Writing Tips D. Resubmissions E. Review Criteria and Scoring F. Summary Statements G. New Investigators (NIs) and Early Stage Investigators (ESIs) H. Preparing and Submitting Critiques I. Review Meeting Processes

---

  A. Summary of Major Changes

1. When did application changes go into effect?
2. What were the major changes to applications?
3. Why were applications restructured?
4. Did the changes apply to all applications, whether they were to be submitted on paper or electronically?
5. How did applicants know which application package to use?
6. What were the changes to T, K, and F applications?
7. What were the changes for the Personal Statement of the Biosketch?
8. When did the transition to the new application forms and instructions take place for applicants eligible for continuous submission?

  B. Reduced Page Limits

1. What were the shortened page limits?
2. What was the reason for reducing page limits?
3. How did the NIH make the decision to shorten applications?
4. Were there any changes to amount or type of materials allowed by the Post-Submission Materials Policy?
5. Could an applicant bypass the page limits for each section; for example, was the applicant able to start on one page for Specific Aims, and then continue writing the Specific Aims on the first page of the Research Strategy section?
6. Did the new page limits apply to multi-component applications?
7. Were there any sections of the application that did not receive a page limit?
8. How did reviewers handle application materials that were included in sections without page limits?
9. Did other agencies change their page limits to match NIH?

  C. Application Writing Tips

1. Was grant-writing advice provided to applicants to cope with the application changes?
2. Was there any advice about font size and styles?
3. Were applicants advised on how to organize the Research Strategy section to address the Significance, Innovation, and Approach for each Specific Aim, assuming they had more than one Specific Aim?
4. Was there any change to the electronic upload process for the research plan attachments?

  D. Resubmissions

1. What was the rationale for changing the Resubmission Policy?
2. What were the changes in the Resubmission Policy, and what were the implementation dates?
3. When was the last date to submit A2 applications?

  E. Review Criteria and Scoring

1. How did review criteria change?
2. Did the scoring scale used by NIH study sections change?
3. What else changed about the scoring system?
4. What about the scoring system stayed the same?
5. What was the rationale for the scoring change?
6. Is the new 1 - 9 scale a stanine system?
7. Why are individual review criteria being scored?
8. What if significance is high but feasibility is low? Or questionable?
9. Do all applications receive criterion scores?
10. For multi-project applications, are there criterion scores for all core review criteria? Cores previously could be given non-numerical scores, i.e., acceptable or unacceptable. Did that change?
11. Are individual reviewer criterion scores open to discussion by the entire panel?
12. What was the rationale for not requiring reviewers to enter individual criterion scores?
13. How is the overall impact/priority score that appears on the face page of the summary statement and in the applicant’s Commons account calculated?
14. Are all scores rounded up? Doesn’t that make it extremely unlikely to receive a 10 (i.e. only unanimous 1s )? Doesn’t that make the range 20-90, not 10-90?
15. Why was there a need for rounding? Wouldn’t having more digits help enable discrimination between applications?
16. Why is the percentile rounded up, while the scores are rounded traditionally?
17. Does this new scoring system result in more tie scores?
18. Is "not recommended for further consideration" allowed if there is a serious human or animal subjects issue?
19. Do the U mechanisms fall into the new scoring system?
20. What does my score mean?

  F. Summary Statements

1. What stayed the same for summary statements?
2. How did summary statements change?
3. How are the criterion scores displayed in the summary statement?
4. What if the scores in the table do not agree with scores that may have been entered with the written critique?

  G. New Investigators (NIs) and Early Stage Investigators (ESIs)

1. What is the implication/significance of ESI versus NI? Since ESI is a subgroup of NI, what is the purpose of ESI?
2. Who has the ability in the electronic system to update NI or ESI status?
3. Is verification of degree completion date required and accomplished via a third party?
4. In terms of qualification for the ESI designation, why are postdoctoral periods treated differently than medical residency periods, especially since residencies prepare for clinical research?
5. Can individuals that are within 10 years of completing a mentored career development award be considered ESIs?
6. Does ESI status extend beyond 10 years if the A1 is submitted after the 10-year period, but the A0 was submitted before 10 years have passed?
7. What constitutes a "terminal research degree"?
8. How do the following situations affect NI status:
9. Are applications from New/Early Stage Investigators percentiled with other applications?

  H. Preparing and Submitting Critiques

1. What is IAR? What does it stand for?
2. Do reviewers fill in their critique templates on-line in IAR?
3. Are the templates in Microsoft Word 2003 or 2007?
4. Why don’t the hyperlinks work in Microsoft Word 2003?
5. Is there a specific definition of impact?
6. Where on the template are scores entered?
7. It is nice to have example documents that model comments for strengths and weaknesses. Was there a revision to the original guidance that limited strength and weakness comments to ¼ page in length?
8. How do you reconcile the shorter critique with the increased reliance on critique information for breaking ties and making award decisions?
9. Where do reviewers comment on multiple PD/PI Leadership Plan?
10. Is there a short “overall evaluation” critique template that a discussant can fill out in addition to adding numerical scores for the individual criterion?
11. Is there a short “overall evaluation” critique template that a discussant can fill out in addition to adding numerical scores for the individual criterion?

  I. Review Meeting Processes

1. Could you please explain how "resubmission, renewal and revision" are review criteria?
2. How do you put in criterion scores for an application with more than 5 scored review criteria?
3. Is there an error message if reviewers try to enter scores without uploading their critiques?
4. Do reviewers have to enter individual criterion scores even if they do not enter text for a criterion?
5. Criterion scores cannot be entered into IAR without critiques but can the preliminary score be entered without a critique?
6. Does a reviewer have to re-upload their critiques if they revise their score?
7. Are reviewers still restricted from resubmitting, for example making corrections, until the final edit phase in IAR?
8. Can discussants read the critiques without posting any critique or score on IAR during the read phase?
9. Can reviewers nominate lower half applications for discussion?
10. How can one reconcile the previous admonitions to not compare applications during review with the current clustering system which appears to encourage it?
11. For which mechanisms should ESI/NIs be clustered?
12. Do ALL review groups have teleconferencing available?
13. To what extent did the SROs provide training at review meetings?

Back to Top

---

  A. Summary of Major Changes

1. When did application changes go into effect?

   All competing applications submitted for due dates on or after January 25, 2010 required use of the new forms and compliance with the shorter page limits.

2. What were the major changes to applications?

   Application forms and formats were revised mainly in three sections: Research Plan, Resources, and Biographical Sketch. In addition, many page limits changed. For a complete list of the changes (which applied to all competing applications), refer to the Details of Application Changes,  watch the short video overview on the changes, or visit the Shortened/Restructured Applications page on this website.

3. Why were applications restructured?

   The new application format aligned the structure and content of the application with review criteria. This change was designed to help ensure that reviewer and applicant expectations would coincide, for a more efficient and transparent application process.

4. Did the changes apply to all applications, whether they were to be submitted on paper or electronically?

   Yes. Changes were made to both electronic and paper application packages. Details were provided in each funding opportunity announcement (FOA), as well as in the updated application instructions.

5. How did applicants know which application package to use?

   For applications intended for due dates on or after January 25, 2010, applicants were instructed to choose the ADOBE-FORMS-B application package (for the electronic SF424 (R&R) application) and the application package with the “06/2009” revision date (for the paper-based PHS398 application).

6. What were the changes to T, K, and F applications?

   Application changes to the Biosketch and the Resources section applied to Training (T), Career Development (K), and Fellowship (F) applications, and application changes (including page limit changes) to the Research Strategy section applied to K and F applications. In addition, the January 25, 2010 due date marked the transition of T applications from paper-based (PHS398) to electronic [SF424 (R&R)] format.

7. What were the changes for the Personal Statement of the Biosketch?

   A Personal Statement was a new expectation for all personnel submitting a Biographical Sketch (including senior key personnel), except participating faculty on institutional training grants.

8. When did the transition to the new application forms and instructions take place for applicants eligible for continuous submission?

   Applicants eligible for continuous submission who were submitting R01, R21, and R34 AIDS applications continued using the old SF 424 (R&R) ADOBE-FORMS-A through February 7, 2010, and used the new SF 424 (R&R) ADOBE-FORMS-B thereafter.
   
   Applicants eligible for continuous submission who were submitting R01, R21, and R24 non-AIDS applications continued using the old SF 424 (R&R) ADOBE-FORMS-A through January 25, 2010, and the new SF 424 (R&R) ADOBE-FORMS B thereafter.

Back to Top

---

  B. Reduced Page Limits

1. What were the shortened page limits?

   The page limits were shortened for most types of applications, and generally were reduced to 6, 12, or 30 pages for the Research Strategy section, with one additional page for Specific Aims, and one additional page for introductions to resubmissions and revisions.  The table of page limits contains a historical record of all the page limits that were effective for due dates on or after January 25, 2010.

2. What was the reason for reducing page limits?

   The main goal of shorter applications was to help focus both applicants and reviewers on the essentials of the science rather than the administration of applying. Other goals were to avoid information overload, and potentially enable a larger number of reviewers to read each application and participate in the review in a more informed manner.

3. How did the NIH make the decision to shorten applications?

   The NIH grant application is among the longest that is used by funding agencies and foundations worldwide.  During the diagnostic phase of the peer review self-study, the overwhelming opinion from the working groups was that a significant reduction in page numbers was necessary to achieve the desired enhancements, to a page limit somewhere between 7 and 15 pages. 
   
   The working groups considered the response to an NIH-issued RFI, which showed that 70% of all respondents preferred that applications be shorter than 25 pages. The majority of the respondents did not believe that a shorter application would compromise their ability to present scientific ideas (68% responded that ideas could be communicated equally well comparing 25 to 15 pages, and 19% indicated that shorter applications would enhance communication).  Respondents also indicated that a shortened application would take either the same (27 percent) or less (50 percent) time to prepare. Thus, for over three quarters of those responding, shorter applications would not be more burdensome.

4. Were there any changes to amount or type of materials allowed by the Post-Submission Materials Policy?

   The NIH Policy on Post-Submission Application Materials (NOT-OD-10-115) was revised for applications submitted for the September 25, 2010 due date and thereafter.  Please visit the Peer Review Policies and Practices FAQs for updated information on this policy.

5. Could an applicant bypass the page limits for each section; for example, was the applicant able to start on one page for Specific Aims, and then continue writing the Specific Aims on the first page of the Research Strategy section?

   No, it was made clear that content could not be “blended” or continued from one section to another.  In addition, electronic page limit validations occur separately for each application section.

6. Did the new page limits apply to multi-component applications?

   Yes. As for single component applications, the page limit for the research strategy for cores and projects of multi-component applications was reduced to 6 or 12 pages, in most cases. Any exceptions were detailed in the FOA.

7. Were there any sections of the application that did not receive a page limit?

   Some sections of the applications, such as Protection of Human Subjects, did not receive a page limit.  Sections with page limits were listed in the Table of Page Limits.  Applicants were advised not to circumvent page limit restrictions by using the sections without page limits to add extra information from a page-limited section.

8. How did reviewers handle application materials that were included in sections without page limits?

   Reviewers were advised that they need not consider any excess text that is inappropriately included in a section of the application that has no page limits. In addition, reviewers were advised to note in their critiques the inclusion of excess text, so that it could become part of the Summary Statement.

9. Did other agencies change their page limits to match NIH?

   The decision to follow NIH’s page limits was made on a case by case by each agency, and provided in the FOA.

Back to Top

---

  C. Application Writing Tips

1. Was grant-writing advice provided to applicants to cope with the application changes?

   Yes, applicants were instructed to pay attention to any special features of the FOA, to read and follow the application instructions carefully, and to address the review criteria.

2. Was there any advice about font size and styles?

   Applicants were reminded to follow the application instructions, which specified “Use an Arial, Helvetica, Palatino Linotype, or Georgia typeface, a black font color, and a font size of 11 points or larger. (A Symbol font may be used to insert Greek letters or special characters; the font size requirement still applies.)”

3. Were applicants advised on how to organize the Research Strategy section to address the Significance, Innovation, and Approach for each Specific Aim, assuming they had more than one Specific Aim?

   Applicants were advised to use their discretion in organizing the information to best convey the desired information to the reviewers.  For example, if an application had two specific aims, an applicant might choose to organize the Research Strategy in one of the following two ways:

   Significance:  Specific Aim 1, Specific Aim 2 
   Innovation:  Specific Aim 1, Specific Aim 2 
   Approach:  Specific Aim 1, Specific Aim 2

   OR:

   Specific Aim 1:  Significance, Innovation, Approach
   Specific Aim 2:  Significance Innovation, Approach

4. Was there any change to the electronic upload process for the research plan attachments?

   Yes, there was a reduction in the number of distinct required uploads: three previous uploads (Background and Significance, Preliminary Studies/Progress Report, Research Designs and Methods) were combined into one upload for the Research Strategy. Applicants were advised to use headings for the Research Strategy sections of Significance, Innovation, and Approach, as well as Preliminary Studies for new applications and Progress Report for renewal and revision applications, as applicable.

Back to Top

---

  D. Resubmissions

1. What was the rationale for changing the Resubmission Policy?

   During the past several years, NIH paylines have been very low, and a large number of excellent applications have had to be revised and resubmitted, even though the underlying science did not change significantly. In effect, excellent investigators with great projects had to wait in line. To try to address this problem and fund highly meritorious science, the number of allowed resubmissions was decreased so that the success rate of new and first resubmission applications would increase. The changes to the resubmission policy were described in detail in NIH Guide Notice NOT-OD-07-015 and NIH Guide Notice NOT-OD-09-016.

2. What were the changes in the Resubmission Policy, and what were the implementation dates?

   Previously, an application could be resubmitted twice. The changed policy, which went into effect for applications intended for due dates of January 25, 2009 and later, allowed for applications to be resubmitted only once. By reducing the number of resubmissions, the goal was to permit funding of larger numbers of new and first resubmission applications and thereby to allow investigators to spend less time revising and resubmitting and to start doing their research projects sooner.

3. When was the last date to submit A2 applications?

   Valid A2 applications were allowed until the January 7, 2011 due date, if the A0 application was submitted for the January 25, 2009 due date or an earlier due date.

Back to Top

---

  E. Review Criteria and Scoring

1. How did review criteria change?

   For research grants and cooperative agreements, the scored review criteria remained Significance, Investigator, Innovation, Approach, and Environment. In addition, the “Overall Impact” was added for all types of funding mechanisms. For more details on Overall Impact, and the difference between Overall Impact and Significance, visit the Peer Review Policies and Practices FAQs.

2. Did the scoring scale used by NIH study sections change?

   Yes. The scoring system was changed to use a 9-point scale, with a score of 1 indicating an exceptionally strong application with essentially no weaknesses. For more information on the Scoring System, see the Scoring System and Procedure document, or visit the Peer Review Policies and Practices FAQs.

3. What else changed about the scoring system?

   An important change was that all applications – whether they have been discussed in the review meeting or not – will receive criterion scores.

4. What about the scoring system stayed the same?

   The scoring change did not affect the practice of using preliminary scores to help determine which applications will be discussed at a review meeting.

   The best score remained "1".

   The non-numerical scoring options remained the same: 

   o CF: Conflict 
   o NP: Not Present 
   o DF: Deferred 
   o ND: Not Discussed (terminology change, formerly Unscored) 
   o AB: Abstention 
   o NRFC: Not Recommended for Further Consideration

5. What was the rationale for the scoring change?

   The major reason for changing the scoring system was to create a rating scale that realistically reflected the range of quality of applications that most study sections actually see and, thereby, to encourage routine use of the entire rating scale. Behavioral studies have noted that individuals can only distinguish between 7 and 11 categories. At least two different studies of the peer review system have noted that the previous 41-point scale is not in alignment with human abilities to categorize. The new system attempted to remedy that by using the 1 to 9 scale not only for overall impact/priority scores, but also for individual review criterion scores.

   The change was designed with the goal of giving applicants and NIH staff an easier time understanding the basis of the overall score that an application received by giving explicit ratings to each of the five scored review criteria. The goal is to provide more information about the strengths and weaknesses of each application and so each reviewer’s criterion scores will be reported on the summary statement.

6. Is the new 1 - 9 scale a stanine system?

   No. Stanines impose a specific distribution of scores based on pre-determined standard deviations from the mean. The NIH scoring scale does not impose such a distribution. Percentiling is used to achieve comparability of scores across review groups (see section on Percentiling  for more details).

7. Why are individual review criteria being scored?

   By giving explicit ratings to each of these scored criteria, applicants and NIH staff making funding decisions will have an easier time understanding the basis of the overall score that an application received. The goal is to provide more information about the strengths and weaknesses of each application and so each reviewer’s criterion scores are reported on the summary statement. Having reviewers add a numerical score to each of the scored criteria is intended to aid in interpretation of their written comments. For example, a reviewer may have a number of criticisms, but these may be considered minor and easily fixed. The words alone might not indicate this intent, but assigning a good score would provide additional clarity. Likewise, one major weakness could outweigh several areas of strengths and a score in the poorer range would indicate that this weakness was considered major. The reasoning is that providing more and clearer information to the applicant, NIH staff, and advisory councils will permit them to make better-informed decisions.

8. What if significance is high but feasibility is low? Or questionable?

   If it is the opinion of the reviewers that it is not likely that the project will answer the questions it poses, then overall impact is likely to be low. The degree of uncertainty about feasibility will likely determine whether this is rated as a minor, moderate or major weakness.

9. Do all applications receive criterion scores?

   All applications – whether they have been discussed in the review meeting or not –receive criterion scores. Reviewers use the criterion scores to help them determine the overall impact or priority score, but the criterion scores are not be weighted explicitly. It is up to each reviewer to determine an overall score that best describes the likely overall impact that each application will have. Such determinations reflect each reviewer’s best estimate of how much impact the application will have, given the importance of the questions being asked and the likelihood that the project will succeed, given the combination of investigators, approach and environment that are described in the application.

10. For multi-project applications, are there criterion scores for all core review criteria? Cores previously could be given non-numerical scores, i.e., acceptable or unacceptable. Did that change?

    Please see the guidelines for P01 documents at http://grants.nih.gov/grants/peer/reviewer_guidelines.htm.

11. Are individual reviewer criterion scores open to discussion by the entire panel?

    The review panel does NOT vote on each criterion. However, during the discussion of an application, review committee members could choose to discuss individual criterion scores to make sure they understand the points raised by an assigned reviewer. It is important to understand that criterion scores are intended to be pieces of information to help understand each reviewer’s evaluation, but are not intended to be the sole basis of how to decide on an overall impact/priority score.

12. What was the rationale for not requiring reviewers to enter individual criterion scores?

    In some cases, an SRO may wish to bring in a reviewer who is a content expert on a particular part of an application. For example, a statistician might be used as a reviewer for an application that relies heavily on the use of statistics to analyze a biomedical issue. In this case, it would be inappropriate for the statistician to score parts of the application for which he/she does not have expertise.

13. How is the overall impact/priority score that appears on the face page of the summary statement and in the applicant’s Commons account calculated?

    To calculate the average of the final overall impact/priority scores given by all eligible SRG members, the average is multiplied by ten and rounded to the nearest whole number (10 – 10.49 = 10; 10.5 – 11.49 = 11; etc.).

14. Are all scores rounded up? Doesn’t that make it extremely unlikely to receive a 10 (i.e. only unanimous 1s )? Doesn’t that make the range 20-90, not 10-90?

    Only the percentiles are rounded up. The overall impact/priority scores are rounded traditionally, making the range 10-90. Getting a score of 10 would not necessarily require total unanimity. For example, if 24 reviewers scored 1 and one reviewer scored 2, the average would be 26/25 = 1.04. Multiplication by 10 yields 10.4, which would be rounded to 10 rather than 11.

15. Why was there a need for rounding? Wouldn’t having more digits help enable discrimination between applications?

    The scores are rounded to prevent the appearance of a precision that does not exist. Tie scores indicate that the tied applications cannot be reliably distinguished from each other based solely on scientific and technical merit.

16. Why is the percentile rounded up, while the scores are rounded traditionally?

    Percentiles are rounded up because we do not use a zero percentile, and the desire was to have a whole number percentile. So percentiles from 0.1 to 0.9 are rounded to 1.

17. Does this new scoring system result in more tie scores?

    With only nine possible rating options, the distribution of mean scores was expected to result in more tie scores, which essentially indicate that the scale is reliable since identical mean scores occur when there is reviewer agreement on the assigned score. Tie scores indicate that the tied applications cannot be reliably distinguished from each other based solely on scientific and technical merit. As is true with all funding decisions, Advisory Councils and NIH must weigh additional factors in the decision process, which include the need to factor in mission relevance; portfolio balance; specified Congressional appropriation requirements; and other relevant factors.

18. Is "not recommended for further consideration" allowed if there is a serious human or animal subjects issue?

    YES.

19. Do the U mechanisms fall into the new scoring system?

    Yes, all mechanisms (with only a few exceptions) use the new scoring system.

20. What does my score mean?

    For help interpreting new scores please read “Need Help Interpreting the New Review Scores?”

Back to Top

---

  F. Summary Statements

1. What stayed the same for summary statements?

   The face page and first-order organization of the summary statement remained the same; Resume and Summary of Discussion for discussed applications; for all applications, the typical summary statement sections include the applicant abstract and public health relevancy statement; reviewer critiques, final resume section to summarize the reviewers’ recommendations; roster. The summary statement contains reviewer critiques presented as Critique 1 – Critique n, ordered by assigned reviewer. Only discussed applications will receive an overall impact/priority score.

2. How did summary statements change?

   The order of the review criteria has changed: Investigator(s) and Approach swapped positions; the new order (for research applications) is Significance, Investigator(s), Innovation, Approach, and Environment. A table at the beginning of each critique summarizes the reviewer’s scores for each criterion.

3. How are the criterion scores displayed in the summary statement?

   Criterion scores are added automatically by the Internet Assisted Review (IAR) system as a table at the beginning of each reviewer’s critique.

4. What if the scores in the table do not agree with scores that may have been entered with the written critique?

   The scores that IAR inserts in the table are accepted as final. Reviewers are instructed NOT to enter scores with their critiques and that errant scores in the critiques may be removed in finalizing the summary statement.

Back to Top

---

  G. New Investigators (NIs) and Early Stage Investigators (ESIs)

1. What is the implication/significance of ESI versus NI? Since ESI is a subgroup of NI, what is the purpose of ESI?

   A New Investigator (NI) is an NIH research grant applicant who has not yet competed successfully for a substantial, competing NIH research grant. An Early Stage Investigator (ESI) is a new investigator who has completed his or her terminal research degree or medical residency—whichever date is later—within the past 10 years. The ESI designation helps to differentiate between established and early-career investigators, and helps NIH meet the goal of accelerating the transition to an independent scientific career. It is expected that ESIs will constitute the majority of funded NIs.

2. Who has the ability in the electronic system to update NI or ESI status?

   NI and ESI status is determined automatically by the functionality built into eRA Commons, based on the investigator’s record of receiving NIH grants and the date of their terminal degree and/or completion of medical residency. If an extension to the ESI 10-year period is granted by the ESI Extensions Committee, the eRA Commons account is updated by eRA staff.

3. Is verification of degree completion date required and accomplished via a third party?

   Degree and medical residency completion dates are provided by the applicant. Applicants are expected to provide true, accurate, and complete information and to produce documentation when requested. (Making false statements to the federal government can result in severe penalties).

4. In terms of qualification for the ESI designation, why are postdoctoral periods treated differently than medical residency periods, especially since residencies prepare for clinical research?

   The 10-year ESI period was designed to allow for a consistent time period of research activities between applicants. While postdoctoral periods generally are devoted to research and research training activities, medical residency periods generally are not.

5. Can individuals that are within 10 years of completing a mentored career development award be considered ESIs?

   Individuals that are within 10 years of completing a mentored career development award are only considered ESIs if they are also NIs who are within 10 years of completing their terminal research degree or medical residency.

6. Does ESI status extend beyond 10 years if the A1 is submitted after the 10-year period, but the A0 was submitted before 10 years have passed?

   If an original application is flagged as an ESI application and it is revised and resubmitted as an amended application (A1) within 13 months after the submission date of the original application, it will retain its ESI status even if the eligibility period has expired. If the amended application (A1) is resubmitted more than 13 months after the submission date of the original application, the ESI status will be recalculated based on the submission date of the revised application.

7. What constitutes a "terminal research degree"?

   A terminal research degree is the most recent research degree received by an investigator. In most cases, the terminal research degree will be a Ph.D., M.D., or equivalent. Individuals without a doctoral degree should enter their highest research degree, which could be a Masters or a Bachelors degree in some cases. An M.B.A. or an M.F.A. and similar degrees normally are not considered research degrees. However, some types of research Master’s degrees (like M.S. or M.P.H) can be considered terminal if they mark the beginning of a period of research or research training, or the new degree substantially expands or advances an individual’s skills within a field related to the NIH mission. For example, an M.P.H., or a comparable research degree earned after the medical degree, may initiate a period of research and should be listed as the terminal research degree. The conferral date for the terminal research degree marks the beginning of the ESI status period.

8. How do the following situations affect NI status:

   • PI of an NIH contract? Does not disqualify NI status.
   
   • PI of a grant with another Federal agency? Does not disqualify NI status.
   
   • PI of an SBIR/STTR? Does not disqualify NI status. 
   
   • PI of a U01, specifically for a foreign investigator? Receipt of U01 removes NI status. 
   
   • Inheriting an R01 from a PI who moved away or died? Does not disqualify NI status. 
   
   For a complete list of NIH grants that do not disqualify a PD/PI from being considered a New Investigator, seehttp://grants.nih.gov/grants/new_investigators/index.htm.

9. Are applications from New/Early Stage Investigators percentiled with other applications?

   Percentile bases are determined for study sections, and there is not a separate percentile base for ESIs, NIs, or established investigators. There is also a CSR-all base that is used for percentiling applications reviewed in Special Emphasis Panels (SEPs), or in some instances, Institutes/Centers may have their own base for percentiling SEPs reviewed in their respective Institute/Center.

Back to Top

---

  H. Preparing and Submitting Critiques

1. What is IAR? What does it stand for?

   Internet Assisted Review (IAR) is a web-based system that has been used to manage the electronic submission process of reviewer critiques for several years.

2. Do reviewers fill in their critique templates on-line in IAR?

   No, reviewers download a Word file that contains the critique template for the mechanism they are reviewing. They fill out the template in Word, and then upload the entire Word file into IAR.

3. Are the templates in Microsoft Word 2003 or 2007?

   The templates are compatible with both formats. They are also compatible with OpenOffice, an open-source suite of office software.

4. Why don’t the hyperlinks work in Microsoft Word 2003?

   The hyperlinks that are associated with each review criterion only work with Microsoft Word 2007. These hyperlinks all point to the same Web site, but bring the user to different “anchor” points on the page that correspond to a particular review criterion or consideration. For users with Microsoft 2003, the hyperlinks associated with each review criterion do not work, but an accessible hyperlink is provided at the top of the template that links to the same Web site, so that the same information is available to the end user.

5. Is there a specific definition of impact?

   The definition of impact is slightly different for different types of grant mechanisms. Please see the definition which is provided for each type of critique, via the link on the critique template.

6. Where on the template are scores entered?

   Scores are not entered on the critique template; rather, they are entered directly into IAR.

7. It is nice to have example documents that model comments for strengths and weaknesses. Was there a revision to the original guidance that limited strength and weakness comments to ¼ page in length?

   Although it was recommended that no more than ¼ page of text accompany the bullets for any particular criterion, no hard limits were imposed. The responses posted on the critique example are of varying length, depending on the particular criterion or consideration. If there are a lot of strengths and weaknesses that have an impact on the overall score, then they should be listed. It is the explanatory prose that reviewers are asked to limit. However, some points can in fact take a considerable amount of text to explain, which is why no absolute limits are imposed. Minor weaknesses that do not contribute to the overall score (and whose fixing might therefore NOT be expected to result in an improved impact score) can be listed in the “Advice to Applicant” box if the reviewer thinks that these may be helpful to the applicant.

8. How do you reconcile the shorter critique with the increased reliance on critique information for breaking ties and making award decisions?

   The new critiques convey the same or more evaluative information as they have in the past. One goal of changing the critique template was to remove needless repetition and restating of what is in the application.

9. Where do reviewers comment on multiple PD/PI Leadership Plan?

   Multiple PD/PIs is included in the enhanced investigators criterion, so it is commented on under the investigator(s) criterion on the critique template.

10. Is there a short “overall evaluation” critique template that a discussant can fill out in addition to adding numerical scores for the individual criterion?

    No, but discussants can use the critique template for the mechanism they are reviewing and only fill out the “Overall Impact” section of the template.

11. Is there a short “overall evaluation” critique template that a discussant can fill out in addition to adding numerical scores for the individual criterion?

    The additional comments box provides an opportunity to offer advice, to make helpful suggestions, or to point out minor issues that did not contribute to the overall score. Actual strengths or weaknesses that contributed to the impact score should be included under the appropriate criterion.

Back to Top

---

  I. Review Meeting Processes

1. Could you please explain how "resubmission, renewal and revision" are review criteria?

   Each resubmission, renewal or revision should be evaluated on criteria that go beyond that of a new grant application. See the enhanced review criteria for specifics on each of these criteria.

2. How do you put in criterion scores for an application with more than 5 scored review criteria?

   You cannot put more than 5 criterion scores into IAR. Any additional scored criteria has to be added into the text of the critique and added manually on the summary statement.

3. Is there an error message if reviewers try to enter scores without uploading their critiques?

   Yes.

4. Do reviewers have to enter individual criterion scores even if they do not enter text for a criterion?

   No, entering criterion scores is not mandatory. Text-based evaluations of the criteria are entered into a word document with sections for each review criterion and consideration. The reviewer is free to leave sections of the critique template empty if they do not apply for his/her evaluation.

5. Criterion scores cannot be entered into IAR without critiques but can the preliminary score be entered without a critique?

   No. To enter any score a critique must be entered, with the exception of some mail and unassigned reviewers who do not need to enter a critique.

6. Does a reviewer have to re-upload their critiques if they revise their score?

   No. However, if they wish to change any of their written critique to go along with their revised score, they should upload a revised critique.

7. Are reviewers still restricted from resubmitting, for example making corrections, until the final edit phase in IAR?

   The previous rules apply.

8. Can discussants read the critiques without posting any critique or score on IAR during the read phase?

   The same rules apply. If you are a discussant or other assigned reviewer and you have not posted your scores/critique, you will not be able to read the critiques of others.

9. Can reviewers nominate lower half applications for discussion?

   Yes.

10. How can one reconcile the previous admonitions to not compare applications during review with the current clustering system which appears to encourage it?

    The intent is to facilitate the discussion so that applications that are under the same types of review criteria and emphasis are considered together (NIH NI/ESI Policies). This is not meant as a means to compare applications; applications are reviewed not against each other but according to published review criteria. The intent of clustering is to help reviewers’ frame their discussions to be consistent with the emphases placed on the different categories of research.

11. For which mechanisms should ESI/NIs be clustered?

    Only R01 applications from ESI/NIs will be clustered for review.

12. Do ALL review groups have teleconferencing available?

    All SROs have telephones available at the review meeting, with rare exceptions.

13. To what extent did the SROs provide training at review meetings?

    In preparation for the May/June round (2009 and 2010), SROs were provided formal training and communications and training materials so that they could inform and train reviewers.