Introduction
Like all scholarly works based on good science, IAC reports should carry conviction and be based on facts and rigorous analyses. For this purpose, peer review by an expert group should be an integral and constructive part of the study process. Review provides a final opportunity for IAC Study Panels to test their facts, reasoning, conclusions and recommendations before release of the report to the public.
Consequently, all IAC reports must undergo an independent review by anonymous experts who were not involved in the preparation and drafting of the report. Experience shows that the revisions generated through the peer review process always improve a report considerably.
For the IAC itself, review provides the mechanism by which the imprimatur of the IAC Board is placed on the report.
Purpose
The purpose of review is to assist the Study Panel in making its report as accurate and effective as possible. Review not only fulfills the IAC's institutional obligation to exercise oversight, but it also greatly aids the Panel by providing preliminary reactions from a diverse group of experts or stakeholders that is selected to resemble the final audiences of the report.
Reviewers are asked to consider whether in their judgment the evidence and arguments presented are sound and the report is fully responsive to the study charge. They are not being asked whether they concur with the findings.
Process
Reviewers are appointed by the IAC Co-Chairs after consultation with the IAC Board. The IAC Co-Chairs also appoint two review monitors to oversee the review process. A draft report is sent to reviewers only after the Study Panel has indicated that it is satisfied with its form and content.
Reviewers receive the complete report, along with the charge given to the Study Panel and these Review Guidelines, including the Annex. Reviewers are asked to provide written comments on any and all aspects of the draft report, but to pay particular attention to the review questions set forth in the Annex. The Study Panel is expected to consider all review comments and to provide written responses, which are evaluated by the review monitors.
When the review monitors are satisfied that the Panel has adequately dealt with the issues raised by the reviewers, the IAC Co-Chairs decide that the report is ready for publication. The IAC Co-Chairs then inform the IAC Board whether, in their judgment, the study and review have been conducted satisfactorily in accordance with the IAC Rules of Procedure. If after several reiterations, the review monitors and the Study Panel become deadlocked on one or more particular issues raised by reviewers, the matter is referred to the IAC Co-Chairs for resolution. The IAC Co-Chairs have, in all cases, final authority as to whether to recommend publication.
All IAC study reports are to be made public. However, the IAC Board will decide on the precise mechanisms of dissemination for each report.
Confidentiality
To encourage reviewers to express their views freely, review comments are treated as confidential documents and are given to the Study Panel with identifiers removed. After submitting their comments, reviewers are asked to return or destroy the draft report and to refrain from disclosing their comments or the contents of the draft.
The names and affiliations of reviewers and the review monitors will be made public in the report when it is released, along with a disclaimer that makes clear their role in the process.
Annex
Although criteria may differ from report to report, the following questions may be useful to reviewers in formulating their comments.
1. Is the charge clearly described in the report? Are all aspects of the charge fully addressed? Do the authors go beyond their charge or their expertise?
2. Are the conclusions and recommendations adequately supported by evidence, analysis, and argument? Are uncertainties or incompleteness in the evidence explicitly recognized? If any recommendations are based on value judgments or the collective opinions of the authors, is this acknowledged and are adequate reasons given for reaching those judgments?
3. Are the data and analyses handled competently? Are statistical methods applied appropriately?
4. Are sensitive policy issues treated with proper care? For example, if the report contains recommendations pertaining to the functioning of an organization or program, are alternative options considered?
5. Are the report's exposition and organization effective? Has the report been clearly and effectively written for the target audiences? Is the title appropriate?
6. Is the report fair? Is its tone impartial and devoid of special pleading?
7. Does the executive summary concisely and accurately describe the key findings and recommendations? Is it consistent with other sections of the report?
8. What other significant improvements, if any, might be made in the report?
In providing comments, reviewers are encouraged to distinguish issues they consider to be of major concern from other, less significant points.