This document provides detailed guidelines to Area Chairs (AC) for MICCAI 2021. Please read the instructions carefully if this is your first time as AC for MICCAI, and just as carefully if you have been on the program committee before, as there are some changes in the review process compared to previous MICCAIs. Specifically, we adopted the AC model, in which each AC has specific expertise and more responsibilities for the papers assigned to him/her. Being an AC is a demanding task that involves managing the review process for a set of papers, selecting papers for orals and awards, assessing reviewer quality, and interacting with the Program Chairs (PCs) and other ACs. You have a strong influence on the final decision, please take it seriously and be fair.
Having successfully moved to a double-blinded peer-review process at MICCAI 2017, there will be no face-to-face Program Committee meeting for MICCAI 2021. Instead there will be a teleconference before the review process begins on March 8 and March 9 to explain the process and to answer any questions related to the review process, and a second teleconference near the end of the process, where the MICCAI 2021 PCs will communicate the overall results and statistics of the review process and any arising issues to the ACs, while at the same time collecting feedback from the ACs.
The role of AC therefore continues to entail a significant commitment.
Important Dates for Area Chairs is available from this page.
ACs are expected to play an active role in ensuring high quality, timely, fair and constructive reviews. ACs also help the PCs select the best papers to be presented at MICCAI 2021. The ACs are involved in the following steps of the review process
Authors will submit their papers using Microsoft's Conference Managing Toolkit (CMT). The MICCAI Conference review process will be double-blind, i.e. the ACs and reviewers won't know the authors, and the authors won't know the reviewers or ACs. Paper assignment to ACs will be completed automatically using CMT and the Toronto Paper Matching System (TPMS), which has been successfully used for past MICCAIs. The system is complex, considering paper matching scores, subject areas, conflicts and global load balancing - but remember, it is automatic and not necessarily optimal for each individual AC. If you feel a paper has been incorrectly assigned to you, please let program-chairs@miccai2021.org know; we will re-assign the paper to another AC and you will get a replacement paper that has been manually selected. We anticipate that such reassignments will be rare. The quality of the conference depends to a large degree on the reviewers, therefore we encourage ACs to urge their network, especially postdocs and students from their group, to sign-up for reviewing.
Note that MICCAI 2021 will be running double-blind - neither reviewers nor ACs will see the author list, nor will the authors see the reviewers or ACs. Reviewers will not see the ACs names. The CMT system will provide a list of reviewers for each paper to the AC. This ordered list is generated based on subject areas (relevance) and TPMS. Based on their expertise and judgment, the AC will select a ranked list of 10-15 possible reviewers; this is needed to ensure that the bidding and matching system works smoothly. You can filter the list by subject areas and sort by relevance according to keyword matching [score 0-1.0, 1.0=best match] or TPMS rank [1 - 1400,1=best match] to help you choose reviewers. Select your reviewers and save for each paper.
While possible, it is difficult to add a reviewer who is currently not in the reviewer database because we cannot automatically identify conflicts and maintain load balancing through CMT and TPMS. If you wish to add a reviewer not in the database, you must (1) ensure that the person is willing to review, and then (2) contact submission platform manager Kitty Wong (submission_support@miccai2021.org) and we will include the reviewer manually within CMT for that paper during Phase 2. Given the overhead involved in this process, it is expected that it will be a relatively infrequent exception.
Once all ACs have selected the initial set of reviewers, a reviewer paper bidding process will start, after which the CMT system will re-optimize matching of reviewers to papers based on a large weighting placed on the AC's suggested ranked list of reviewers and the reviewers ranking of paper preferences. This final optimization will consider the TPMS score, relevance, and conflict domains from CMT while performing load balancing across all papers, reviewers and ACs. The highest scoring three reviewers on the revised list will be sent the paper for review automatically.
*New*
Reviewers will be given 3 days to accept or decline the review assignment. This is to remove unresponsive reviewers and to give reviewers the chance to decline in case they feel unable to perform a good and unbiased review. We will ask reviewers to use the "decline” option only if absolutely needed.
Each AC will be responsible for shepherding the review process for a maximum of 25 papers. The main duty of the AC during this phase is to monitor the quality of the reviews. If reviews are too brief, lack quality, or are inappropriate in some way then it is the ACs responsibility to contact the reviewer and ask them for improvements. Should the AC be unsatisfied with the quality of a review, and fail to get further feedback from the reviewer, then the AC will be able to ask (program-chairs@miccai2021.org) for additional reviewer(s)' input on the paper, beyond the original three reviewers. Any reviewer who does not provide a quality review will be identified in the reviewer database and may not be re-invited next year.
The submission platform manager will be responsible for reminding reviewers about deadlines and for monitoring whether reviewers have logged into the system. If 3 reviews have not been provided by the required date then the PCs will work in collaboration with the ACs to secure additional reviews.
After the review deadline April 20, ACs will need to provide meta reviews which summarize the key strengths and weaknesses of the paper identified by reviewers, express the opinion of the AC about the paper, and justify the ACs preliminary recommendation. If all reviewers agree on a paper and the AC follows their recommendation, the report can be simple. If there is disagreement among the reviewers, it is the ACs job to reconcile the reviews, weigh the pros and cons of the paper, and make a well-founded justification for their preliminary recommendation. If you as AC disagree with the reviewers, you can go against their recommendation, but you will need to explain why. We expect ACs to be able to make decisions based on the reviews and their own reading of the papers in most cases, but if you would like to ask for clarification from the reviewers, you can communicate with your reviewers within CMT. Detailed instructions will be posted on: https://miccai2021.org/en/INSTRUCTIONS-TO-AREA-CHAIRS.html
For papers that will be sent for rebuttal, ACs should identify any questions/points that need to be specifically addressed. Please remind authors that the purpose of the rebuttal is to provide clarification or to point out misunderstandings, it is not to promise additional experiments! Please also see section 2. Best Practices of Being an Area Chair below.
ACs should also select appropriate keywords and categories for each paper; this information will be used to help in planning poster and oral sessions.
ACs will then rank the quality of the papers in their batch and score each paper as an early accept/ rebuttal / early reject. The PCs will send out guidance on the proportion of papers to place in each category once the total number of papers submitted is known.
We will also provide some aggregate information on the distribution of reviewer scores as each AC may not have a representative sample. Although we do not want to increase the number of rebuttal papers too much, ACs are able to exercise discretion when making their decisions.
New in MICCAI 2021 is a reproducibility checklist that authors have to fill out upon submission which should be taken into account. Please be aware that authors are not required to meet all criteria on the checklist, but rather mark criteria relevant to their paper.
During this period, the PCs may contact the ACs for suggestions of suitable reviewers for papers that are missing 3 reviews. The assignment of additional reviewers will be made by the submission platform manager as it will be necessary to check for conflicts first.
New this year: Please be aware that we intend to make reviews including meta-reviews and rebuttals of accepted papers publicly available (without disclosing the reviewers/ACs names).
The rebuttal process provides a way for authors to correct possible misinterpretations in the reviewers' findings and inform the AC's final recommendation. The rebuttals will be published alongside the reviews and meta-review if the paper is accepted. Also authors of early accept pacers will be invited to write a response that will be published alongside the reviews. Once the reviews of the papers have been sent to the authors, authors have one week to submit their rebuttal/response. During this period the PCs will assign an additional 2 ACs to each paper sent for rebuttal.
Rebuttals and all reviews will be visible to the primary AC and 2 secondary ACs. The primary AC will update their meta review in light of the rebuttal and the 2 new ACs have to provide a meta review explaining their recommendation as well. All 3 ACs will then rank the papers in their batch in this stage and make a binary accept/reject recommendation, which will be used by the PCs to make the final decisions. We expect ACs to be able to make their independent recommendations based on the reviews and their own reading of the papers in most cases, but if you would like to ask for clarification from the reviewers or the other ACs assigned to a paper, you can communicate the other assigned ACs and reviewers within CMT. Detailed instructions will be posted on: https://miccai2021.org/en/INSTRUCTIONS-TO-AREA-CHAIRS.html
The PCs will provide guidance about the proportion of papers to be accepted or rejected, but it is anticipated that there will be some variation between individual ACs.
We invite and strongly encourage all ACs to participate in a teleconference, where the MICCAI 2021 PCs will communicate the overall results and statistics of the review process and present any arising issues to the ACs. In these teleconferences, ACs also will be asked to provide feedback on the overall review process and formulate recommendations on any need for adjustments or improvements. Specific details about the program and responsibilities will be announced prior to these teleconferences.
This paragraph should remind all ACs of best practices[1] of being an AC.
[1] Acknowledgements: These guidelines were inspired by CVPR 2014 guidelines: http://www.pamitc.org/cvpr14/ac_guidelines.php