The 2021 MICCAI conference will be held in Strasbourg, France. The conference will host a variety of workshops, challenges, and tutorials. Challenges have become an integral part of MICCAI conferences in the past years. Their aim is to provide a fair and direct comparison of different methodological solutions to a common problem. Challenges should address a well-defined open problem relevant to the MICCAI, provide high-quality data for testing/training algorithms, and define a clear assessment procedure. Examples of topics from previous challenges include, but are not limited to, segmentation, registration, computer-aided detection and diagnosis, endoscopic vision, and brain connectivity. Proposals related to accessible, fair and responsible healthcare are particularly welcome.
The sessions may be designed as oral/poster presentations of the results that each participant achieved prior to the conference and/or hands-on competitions during the event. The MICCAI supports open data; hence, we encourage the organizers to use open-source data licenses and/or keep the challenge open to submissions beyond the conference, providing a sustained platform for benchmarking.
Challenges will take place on September 27th and October 1st 2021. All events will be located at the main conference venue. Each challenge may be organized as a half-day (8:00am-12:30pm or 1:30pm-5:00pm) or a full-day (8:00am-5:00pm) meeting. The reserved rooms that are available will be listed on the MICCAI 2021 website.
According to a recent study on common practice in biomedical image analysis [1], reproduction and adequate interpretation of challenge results are often not possible, because only a fraction of relevant information is typically reported. To address this issue, we have implemented an online platform [2] for challenge proposal submissions with structured descriptions of challenge protocols based on [3]. The parameters for challenge design have been developed in collaboration with multiple institutions worldwide.
This framework will ensure the completeness of the information for a high-quality challenge review as well as structured access to the challenge information in the future. Additionally, It may help authors improve the design of their challenge.
The proposal submission site [2] starts accepting submissions on 2 November 2020. Proposal submissions are due on 10 December 2020.
The OpenReview platform [4], which is specified in peer review processes, will be used for the review process this year.
A recent MICCAI paper [5] showed that common practice in biomedical challenge organization can be exploited to tune challenge rankings. To prevent such incidents and to improve the quality of challenges, the MICCAI Board Challenge Working Group [6] and the MICCAI 2020 Satellite Event team decided to introduce the concept of challenge registration. Similar to how clinical trials have to be registered before starting, the complete design of accepted MICCAI challenges will be put online before the challenges take place. Changes to the design (e.g. to the metrics or ranking schemes applied) must be well-justified and officially registered online (as a new version of the challenge design).
To facilitate this process for the challenge organizers, the challenge submission system [2] has been updated in a way that allows for straightforward export of the challenge design as a PDF document. The satellite event team will export the document and have it signed by the contact person of the challenge before uploading it.
Registering challenges is an important step towards higher quality challenges. It not only has the potential to lead to more thoughtful challenge designs but also ensures that all the necessary information is available for challenge participants. Furthermore, all changes will be transparent to the community, ensuring increased quality control.
Organizing a challenge based on an accepted challenge proposal may be time-consuming, especially when large-scale data collection and annotation is necessary. To address the traditionally tight schedule between MICCAI challenge acceptance and organization we also offer the possibility of an early review of MICCAI 2022 challenge proposals this year. Organizers interested in organizing a challenge in 2022 may use this year's submission system [2] and thus receive an early acceptance that leaves time for careful preparation of the challenge. For MICCAI 2021, three challenges were already accepted by this process:
Due to the limited number of rooms available, and to remain relevant to the conference participants, the submitted challenge proposals will be selected based on the following criteria:
Proposals will be reviewed by at least two reviewers. The reviewers will be chosen based on their expertise in the area of a proposal. Reviewers will provide feedback and assist the challenge organizing committee in making a final decision. To ensure a rich and diverse program, we reserve the right to ask the applicants of proposals with substantial overlaps to merge or alter their scopes. We also reserve the right to ask applicants to provide supplementary materials and to revise their proposals in order to ensure the quality and standard of the MICCAI challenges.
Authors will assign to the MICCAI Society the right to distribute challenge materials (such as the agenda or the challenge design document) to MICCAI members and to challenge and conference attendees, independently of the copyright ownership planned by the challenge organizers.
MICCAI 2021 will not pay for travel arrangements or stipends for the organizers of satellite events nor provide concessions on registration fees. The MICCAI 2021 Satellite Events Committee has limited funds for waiving the registration fees for a number of speakers, including those from outside the MICCAI community. Organizers and speakers who only plan to attend the satellite event in which they are involved are expected to register for the event unless registration fee waivers have been requested and granted.