Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Fenqiang Zhao, Zhengwang Wu, Li Wang, Weili Lin, Shunren Xia, Gang Li

Abstract

Spatiotemporal (4D) cortical surface atlas during infancy plays an important role for surface-based visualization, normalization and analysis of the dynamic early brain development. Conventional atlas construction methods typically rely on classical group-wise registration on sub-populations and ignore longitudinal constraints, thus having three main issues: 1) constructing templates at discrete time points; 2) resulting in longitudinal inconsistency among different age’s atlases; and 3) taking extremely long runtime. To address these issues, in this paper, we propose a fast unsupervised learning-based surface atlas construction framework incorporating longitudinal constraints to enforce the within-subject temporal correspondence in the atlas space. To well handle the difficulties of learning large deformations, we propose a multi-level multi- modal spherical registration network to perform cortical surface registration in a coarse-to-fine manner. Thus, only small deformations need to be estimated at each resolution level using the registration network, which further improves registration accuracy and atlas quality. Our constructed 4D infant cortical surface atlas based on 625 longitudinal scans from 291 infants is temporally continuous, in contrast to the state-of-the-art UNC 4D Infant Surface Atlas, which only provides the atlases at a few discrete sparse time points. By evaluating the intra- and inter-subject spatial normalization accuracy after alignment onto the atlas, our atlas demonstrates more detailed and fine-grained cortical patterns, thus leading to higher accuracy in surface registration.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87196-3_25

SharedIt: https://rdcu.be/cyl2v

Link to the code repository

https://github.com/zhaofenqiang/GroupwiseReg

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

A new 4D cortical surface atlas building approach is presented in this paper. In particular, the proposed framework (unsupervised) is able to perform simultaneous atlas synthesis and individual-to-atlas registration.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Quantitative and qualitative comparisons, and ablation studies together demonstrate the effectiveness of the method and its individual components.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The second contribution listed in the conclusion is based on section 2.2 where the coarse-to-fine idea and other parts are based on existing methods, in this case, it is not recommended to list it as a major contribution.
- Format and scale of the figures, and the presentation of the paper need to be improved.
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors acknowledged they will make the code and trained models publicly available.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

A lot of details, figures, results, and discussions are presented in this submission, as a result, some of the figures are too small to read. Zooming in to see the details in figures is an option, but it negatively affects the reading experience. Please consider moving some of the figures or results to supplementary material and only keep the important figures/results in the main paper. In particular, Fig. 2 is too small, it’s better keep it the same scale as Fig. 1.

Please revise the second contribution in conclusion as it is mostly based on the literature. Having one strong contribution is better than listing several irrelevant ones.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

My recommendation is based on the technical contribution, result, and presentation of this submission. The presentation of the paper has to be improved based on above suggestions.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Review #2

Please describe the contribution of the paper

This paper develops spherical u-net towards a registration method spatio-temporal longitudinally consistent surface registration. The method is evaluated on the baby connectome data (I think?)
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

A really good example of deep learning surface registration A novel approach to longitudinally consistent image registration Methods very clearly defined Literature review comprehensive Architecture diagrams very clear Validated against an existing atlas estimated for a similar (the same?) dataset
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The abstract and first paragraph of this paper overlap with submission 2146 No details were given on initialization Validation is slightly weak specifically there is no evaluation of whether the atlas is biased to an individual; plus it should have been possible to set up the same model with spherical demons and benchmark against that The source of the data isn’t clear I don’t think you can really call this multimodal and essentially its just shape alignment Training approach isn’t clear
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The code for spherical demons is publicly available. The method is clear to follow however the data set used isn’t clear.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

This is another really nice paper from the spherical u-net team (I assume). Here, I think this is the correct abstract for this paper - where it is wrongly present in 2146 as well. It’s difficult to choose which of these is stronger as I think both have a number of strengths. In general I like that this is taking an entirely novel approach to longitudinally consistent surface registration but I think the validation is not as strong. Mostly I’m concerned that the atlas is coming out biased to a single individual and this is not evaluated. Ideally it would be good to investigate what happens when run training several times with different sampling schemes for the training set and/or different training sets. If it is biased it might explain why it’s so much sharper. Also the method isn’t validated against another technique. I think it probably wouldn’t be a very big job to implement an approximate set up to test against spherical demons. Or running the same method used for the UNC atlas on the exact same data split? This brings me to the fact that the source of the data isn’t actually specified so some of these things are difficult to judge. Regardless this is a very interesting and novel paper which in my opinion would be of significant interest to the MICCAI community. Minor issues Section 2.2. Please elaborate on ‘the polar distortion issue in spherical surface issue’ as this is not clear
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Another excellent paper methodologically but the validation might benefit from more transparency
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

5
Reviewer confidence

Very confident

Review #3

Please describe the contribution of the paper

The paper proposed a 4D atlas estimation method for infant cortical surfaces. The cortical surfaces are represented by the mean curvature and average convexity features for the atlas estimation. The cortical surfaces were deformed by the newly proposed Multi-Level Multi-Modal Spherical Registration (MM-SReg) network to a 4D atlas synthesized by a generator. The longitudinal constraint is incorporated to enhance the performance of the atlas estimation.

Incorporating the multi-level strategy and longitudinal constraint showed better performance in the ablation study for individual alignment. The proposed atlas showed the better quality than the UNC 4D atlas.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper is clearly written
- Introducing a longitudinal constraint to the infant cortical surface atlas estimation problem might be novel.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- It is not clear why the geometric features projected on a sphere needs to be used rather than using cortical surfaces themselves for the atlas estimation.
- The proposed MM-SReg network was not used at all for the comparisons with the 4D UNC atlas.
- The final alignment seems to depend heavily on the mean curvature map although the average convexity is used for the multi-level registration steps.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The reproducibility seems to be good. The authors briefly explained the implementation details and checked on the release of public code in the checklist.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

The paper presented a 4D atlas estimation method for infant cortical surfaces. The proposed architecture consisted of an atlas synthesis network and the MM-SReg network for the alignment.

The paper was well-organized and easy to follow. Incorporating the multi-level architecture for registration was well-motivated and led to better individual registration performance for inter-/intra-subject alignments. The longitudinal constraint also contributed to increase the performance in the ablation study.

It was not perfectly clear to me why the geometric features (the average convexity and the mean curvature) projected on a sphere were chosen for the atlas estimation of cortical surfaces. The features must have been extracted from a cortical surface (probably obtained by a segmentation method) and projected on a sphere. There are a lot of intermediate steps that could have led to approximation errors from cortical surfaces. As mentioned in the paper, the geometric features were not perfectly aligned. It would have been more clear if the authors explained a reason why the geometric features on a sphere was chosen over using cortical surfaces directly.

The final alignment stage of the MM-SReg used the mean curvature. The average convexity maps were then aligned by the deformation estimated with the mean curvature. I wonder why one should not use the average convexity and the mean curvature simultaneously (possibly with some weights) rather than estimating an atlas sequentially with different features.

For the comparisons with the 4D UNC atlas, the Spherical Demon registration method was chosen to align images to both 4D UNC atlas and the proposed. I wonder why the proposed MM-SReg was not used at least for the proposed method. The proposed MM-SReg showed better registration performance in the ablation study presented in the previous section. And the better registration performance eventually led to better atlas estimation. I am not sure why it wasn’t used in the final atlas evaluation stage where the registration performance was also essential. Since the MM-SReg was also a main novel component proposed in the paper, I think it needed to be evaluated for the performance of atlas estimation in addition to the Spherical Demon. Also, it would have been nice if the Spherical Demon were included in the individual alignment evaluations.
Please state your overall opinion of the paper

borderline reject (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Although the paper definitely has interesting components on infant cortical surface atlas estimation, some choices in the methodology and the evaluation were not clear to me.
What is the ranking of this paper in your review stack?

4
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

Reviewers in their detailed assessments agree on the very good clarity and organization of the paper, and on interesting novel aspects of the work. Reviewers also point to weaknesses that will need to be adressed would this paper be selected for publication. Of particular interest is an observed overlap/similarity of parts with another MICCAI submission #2146. As one reviewer notes, “abstract and first paragraph of this paper overlap with submission 2146”. It seems that the same team did submit another paper with same/similar abstract or introduction? In a rebuttal, this needs to be addressed and discussed in detail. Other weaknesses are found to be related to validation, and comparison/validation to other/another technique(s). There were also questions on the choices of the methodology as discussed by rev#3. Whereas authors specify that code will be made available, the source of the data used for the experiments does not seem to be clear and not available for others to check reproducibility. Authors are strongly encouraged to release the data together with the code.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

3

Author Feedback

We thank all reviewers’ comments and appreciate their recognition on the key contributions of our paper: 1) “a novel 4D cortical surface atlas building approach” (all reviewers); 2) “a really good example of deep learning surface registration”, “an excellent paper methodologically” (R2); 3) “comparisons demonstrate the effectiveness” (R1); 4) “well motivated and organized” (R3).

Response to R2’s concern on overlap of abstract and first paragraph with another submission 2146. The two submissions are completely independent and have different aims and technical contributions. We have double checked them using a plagiarism checker tool and the overlap reported is only 1%, which can be found in an anonymized repository https://anonymous.4open.science/r/MICCAI2021_rebuttal/.

R2’s concern on comparison with other methods. 1) R2 suggests comparing with Spherical Demons-based traditional atlas construction method that uses several rounds of atlas estimation and individual-to-atlas registration. As illustrated in the paper, such methods have three major drawbacks: slow, temporally-discontinuous and longitudinally-inconsistent, which are successfully addressed by our method. Specifically, our method substantially improves the speed, seconds for ours vs. hours or even days for traditional methods. 2) Compare with the “methods used for the UNC atlas on the exact same data”. However, both their dataset and method are not publicly available. After all, our goal is to construct and release the atlas to the community, thus we directly compare with the state-of-the-art UNC atlas in Sec. 3.3, which is more straightforward, meaningful and convincing.

R3 questions why choosing geometric features projected onto a sphere as the target cortical atlas representation. This is because spherical representation offers a simpler geometry for aligning cortical features, such as the ‘sulc’ and ‘curv’ in the paper. After the atlas is estimated and the individual-to-atlas deformations are computed, it is easy to derive other cortical features on the atlas accordingly and map them back to original surface for visualization, which can be referred to [6,8,9,17]. We will make the reference clearer.

R3 questions why using sulc and curv sequentially instead of together. As explained in Sec. 2.2, this is because 1) sulc, the coarse cortical folding feature, is relatively easy to align and thus provides a good initialization for further aligning fine feature curv. This order is popular and effective for cortical feature alignment as detailed in [6,13,18]; 2) using them separately can offer flexibility in the choice of aligned cortical features (e.g., one can replace curv with another feature of interest). We will make this clearer in revision.

R3 questions why not using MM-SReg in the comparison with UNC atlas. As already emphasized in Sec. 3.3, we use Spherical Demons (SD) because it is a third-party registration tool that does not rely on the training data. Thus, we can fairly and exclusively compare the atlases’ quality. If we use MM-SReg for aligning surfaces to our atlas and SD for UNC atlas as suggested by R3, even our atlas obtains better results, we will not know which factor (registration method or atlas) leads to that. After all, our goal is to construct and release better atlas. Other comparisons suggested by R3, such as evaluating MM-SReg and SD in individual registration, may distract the focus from the original goal on atlas building. We will make this clearer.

R2’s concern on biased atlas. As stated in Sec. 2, the atlas is regularized by the 2nd term in Eq. 1 to be as central as possible, thus the final atlas’s bias is very subtle.

R2’s concern on data availability. The raw data used in the paper is under releasing. We will provide detailed dataset information in revision.

As suggested by R1 and R2, the figures will be revised to be easily inspected, and more details of initialization, training and registration will be added and referred.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

Authors in their feedback answered to concerns on overlap with another MICCAI submission #2146, where they stated only a 1% overlap. This paper provides a new solution to atlas-building of brain surfaces that overcomes major drawbacks of previous solutions which were temporally discontinuous and longitudinally inconsistent. Authors answered most questions of reviewers, although some critiques and major points of rev#3 remained unanswered. Would this paper be accepted for MICCAI, authors are strongly encouraged to provide improvements based on reviewer’s comments.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

5

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This paper develops a novel 4D infant atlas building method. The rebuttal adequately addresses concerns from the reviewers including overlap and anonymity.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

2

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The authors introduce a longitudinal surface deformable registration and atlas construction neural network, which they apply to construct a 4D infant cortical surface atlas. The application of 4D atlas construction via deep learning in infantile brain imaging appears new. The technical concerns raised by the reviewers (especially Reviewer 3) seem to have been sufficiently addressed in the rebuttal.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

8

back to top

Learning 4D Infant Cortical Surface Atlas with Unsupervised Spherical Networks