Back to top List of papers List of papers - by topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Yuchen Pei, Liangjun Chen, Fenqiang Zhao, Zhengwang Wu, Tao Zhong, Ya Wang, Changan Chen, Li Wang, He Zhang, Lisheng Wang, Gang Li
Abstract
Brain atlases are of fundamental importance for analyzing the dynamic neurodevelopment in fetal brain studies. Since the brain size, shape, and anatomical structures change rapidly during the prenatal period, it is essential to construct a spatiotemporal (4D) atlas equipped with tissue probability maps, which can preserve sharper early brain folding patterns for accurately characterizing dynamic changes in fetal brains and provide tissue prior informations for related tasks, e.g., segmentation, registration, and parcellation. In this work, we propose
a novel unsupervised age-conditional learning framework to build temporally continuous fetal brain atlases by incorporating tissue segmentation maps, which outperforms previous traditional atlas construction methods in three aspects. First, our framework enables learning age-conditional deformable templates by leveraging the entire collection. Second, we leverage reliable brain tissue segmentation maps in addition to the low-contrast noisy intensity images to enhance the alignment of individual images. Third, a novel loss function is designed to enforce the similarity between the learned tissue probability map on the atlas and each subject tissue segmentation map after registration, thereby providing extra anatomical consistency supervision for atlas building. Our 4D
temporally-continuous fetal brain atlases are constructed based on 82 healthy fetuses from 22 to 32 gestational weeks. Compared with the atlases built by the state-of-the-art algorithms, our atlases preserve more structural details and sharper folding patterns. Together with the learned tissue probability maps, our 4D fetal atlases provide a valuable reference for spatial normalization and analysis of fetal brain development.
Link to paper
DOI: https://doi.org/10.1007/978-3-030-87234-2_23
SharedIt: https://rdcu.be/cyl8j
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The submission describes a deep learning-based framework that learns a spatio-temporal probabilistic atlas of the fetal brain using multichannel information. Besides the intensity-based input images, probability maps of 3 major tissue classes are also used as input in order to make the registration results more robust.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Overall, well written.
The input data set is relatively small, but it might be considered a big one considering the subject group. The data resolution is exceptional, and probably only available with research protocols.
Ablation studies are executed to characterize different aspects of the pipeline.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Some details are missing. The framework is very similar to already introduced work, so the novelty is low.
There is a lack of comparison to relevant work. Why did not the authors compare their new framework to spatiotemporal fetal atlases directly (Zhan2013, Serag2012, Gholipour2017)?
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
No code/data sharing was mentioned in the submission.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
Some clarification on the data segmentation would be great. Were the input data sets segmented by ADU-Net? If yes, was the current implementation used (that relied on 6mo olds), or was it re-trained on fetuses? More details about this would have been welcome given that the segmentation maps will serve as a crucial input to the atlas building pipeline.
How was the affine alignment achieved? I was also a bit confused about how the scaling factors were recovered for the atlases given that it is stated that all scans were initially affinely aligned.
Architecture (Fig 1) seems to be the same as [6], with the additional segmentation-derived layers added to the input, but there is no comparison to that work in the paper. Authors even mention that the method would work poorly, so at least some failed experimental outcomes would be welcome. Conceptually, is “Ours-w/o A” basically [6]?
Training is done in a three-step training framework; joint optimization apparently underperforms. Is the first step of the optimization equivalent to VXM? (Optional channels were also introduced there.)
It is claimed that the tool will be compared “with state-of-the-art atlas construction methods”. It is only one method that is considered. Even though the chosen one is a standard atlas creation method, I am not sure why the current submission is compared to more relevant competitors. Spatio-temporal fetal atlases or the method described in [6].
Are the results in Table 1 statistically significant?
Results in Fig 3: some of the edges actually seem to be more washed out for the here resented pipeline (esp WM-CSF).
How much tuning of the hyperparameters was done?
“ambiguous appearance” – I feel when such a statement is made about current methods this should be justified by at least a figure.
Additionally, using terminology such as “Besides”, “It is obvious” is very colloquial.
- Please state your overall opinion of the paper
probably reject (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The experimental comparison was not done to the relevant frameworks.
Lack of clarity on certain issues as explained above.
- What is the ranking of this paper in your review stack?
3
- Number of papers in your stack
4
- Reviewer confidence
Very confident
Review #2
- Please describe the contribution of the paper
The authors present a framework for generating an age-conditional fetal brain atlas in MRI. Their method learns templates using both intensity images and tissue information (CSF, WM, GM), necessary for the rapidly changing fetal brain throughout gestation. They use existing registration frameworks to find diffeomorphic transformations to register to the atlas. Their framework is unsupervised, and they demonstrate superior performance compared to the SyGN baseline.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This paper has several strengths and addresses the challenging problem of developing a fetal brain atlas.
- Novel application: age-conditional fetal brain atlases
- Novel formulation: they combine both imaging information and tissue properties to improve the training of the atlas, enabling further downstream tasks, such as segmentation
- Produce a diffeomorphic atlas using the voxelmorph framework
- Strong evaluation: they compare both to state-of-the-art and perform an ablation study, addressing the utility of incorporating anatomical information
- Evaluation includes held out test sets, demonstrating generalizability
- Learning framework is unsupervised
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Although the authors claim the learning framework is unsupervised, manual segmentations are required to extract the CSF, WM, and GM
- The gestational age is limited, and it would be interesting to see how the atlas would perform with later gestation, where greater changes in the fetal brain will occur
- Are there limitations to requiring pre-affine alignment? Does variability in brain size get captured properly?
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The model is described in sufficient detail, and the authors outline their experimental procedure clearly. The authors also claim they will release their atlases to the community.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
The paper is well-written and proposes a solution to a challenging clinical problem. I have two main constructive criticisms. The first, the requirement of pre-affine alignment makes sense, but I would like the authors to comment on how variability in brain shape may affect the atlas performance and this step. I suspect that since the gestational age range is fairly narrow, this may not be reflected.
The second,
- Please state your overall opinion of the paper
accept (8)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper produces a novel solution to a challenging clinical problem. Designing fetal brain atlases is an active area of research, and is very challenging due to the rapid development of the fetal brain throughout gestation. The authors address this by proposing an age conditional atlas, trained using a diffeomorphic registration learning framework. Furthermore, they include anatomical information, as the fetal brain anatomy changes rapidly throughout gestation, while image quality remains poor. The paper is clearly written and easy to follow.
They demonstrate significant improvements over the state-of-the-art, particularly in registration accuracy of gray matter. Furthermore, they conduct an ablation study and evaluate the significance of their anatomical constraint, justifying the reasons for their model design.
Their qualitative figures demonstrate the improvements brought by their atlas, and this work represents an important contribution in fetal brain atlas development.
My on
- What is the ranking of this paper in your review stack?
1
- Number of papers in your stack
5
- Reviewer confidence
Confident but not absolutely certain
Review #3
- Please describe the contribution of the paper
The paper presents a deep-learning-based method for fetal atlas construction. The main contributions are the formulation of the atlas construction task and the use of anatomical constraints for the registration. The evaluation suggests that the proposed method produces more accurate segmentation compared to previously suggested fetal atlas construction methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- New formulation of the fetal atlas construction problem.
- Comparison with a previously published method.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
One main issue is the clinical value of the proposed approach. Is this method provides any additional information we cannot get by standard approaches? The segmentation accuracy seems to be a byproduct and not the metric of interest for clinical applications of the atlas.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
- Authors claim that they will release their atlas to the public
- Technical training parameters are provided
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
See section 4 above.
- Please state your overall opinion of the paper
accept (8)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The presentation is clear, formulation is new. While the clinical value is questionable the paper can be still considered as a good paper.
- What is the ranking of this paper in your review stack?
1
- Number of papers in your stack
5
- Reviewer confidence
Confident but not absolutely certain
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Reviewers in their detailed assessments agree on the very good clarity and organization of the paper, and on interesting novel aspects of the work. Reviewers also point to weaknesses that will need to be adressed would this paper be selected for publication. In particular, reviewer #1 lists several major weaknesses such as similarity to existing work on spatiotemporal fetal atlas methods by several other groups. A detailed assessment of such similarities/overlap will be necessary for a final judgement for this paper to meet MICCAI standards in regard to describing similarities/differences/innovations w.r.t. to existing methods.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).
3
Author Feedback
Meta-reviewer’s comments “describing similarities/ differences/ innovations w.r.t to existing method”. 1) Innovations: our proposed method is a novel unsupervised learning-based fetal atlas construction framework as affirmed by R2 and R3, which can jointly perform the diffeomorphic deformable registration and atlas construction. The proposed method combines both image intensity and tissue information to improve the quality of the atlas, enabling further downstream tasks, such as segmentation and analysis (R2). Spatiotemporal characteristics of fetal brain development are effectively captured by our constructed atlases, which demonstrate sharper structural details, higher accuracy and better time efficiency than conventional fetal atlases (R2, R3). 2) Reference [8][15][18] are conventional methods for constructing fetal brain spatiotemporal atlases, without leveraging the powerful deep learning. Specifically, they require pair-wise registration from each individual to the corresponding template in an iterative procedure, thus leading to an extremely long runtime for atlas construction. To compute the continuous template at any given age, kernel regression method is introduced using weighted supports from their temporal neighbors for some populations of interest. However, since fetal brain development is complicated and nonlinear, the kernel needs to be tuned very carefully to fit for both the complex development and distribution of the dataset. [6] proposes a learning-based atlas construction method for the adult brain, but works poorly on fetal brain as it only uses the noisy intensity information and many structural details cannot be well captured and modeled (Fig. 3(a) Baseline). 3) Comparison experiment: [8] uses the symmetric groupwise normalization (SyGN) to construct atlases. SyGN considers both shape and appearance and unbiases toward any specific individual and is recognized as the state-of-the-art template building method (Dong. et al Science Bulletin 2020). Hence, we follow [8] and use SyGN algorithm to construct the competing atlas for comparison, called MC-SyGN, which also uses multi-channel features (intensity and tissue maps). Compared to MC-SyGN, correlation coefficient (CC) obtained by our method increases from 94.7% to 97.8%, as shown in Table 1. Particularly, our method greatly improves the accuracy of registration-based segmentation. In Table 1, our work is also compared with [6] (‘Baseline’), which only uses intensity information to construct atlases and achieves 96.1% for CC, 1.7% lower than ours. In Fig. 3, our constructed atlases can capture more complex structural details, which are missed in atlases constructed by [6]. “Ours-w/o AC“ means the multi-channel model without anatomical constraint, not [6]. Compared to Ours-w/o AC, CC of ours increase from 96.8% to 97.8%. This also validates the contribution of the anatomical constraint. The results in Table 1 are statistically significant (p<0.05). R1, R2 question how affine alignment is performed. Due to the rapid fetal brain size changes, similar to [6], we preprocess the data using affine alignment to help network focus on local shape difference, not brain size. We use ANTs software to perform affine registration. Affine transformation is computed for each subject and an average affine transformation in a subgroup is computed by ANTs. When the network generates atlases, the scaling factors can be recovered from the transformations and then used to rescale the constructed atlases to reflect global volume changes. We will make this clear in final version. R1, R2, R3 concernd on segmentation preprocessing. We adopt the same strategy as in [17] to retrain a model on our fetal dataset. The achieved segmentations are then manually corrected by experts. Based on our experiments, incorporating tissue segmentation maps can greatly improve the quality of the atlas. Besides, tissue segmentation is clinically important to volumetric and morphometric analysis [8].
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper has novelty in its approach and methodology and seems an important contribution to longitudinal image analysis of fetal MRI brain structures - in particular as this seems a generic concept to be applied to other driving applications as well. The rebuttal clarifies major concerns by reviewers, in particular reviewer 1. Reviewer’s suggestions for clarifications and improvements should definitely be included in a final submission would MICCAI decide this paper to be accepted.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).
2
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper introduces a novel unsupervised learning-based fetal brain atlas construction framework that jointly performs diffeomorphic deformable registration and atlas construction. Besides the intensity-based input images, the three main tissues probability maps can also be used as input in order to make the registration results more robust (according to the results). The paper is very well written and organized, though I think some figures could be optimized as redundant information is presented there (atlas at 2 different GA are repeated in multiple figures, optimizing the figures could allow space for having an additional one requested by R1). The more confident reviewer is though raising an important concern on the chosen baseline atlas construction method used for comparison, as other more competitive spatio-temporal atlas could have been selected. But the authors rebuttal that those more relevant spatio-temporal techniques from a more theoretical discussion: they need more tedious pair-wise registrations and the used regression approaches seem not as efficient as a deep learning based method (that can deal very efficiently with non-linearities, as the one given by fetal brain growth). This is a valid hypothesis, still remain to be proven. However, I think though that the baseline used is also valid as a recent approach, so the presented framework is promising and its contribution interesting for MICCAI.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).
7
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The rebuttal addressed the key criticism on relation and comparison to related work [6].
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).
7