Back to top List of papers List of papers - by topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Arnaud Boutillon, Pierre-Henri Conze, Christelle Pons, Valerie Burdin, Bhushan Borotikar
Abstract
Automatic segmentation of magnetic resonance (MR) images is crucial for morphological evaluation of the pediatric musculoskeletal system in clinical practice. However, the accuracy and generalization performance of individual segmentation models are limited due to the restricted amount of annotated pediatric data. Hence, we propose to train a segmentation model on multiple datasets, arising from different parts of the anatomy, in a multi-task and multi-domain learning framework. This approach allows to overcome the inherent scarcity of pediatric data while benefiting from a more robust shared representation. The proposed segmentation network comprises shared convolutional filters, domain-specific batch normalization parameters that compute the respective dataset statistics and a domain-specific segmentation layer. Furthermore, a supervised contrastive regularization is integrated to further improve generalization capabilities, by promoting intra-domain similarity and impose inter-domain margins in embedded space. We evaluate our contributions on two pediatric imaging datasets of the ankle and shoulder joints for bone segmentation. Results demonstrate that the proposed model outperforms state-of-the-art approaches.
Link to paper
DOI: https://doi.org/10.1007/978-3-030-87193-2_23
SharedIt: https://rdcu.be/cyhLP
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The authors’ proposed a single segmentation network architecture that can be trained on similar domain different datasets and using domain-specific batch normalization and a contrastive regularization to learn weights across multiple domain tasks.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- This paper represents the application of meta-learning in the medical field especially in cases of scarce data.
- This paper possesses the potential to showcase how the medical community can use the learnings of multiple tasks of a similar domain.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The overall experiments, and theoretical explanation looks good. I will just suggest to add more evaluation metrics such as precision, recall, etc. Refer following papers:
- https://arxiv.org/abs/2006.01263
- https://arxiv.org/abs/2006.14822
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
The paper is very well written. its’ easy to follow even in cases of complicated explanations. I will just suggest authors to add 1-2 line definition of regularization while defining the Supervised Contrastive Regularization. Also, consider citing relevant few-shot learning approaches summary such as:
- https://arxiv.org/abs/2102.06285
- https://arxiv.org/abs/2008.06365
- Please state your overall opinion of the paper
strong accept (9)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is a good application of how deep learning architectures can be trained across similar task in different domains or with different tasks in similar domain, and yet their learnings will map to the objective. Such applications has been seen in case of recommendation systems, robotics learning. It’s good to see such applications of multi task learning in medical community.
- What is the ranking of this paper in your review stack?
2
- Number of papers in your stack
5
- Reviewer confidence
Confident but not absolutely certain
Review #2
- Please describe the contribution of the paper
The authors propose to use domain-specific batch normalization and contrastive learning to develop a multi- tissue and domain segmentation model for pediatric MR images.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper provides a novel use of combination of contrastive regularization and domain-specific batch normalization to tackle a multi-domain segmentation problem
- The paper is written clear and concise way
- The evaluation of the proposed method was done properly,
- The comparison of other segmentation approaches are reasonable.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- It is not clear why did authors choose a specific number of epochs to train the model
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
data and code is not available to reproduce results.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
The authors did an excellent job to provide details of their approach. They overlay the previous studies and explicitly identified their contribution to the field.
It could be better to have the images segmented by more than a single rater to asses inter-rater agreement for the task. - Please state your overall opinion of the paper
accept (8)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper is presented a novel use of contrastive loss to improve the domain similarity and margins for the prediction of multi-domain segmentation in a single DL model.
- What is the ranking of this paper in your review stack?
1
- Number of papers in your stack
5
- Reviewer confidence
Confident but not absolutely certain
Review #3
- Please describe the contribution of the paper
The work introduces a method for multi-task multi-domain segmentation of MR pediatric data based on UNet. Several improvements are suggested with the purpose of learning more transferable and effective filters in a limited data setting: domain-specific layers (mainly, batch normalization) and enforcement of cluster-assumption on the intermediate representations from different domains. The suggested modifications are evaluated against the baseline model and the naive joint training approach. The experiments are additionally performed with a modified UNet architecture.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The considered pediatric MRI applications are covered in the scientific literature very sparsely. In this sense, the manuscript bring notable value to medical research corpus;
- The manuscript is excellently written. The applied methodology is scientifically sound, clearly supporting the set hypotheses and showing insights into the impact of the proposed modifications (Table 1, t-SNE in Fig.4 and Supplemental Materials);
- The work brings additional evidence into the utility of domain-/task-specific batch normalization layers, the value of domain-specific representations in very low-data regime (based on the shoulder dataset scores).
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Very limited sample size and only one pair of datasets is considered, thus, limited statistical significance of the findings;
- Generalization of the results to other model architectures (e.g. VGG encoder+UNet decoder, ResNet encoder+FPN decoder, EfficientNet encoder+DeepLab decoder, etc) is also unclear, since the evaluated Att-UNet can be seen as a rather minor modification of UNet;
- The proposed method is positioned as a solution to a segmentation in multi-task multi-domain low-data setting. The authors, however, leave out of the investigation transfer learning approaches, which are commonly used in medical imaging studies in such situations. Importantly, it has been reported in (Raghu et al. 2019 Transfusion), (Mustafa et al. 2021 Supervised) that transfer learning from ImageNet-pretrained models in medical imaging applications is likely beneficial in multiple aspects;
- The rationale behind the proposed domain-specific cluster assumption is unclear. This assumption is neither sufficiently supported by the references to a prior art, nor is strongly intuitively beneficial.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
- Generally, the method is described in sufficient detail for its reproduction. It would be valuable to have reported: the rotation angle range in data augmentations, the parameters of morphological closing (i.e. kernel);
- The authors should provide more specifics on the used MRI acquisition setting - coil, sequence, TR, TE, etc;
- The authors should report distribution of age, sex, and BMI of the subjects, number of healthy/pathological cases.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- In the abstract the authors say: “Results demonstrate that the proposed model outperforms state-of-the-art approaches”. It is not clear what is considered the state-of-the-art approach? If UNet is considered to be the one, the authors should probably switch to a bit more careful statement;
- The authors report “optimal” values for tau and lambda hyper-parameters of the method. It would be appropriate to show their impact on the final results in an ablation study;
- The perplexity values for the t-SNE analyses should be reported.
- Please state your overall opinion of the paper
borderline accept (6)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- Novelty of using a deep learning-based method in the considered pediatric domains;
- Multi-domain multi-task approach in a limited-data setting, yielding a substantial benefit in both tasks;
- Excellent and easy-to-follow writing, sound methodology;
- Missing comparison to transfer learning-based solutions;
- Evaluation of the proposed method is rather limited (only one pair of datasets, essentially single architecture).
- What is the ranking of this paper in your review stack?
3
- Number of papers in your stack
5
- Reviewer confidence
Confident but not absolutely certain
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper introduces a novel combination of contrastive regularization and domain-specific batch normalization for multi-domain segmentation in pediatric datasets. The strengths of the paper are: 1) addressing an sparsely addressed important problem, 2) studying the importance of domain specific representations in low data regimes, 3) very clearly written. There were some concerns regarding limited comparisons to state-of-the-art techniques and limited dataset size; however, these are generally outweighed by the positives and all reviewers agree on the value of the paper.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).
2
Author Feedback
We would like to thank the reviewers for their comments and suggestions. We appreciate the time and efforts that the reviewers dedicated to provide insightful feedbacks on the manuscript. We have incorporated most of the suggestions made by the reviewers.
Here is a response to the reviewers’ comments and concerns.
Reviewer 1 • Regarding the definition of the supervised contrastive regularization, we have added the following information in Section 2.2 of the manuscript: “We designed a novel regularization term aimed at conserving intra-domain cohesion and inter-domain separation in the shared representation.” and “During optimization, only the weights of the encoder were penalized by the proposed regularization.” to emphasize and clarify the role of the regularization.
Reviewers 2 and 3 • Regarding the selection of the hyper-parameter values (ie: number of epochs, lambda, tau, etc), as in any deep learning pipeline, these values were selected based on trial-and-errors (as mentioned in Section 3.2), and we used a cross-validation to avoid overfitting (as mentioned in Section 3.3). Furthermore, as suggested by reviewer 3, it would be beneficial for the manuscript to evaluate the impact of the hyper-parameters lambda and tau on the segmentation performance. We cannot incorporate this suggestion in the manuscript in its current form (due to space issues), but we will expand on this aspect in a journal manuscript.
Reviewer 3 • Regarding the rationale behind the proposed domain-specific cluster assumption, we added the following information to emphasize our hypothesis at the beginning of section 2.2: “We assumed that learning a shared representation with domain-specific clusters would enhance the generalizability of the decoder and improve the accuracy of the segmentation predictions. More precisely, we assume that a local variation in the shared representation should preserve the category of the domain”. However UNet DSL does not respect this condition, as seen in Fig. 4. We thus proposed a Supervised Contrastive Regularization approach to enforce such constraint on the shared representation. Moreover, we have already referred to prior art in the introduction as “In representation learning, a good representation can be characterized by the presence of natural clusters corresponding to the classes of the problem (Bengio et al, 2013 Representation).” which supports our hypothesis that domains should be separated in the embedded space.
• Regarding additional information on the imaging datasets, we have updated Section 3.1 with the details of MRI sequence parameters used during acquisition. We have also incorporated the information regarding the number of healthy and pathological cases in section 3.1. The age distribution has been already reported in the manuscript in terms of minimum, maximum, mean and standard deviation. However, we will have to refrain from adding other requested information (sex and BMI of the subjects) due to space issues.
• Regarding the hyper-parameter values used in our experiments, we have now incorporated the translation range (±25%) and rotation angle (±45°) for data augmentations, as well the parameter of the morphological closing (5x5x5 kernel) in section 3.2. Moreover, we have added the values of the perplexity (p = 30) and learning rate (lr = 200) used for t-SNE in Section 4.2. We hope this information will improve the reproducibility of the experiments.
• Regarding the evaluation against other model architectures and transfer learning approaches, we agree that such experiments would be beneficial to the manuscript and would emphasis the generalization of the method. We cannot incorporate this suggestion in the manuscript in its current form (due to space issues), but we will expand on this aspect in a journal manuscript.