Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Lei Li, Sheng Lian, Zhiming Luo, Shaozi Li, Beizhan Wang, Shuo Li

Abstract

Recently, CNN-based methods lead tremendous progress in segmenting abdominal organs (e.g., kidney, liver, and pancreas) and anomaly tumors in CT scans. Although 3D CNN-based methods can significantly improve accuracy by using 3D volume as input, they need more computational cost and may not satisfy the efficiency requirement for many practical applications. In this study, we mainly aim at improving the 2D segmentation by leveraging the consistency- and- discrepancy-context information from adjacent slices. Specifically, the consistency context mainly considers that the prediction variance of two adjacent slices needs to follow the variance in the ground truth. The discrepancy-context assumes the label difference of adjacent slices usually occurs in the edge area of organs. To fully utilize the above context information, we further devise a two-stage 2.5D segmentation framework based on the U-Net that takes three adjacent slices as input. In the first stage, we encourage the predictions of the three slices following the consistency context. In the second stage, we refine the segmentation result by adopting the prediction discrepancy area of adjacent slices as an extra input. Experimental results on several challenging datasets demonstrate the effectiveness of our proposed methods. Moreover, the adjacent-slice context information considered in this study can be effortlessly incorporated into other segmentation frameworks without extra testing overhead.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87193-2_25

SharedIt: https://rdcu.be/cyhL3

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The authors present a 2-stage 2.5D CNN that leverages consistency- and discrepancy-context information from adjacent slices for 2D segmentation. The consistency context considers the prediction variance between adjacent slices needs follow that of ground truth, and discrepancy-context contains the difference of the edge area of organs.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The paper is well organized and the proposed method is easy to follow. The results show improvements on different 2D backbones equipped with the proposed method.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Although the authors claim the efficiency of the proposed method, it lacks of the results of training time and inference time, and the comparison with 3D models to show the superiority of the proposed method. The authors argued that the method better capture the inter-slice context information of organ/tumor edge regions. In this situation, Dice similarity is insufficient for the evaluation. The inter-slice spacing is various between CT scans and datasets, and it is obviously a key aspect for the proposed method that leverages inter-slice context. It lacks of details that how the authors handle it.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The datasets are public available, and the network structure is listed.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

The authors should have compared with a 3D model and report the training time and inference time to demonstrate the efficiency of the proposed method. It can also show the performance gap between 2D/2.5D and 3D models with different spatial context. Hausdorff Distance or Surface Distance should be considered as an extra metric to show the effectiveness of the proposed method on organ/tumor’s edge regions. The effect of inter-slice spacing should be included. A typo on page 6, line 4: “while computes the segmentation results for all three slices for training and training.”
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Refer to Section 3
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

3
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

The paper introduces a two-stage framework, which is based on a u-net architecture, for 2D organ segmentation in 3D CT scans. The framework operates in a sliding-window fashion using a set of three consecutive 2D axial slices. The first stage generates coarse segmentation predictions. The second stage refines the segmentation by using the prediction discrepancy area of adjacent slices as an auxiliary input. The framework was tested on three publicly available CT datasets using the Dice coefficient as the evaluation metric.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The proposed framework appears to be general enough to be used for other 3D modalities (e.g., MRI).
- Evaluation on the publicly available data
- Paper is easy to follow
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Evaluation experiments do not provide sufficient support for some of the claims.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The paper appears to provide enough details to reproduce the results.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- The abstract as well as the introduction of the paper state that 3D models, as opposed to 2D models, are more computationally costly and are less efficient. However, the authors did not provide an actual comparison. It is unclear if the presented two-step 2D approach is indeed more efficient and is faster than a single-step 3D approach.
- Additionally, no comparison against 3D methods was provided.
- “… a vanilla U-Net model usually suffers the issue of discontinuous prediction, missing the target” - Does a u-net struggle to segment small targets just because of its architecture design? There are a lot of factors that can influence the performance, including the loss function.
- Minor: I suggest replacing “Kid.”, “Liv.”, “Panc.” In Table 2 with just “organ” to make it consistent with the notation in Table 3.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Interesting segmentation framework; Evaluation experiments do not provide sufficient support for some of the claims.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

3
Reviewer confidence

Very confident

Review #3

Please describe the contribution of the paper

This paper investigates segmentation of abdominal organs and tumors in CT scans using a CNN with contextual information. To do this, a two-stage U-Net based 2.5D segmentation network is proposed and public datasets are used to train models.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The paper is well presented overall.

Investigation of consistency context and discrepancy context with 2.5D U-Net based architecture in a two-stage fashion is an interesting and novel topic because it is a deeper analysis than simply using just multiple slices.

The authors use datasets with pancreas, a challenging and highly variant organ, which provides a meaningful comparison about the usage of contextual information.

The results show that consistent improvements for all organs and tumors.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

A major weakness is the lack of experiments using whole 3D volumes and their comparisons with the proposed network. As a reviewer, I am not able to see the trade-offs clearly.

Some parts in the paper are described insufficiently. For example, what is the meaning of “we leverage the discrepancy context as extra attention guidance.” Further explanations on how this attention mechanism is implemented as well as the “segmentation refinement with discrepacny context knowledge” are needed.

Selection of lambda_CR is unclear. How did the authors determine the value? Are there any extensive experiments to show for the optimization of this lambda?
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors selected public datasets to test their proposed network, which is great. However, the lack of code and unclear explanations related to the implementation of discrepancy context are problematic from the point of the reproducibility.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

A discussion about performance of 3D architecture compared to the proposed methodology as well as results of experiments with 3D volumes will help readers to better understand the tradeoffs between 2D, 2.5D, 3D and the proposed network. It will also help to provide more explanation on implementation of discrepancy context knowledge and selection of lambda_CR for experiments.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The major factors are the well presented and novel idea as well as evaluation with difficult organs. I think it would be beneficial for the MICCAI community to see the proposed methodology.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

5
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Strengths: *The paper is well organized and the proposed method is easy to follow.
- The results show improvements on different 2D backbones equipped with the proposed method. *Investigation of consistency context and discrepancy context with 2.5D U-Net based architecture in a two-stage fashion is an interesting and novel topic because it is a deeper analysis than simply using just multiple slices.
- The authors use datasets with pancreas, a challenging and highly variant organ, which provides a meaningful comparison about the usage of contextual information.
- The results show that consistent improvements for all organs and tumors
Weaknesses:
- it lacks of the results of training time and inference time, and the comparison with 3D models to show the superiority of the proposed method.
- Evaluation experiments do not provide sufficient support for some of the claims. *
Overall:
- This is a strong paper that does not consider 3D methods. 3d consideration is not essential.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

1

Author Feedback

N/A

back to top

Learning Consistency- and Discrepancy-Context for 2D Organ Segmentation