Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Jun Wei, Yiwen Hu, Ruimao Zhang, Zhen Li, S. Kevin Zhou, Shuguang Cui

Abstract

Accurate polyp segmentation is of great importance for colorectal cancer diagnosis. However, even with a powerful deep neural network, there still exists three big challenges that impede the development of polyp segmentation. (i) Samples collected under different conditions show inconsistent colors, causing the feature distribution gap and overfitting issue; (ii) Due to repeated feature downsampling, small polyps are easily degraded; (iii) Foreground and background pixels are imbalanced, leading to a biased training. To address the above issues, we propose the Shallow Attention Network (SANet) for polyp segmentation. Specifically, to eliminate the effects of color, we design the color exchange operation to decouple the image contents and colors, and force the model to focus more on the target shape and structure. Furthermore, to enhance the segmentation quality of small polyps, we propose the shallow attention module to filter out the background noise of shallow features. Thanks to the high resolution of shallow features, small polyps can be preserved correctly. In addition, to ease the severe pixel imbalance for small polyps, we propose a probability correction strategy (PCS) during the inference phase. Note that even though PCS is not involved in the training phase, it can still work well on a biased model and consistently improve the segmentation performance. Quantitative and qualitative experimental results on five challenging benchmarks confirm that our proposed SANet outperforms previous state-of-the-art methods by a large margin and achieves a speed about 72FPS.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87193-2_66

SharedIt: https://rdcu.be/cyhMK

Link to the code repository

https://github.com/weijun88/SANet

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

This paper is built upon a multi-stage decoder framework proposed in [19] (Wei, AAAI 2020) by adding the shallow attention module and applying to the small polyp segmentation problem.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The paper performs an intensive evaluation using 5 polyp segmentation datasets and 6 state-of-the-art seegmentation methods.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. The methodological novelty may be limited. The network architecture (Fig. 2) is inspired by the F3net proposed in [19] (Figure 2 in Wei, AAAI 2020), with minor modifications by changing the cross-feature module (CFM) into a shallow attention module (SAM).
2. In Figure 5, the proposed method seems to substantially outperform existings methods such as the PraNet, producing quite different segmentations. However, in terms of the Dice metric, their difference is often quite small in Table 1. Do they perform similarly on most cases?
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors will make the code available upon acceptance of paper.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
1. Page 2: “most polyp sizes are located at (0, 0.1)”.
Please explain the unit for the numbers (0, 0.1).
1. Section 3.3: logit reweighting
The logit reweighting step explicitly shifts the decision boundary by increasing the probability for positive pixels in Eqs. (5) and (6). The motivation is to focus more on foreground pixels. I am not sure whether this is really necessary, as in Eq.(7) there is already a Dice loss which only accounts for foreground pixels. Would this step perform more like manually tweeking the hyperparameters rate^p and rate^n to achieve a higher Dice in the experiments?
1. Section 4.1 It is not very clear what the training set is here. Is there only a single training set or 5 different training sets for 5 datasets? Are the training/test splits similar to other existing works?
2. Section 4.1 What does the mean absolute error (MAE) denote? For which metric is the error calculated?
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The methodological novelty compared to F3Net (Wei, AAAI 2020) may be limited.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

5
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

This paper proposed a new architecture of CNNs to segment the polyps in colorectal images. Their method includes three parts, encoder, multi-stage decoders for fine tuning and a module called SAM for shallow feature de-noising.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- real life issue (both application and small size segments)
- comprehensive experiments
- well writing and description
- technically reasonable
- code sharing
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Nothing special in my mind. But we can not find the results of the methods more focused on polyp segmentation.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The developed codes are shared and they are well organized. Most of the datasets are also publicly available. But, the details of the train and test sets are also required.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- related work section is mostly about medical segmentation, and is rarely focused on polyp segmentation. Needs improvement.
- same issue with the results section. We have only the results of the general medical segmentation methods.
- Finally, how many decoders are used in the exploited method? it is not clear in the text body.
- I can not find the information about the train and test sets. Also, what does it mean in figure 1: “One training set and five testing sets are shown in different colors.”
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- reasonable method
- well organized paper and development
What is the ranking of this paper in your review stack?

3
Number of papers in your stack

5
Reviewer confidence

Very confident

Review #3

Please describe the contribution of the paper

The paper is aimed at improving segmentation of polyps in publicly available colonoscopy datasets and focuses on shallow network with attention mechanism and dual decoder network. The paper writes the title enhanced small polyp segmentation which the reviewer assumes the reader needs to assume that it performs better on both small and large polyps?
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Simple network with real time capability
- Use of probability correlation strategy (PCS) is interesting
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Using a dual decoder and attention mechanism for polyp segmentation is not new (see DDANet: Dual Decoder Attention Network for Automatic Polyp Segmentation)
- Train and validation split is not given. Table 1 will be hard to compare with other SOTA methods.
- Other SOTA methods such as DeepLabv3+, HRNet etc could be validated
- Use of PCS after every network could allow to understand the boost in performance, especially that of PraNet.
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
- Please explicitly provide the train-val split for each dataset
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- mDSC and mIoU are very close metrics. Both can be very deceiving. Please include precision, recall separately as well.
- Has PraNet been run by the authors?
- Inference speed: 72 FPS on what size of images and does 64 FPS with PraNet on the same GPU and same size?
- Other networks such as DeepLabV3+ and HRNet should be explored and compared as they incorporate scale features. Also, could authors use their PCS on top of these models to better understand the performance difference between their chosen architecture with and without PCS.
- Please include failed cases
- Please consider changing the title (if accepted) to more meaningful as the network can segment both small and large polyps
Please state your overall opinion of the paper

probably reject (4)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- Simple network design but require more experiments on other public datasets
- It is not clear how it is effective to small size polyp. Authors have not demonstrated it in terms of splitting the test samples based on the size
- Experimental design needs to be reconfigured as per the title
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

5
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

The paper is well-written. The technical novelty seems limited and the model seems minor modification of [19] yet no comparison with [19] was included. The experimental data used is thorough yet important methodological and experimental details are missing. Effectiveness against small polyp (as the title suggest) is not clear. Why other networks like Deeplabv3++ and HRNet that incorporate scale features are not compared?
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

4

Author Feedback

Thanks all the reviewers for the suggestions. We will further polish the final version. The common questions are firstly answered, then we reply to individual reviewer.

CQ1: Novelties and contributions. CA1: Small polyps usually suffer from poor segmentation, while it is with great clinic significances since small polyps are easily miss-detected. Thus, in this paper we improve small polyps segmentation through enhanced feature extractor, contextual fusion decoders and prediction reweighting. Firstly, we design the shallow attention to focus more on shallow features since they contain more complete details for small polyps. This is the core contribution. Then, to achieve more contextual information, we apply two decoders. Besides, due to unbalanced pixel distribution for small polyp, the edge area is usually blurred. Therefore, we propose the PCS to further refine the boundaries.

CQ2: Small polyp segmentation results? CA2: For large polyp, our proposed model achieves the significant dice gains, i.e., 3.7%(ColonDB), 1.1%(EndoScene), 6.8%(ETIS), 1.5%(ClinicDB). For polyps with different scales, we have shown the performance in Table 1 of the supplementary materials. Note that for small polyps, over 10% mDice and 9% mIoU gains are achieved. This demonstrates the effectiveness for our proposed methods.

CQ3: Comparisons with Deeplabv3++ and HRNet. CA3: Since polyp segmentation datasets are very small compared with nature images, DeepLabv3++ and HRNet suffers from serious overfitting issue. Specifically, the mDice for ClinnicDB, ColonDB, ETIS datasets are 0.52, 0.58, 0.24, and 0.43, 0.49, 0.27 for DeeplabV3+ and HRNet, respectively. It is obvious that the results are far below our methods. Besides, both deeplabV3+ and HRNet take in very high resolution images, which leads to very slow inference speed and cannot meet the real-time requirements. That’s why previous methods like PraNet didn’t compared with these models neither.

CQ4: Dataset CA4: For fair comparison, our model is trained only on 1 dataset (not 5 datasets) but tested on 5 different datasets as PraNet. In practice, we directly use the dataset setting released by PraNet.

Reviewer #1: Q1: Comparison with F3Net A1: F3Net proposed in AAAI mainly focuses on the salient object detection, which cannot be directly used for polyp segmentation. Thus, even one sub-module is inspired by F3Net, we design the whole lightweight SANet framework for polyp segmentation, achieving the better and faster predictions.

Q2: Details A2: “(0, 0.1)” means the proportion of polyp area in the image is smaller than 0.1. rate^p and rate^n are not hyperparameters. They are generated based on the predictions automatically without any tuning. MAE denotes the mean of pixel-wise error, i.e., MAE= sum( ||pred(i,j) - gt(i,j)|| )/N

Reviewer #2: Q1: Results for polys with different size. A1: Please refer to CA2.

Q2: Details. A2: We use 2 decoders as pointed in Section 3.4. We use same dataset settings with PraNet.

Reviewe #3: Q1: Comparison with DDANet. A1: Different with DDANet, please refer to CA1. Besides, our decoders are exactly the same to get more context for small polyps, but DDANet uses the segmentation decoder for feature fusion and autoencoder for the regularization loss. Finally, our model outperforms DDANet by a large margin. We would compare DDANet in the final version.

Q2: Experiments results. A2: We have shown the model performance under different thresholds, as shown in Fig.4, which is similar to the PR curve. The results prove that our model is robust. For the inference time of PraNet, we run PraNet and our model under the same environment. The results shows that our model is much faster, i.e., 72FPS.

Q3: DeepLabV3+ and HRNet A3: Please refer to CA3.

Q4: Effectiveness of PCS. A4: We further applied the PCS to PraNet. As a result, the mDice improves for ClinicDB (89.9 to 90.1), ColonDB (70.9 to 71.2), EndoScene (87.1 to 87.3). It proves the effectiveness of PCS.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The authors have clarified all major concerns raised by the reviewers. The camera ready should be updated to include the justifications and additional results from the rebuttal.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

4

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The authors responded to the major concerns. Thus, I prefer to accept.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

3

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This paper is well-written and it presents some interesting ideas, such as the probability correction strategy.

There are some concerns raised by the reviewers concerning the methodology which have not been satisfactorily addressed in the rebuttal. Although, I incline towards recommending acceptance, I consider the authors should provide a better explanation regarding F3Net. Saying that salience detection cannot be used for polyp segmentation falls a bit short.

If accepted, the papers need to consider either 1) change the title, since the results in the main document do not support it or 2) include Table 1 from the supplementary material in the main text
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

11

back to top

Shallow Attention Network for Polyp Segmentation