Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Yue Zhang, Chengtao Peng, Liying Peng, Huimin Huang, Ruofeng Tong, Lanfen Lin, Jingsong Li, Yen-Wei Chen, Qingqing Chen, Hongjie Hu, Zhiyi Peng

Abstract

Multi-phase computed tomography (CT) images provide crucial complementary information for accurate liver tumor segmentation (LiTS). State-of-the-art multi-phase LiTS methods usually fused cross-phase features through phase-weighted summation or channel-attention based concatenation. However, these methods ignored the spatial (pixel-wise) relationships between different phases, hence leading to insufficient feature integration. In addition, the performance of existing methods remains subject to the uncertainty in segmentation, which is particularly acute in tumor boundary regions. In this work, we propose a novel LiTS method to adequately aggregate multi-phase information and refine uncertain region segmentation. To this end, we introduce a spatial aggregation module (SAM), which encourages per-pixel interactions between different phases, to make full use of cross-phase information. Moreover, we devise an uncertain region inpainting module (URIM) to refine uncertain pixels using neighboring discriminative features. Experiments on an in-house multi-phase CT dataset of focal liver lesions (MPCT-FLLs) demonstrate that our method achieves promising liver tumor segmentation and outperforms state-of-the-arts.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87193-2_7

SharedIt: https://rdcu.be/cyhLz

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

Existing multi-phase feature fusion techniques may neglect the pixel-wise correspondence between different phases, which can lead to redundancy and low efficiency in information aggregation. This paper introduces a spatial aggregation module to encourage per-pixel inter-phase interactions. In addition, an uncertain region inpainting module is used to refine the tumor boundary segmentation. The authors show the efficacy of the proposed method by comparing with state-of-the-art methods and ablation study.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

1) The paper is well-written and clear, the figures help understand the workflow of the proposed method 2) The proposed two modules are novel. Although the idea of using confidence map to refine segmentation is not new, the proposed structure is new. 3) The paper is well motivated. 4) Well-designed experiment
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The dataset illustration is limited, is it public or in house collected? How many cases are there for each of the five tumor types?
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors intend to release the code, which should be reproducible.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

This is a very good paper. Some minor suggestions:

1) A few typos. For example, in Page 3, “In Overall”, “O” should be lower case. In Equation 4, what’s the operation between X and Mconf, if it is element-wise multiplication, it should be the same Equation 1. 2) In page 5, the range of Mconf should be [0,1) instead of (0,1]. You have a “1-“ before the original equation in reference [8], thus when the probability is 0.5, you can actually reach 0.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

novel idea and good experiment results
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Somewhat confident

Review #2

Please describe the contribution of the paper

The paper proposes a framework for multi-phase liver tumor segmentation (LiTS), in which, a spatial aggregation module (SAM) is introduced to aggregate multi-phase information, while an uncertain region inpainting module (URIM) is designed to refine uncertain region segmentation.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

(1) The paper is well organized, and the idea is easy to follow.

(2) Truly has some innovation, e.g., uncertain region inpainting module.

(3) The authors carried out extensive experiments to prove the effectiveness of the proposed method.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

(1) The MPCT-FLLs dataset is not publicly accessible. The method can be evaluated with public datasets.

(2) The authors only considered two phases (PV and ART) for multi-phase LiTS segmentation, rather than four phases in CECT.

(3) The description of the mapping functions is unclear. How to concatenate these features global average pooling (GAP) and convolutional layers with different kernel sizes.

(4) The model complexity could be compared.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Some details of methodology is ambiguous, which may influence the reproducibility.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

(1)The results could be re-organized, since there are 5 tables, and they may share results from same methods.

(2) The results of Line 2 in table 3 and Line 2 in table 4 are very similar, please have a check.

(3) the fusion strategies could be clearly defined for the ease of reference, e.g., simply adding PV- and ART-features at four convolution blocks.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Overall, the paper proposes a new LiTS method which considers multi-phase information fusion, as well as uncertain region refinement, achieving promising results.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Review #3

Please describe the contribution of the paper

The paper presents a new segmentation method for segmenting the liver tumor from multi-phase images. The core innovation lies in an attention-based feature aggregation module as well as an uncertain region inpainting module. The superiority of the approach is demonstrated by comparing against a series of state-of-the-art multi-phase liver tumor segmentation algorithms.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Paper is well organized and clear. Using the spatial aggregation module for fusing the image features across phases, followed by an uncertainty region inpainting module has been well motivated for multi-phase liver tumor segmentation from CT.
- The experimental results compared with a few competitors are promising. Ablation on each component also suggests that both SAM and URIM are effective for this task.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The data acquisition part is missing. According to my understanding, the approach should require aligned inputs for training/testing. But how to obtain aligned input is not illustrated.
- Though a few competitors on CT tumor segmentation are compared, there are more related former studies, e.g., [a][b], for multi-phase CT segmentation which are not discussed. Please properly discuss the differences with these other approaches dealing with multi-phase CT in the introduction/methodology part.
[a] Raju, Ashwin, et al. “Co-heterogeneous and adaptive segmentation from multi-source and multi-phase CT imaging data: a study on pathological liver and lesion segmentation.” ECCV 2020. [b] Zhou, Yuyin, et al. “Hyper-pairing network for multi-phase pancreatic ductal adenocarcinoma segmentation.” MICCAI 2019.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The code is not provided in the submission. The authors are strongly recommended to release the code for reproducing the results in the paper.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Please see above.

It will also be nice to include some analysis regarding the failure cases.

Also, is would be interested to see how URIM refines the outcome in qualitative examples.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Though there are a few minor issues (e.g., missing references), overall the paper presents a nice approach with strong results. Given the importance of the topic and the clarity of the approach, I think this paper should be accepted. Meanwhile, I strongly recommend the authors release the training/testing code, and address the minor issues properly in the next version.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

5
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

Effective feature aggregation across phases/modalities is an important problem in medical imaging. This paper proposed an interesting feature aggregation approach, along with a segmentation refinement strategy for uncertain regions. All reviewers agreed that the paper was well motivated, clearly written, and that experiments well supported the proposed innovations.

I encourage the authors to address reviewer comments. Particularly R3’s question on how you would address using four phases instead of two, which is the most common DCE CT configuration. It seems like you would have now four Siamese streams and feature aggregation would have to be conducted across 4 choose 2 =6 connections. What is the impact on complexity? Is it feasible to use this approach with full DCE CT?
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

3

Author Feedback

Dear Reviewers and Meta-reviewer, We truly appreciate your positive and constructive feedbacks. We will carefully incorporate them in the final version including fixing typos, adding detailed data acquisition and discussion on feature fusion among three or more phases (space permitting). Reviewer #3 commented that the dataset we adopted is not publicly accessible and suggested us validating our method on public datasets. Regretfully, there are not publicly available datasets of multi-phase abdomen CT images yet. We could only collect in-house datasets and conduct our experiments. Reviewer #5 pointed out that we missed discussions on two more related works. We agree that the reference [a] is very relevant to our work and we will supplement the discussions in the Introduction part. However, the reference [b] focused on pancreatic ductal adenocarcinoma segmentation, which is not so relevant (actually, all the competition methods discussed in our manuscript are about multi-phase liver tumor segmentation). Sincerely, Authors

back to top

Multi-phase Liver Tumor Segmentation with Spatial Aggregation and Uncertain Region Inpainting