Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Hongwei Zhang, Dong Zhang, Zhifan Gao, Heye Zhang

Abstract

Joint segmentation and quantification of main coronary vessels are important to the diagnosis and intraoperative treatment of coronary artery disease. They can help clinicians decide whether to carry out coronary revascularization and choose the interventional stent. However, joint segmentation and quantification in a framework is still challenging because of intrinsic distinction of optimization objects for these two tasks. In this paper, we propose a dual-branch multi-scale attention network (DMAN) to achieve synergistic optimization process in a framework. Our DMAN consists of a nested residual module and a attentive regression module. The nested residual module is used to extract and aggregate multi-level and multi-scale features. The attentive regression module introduces a two-phase attention block to express interactive correlation of separated regions and capture the informativeness of the important region in the image. Our DMAN is evaluated over 1893 X-ray coronary angiography images collected from 529 subjects. DMAN achieves the dice coefficient of 0.916 for segmentation and the MAE of 1.30 ± 0.62 mm for quantification.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87193-2_35

SharedIt: https://rdcu.be/cyhMd

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The paper developed a joint network for segmentation and quantification of main coronary vessel. Nested residual module is introduced in the network and two attention indices helps to focus on small lesion features by weighting importance of different features.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Joint segmentation and quantification is clinically relevant
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Two main contribution of the paper residual block and attention are not evaluated. There are many claims in the paper which are not supported by experiments: multi-scale segmentation using U-block. ‘This makes our RSUB more capable of perceiving richer local and global features’
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

I have no comment on this
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

1) Design ablation study with commonly used residual blocks and evaluate the multi-scale property of proposed Ublock Does segmentation performance increases with ONLY adding this block in compare to others? 2) Ablation study for evaluting proposed attentive modules 3)Attentive quantification is not clearly written
Please state your overall opinion of the paper

reject (3)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper is not clearly written, the claims are not supported by experiments
What is the ranking of this paper in your review stack?

4
Number of papers in your stack

4
Reviewer confidence

Confident but not absolutely certain

Review #2

Please describe the contribution of the paper

oint segmentation and quantification of major coronary vessels is important for the diagnosis and intraoperative treatment of coronary artery disease. They can help the clinician determine whether to perform coronary artery revascularization and interventional stents. In order to improve the performance of these tasks, the authors proposed a model that performs joint segmentation and quantification of major coronary vessels at the same time. In addition, stable performance was induced by adding a function that provides attention to various scales to the model. Through this, it provides stable performance compared to the existing method and enables measurement of not only segmentation but also quantification steps of blood vessels at the same time.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The authors proposed a deep learning network that simultaneously divides and quantifies the joints of coronary vessels, which are important for diagnosis and treatment of coronary artery disease. Also, a multi-attention-based model was applied to the proposed network. This provides stable performance compared to the conventional method, and it is possible to measure not only the segmentation but also the quantification stage of blood vessels at the same time. Multi-task learning, which simultaneously performs several roles for classification purposes, has been studied a lot, but networks that simultaneously perform different purposes of segmentation and quantification have been studied relatively little. In this paper, such fusion was successfully performed and stable performance improvement was confirmed.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- A network in which several factors were fused was proposed, and it would have been good to analyze how each proposed factor had an effect on the performance.
- In Table 1, even when the proposed method has the worst performance, it is indicated in bold. Besides that, there are often areas of lack of consistency.
- It is difficult to confirm whether the comparison was fair because the implementation and learning environment of the methods used for comparison were not accurately indicated.
- Compared to conventional methods, more information is required in the learning context. However, it did not show a significant difference in segmentation compared to the existing performance, but it is not clear whether the difference is clinically meaningful. fig. From the results in 3, it is difficult to confirm clearly clinically important performance improvement compared to the existing method.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
- A network in which several factors were fused was proposed, and it would have been good to analyze how each proposed factor had an effect on the performance.
- In Table 1, even when the proposed method has the worst performance, it is indicated in bold. Besides that, there are often areas of lack of consistency.
- It would be good to describe exactly how the other algorithms used in the comparison experiment were implemented and which data were learned. This seems to be able to confirm whether a fair comparison has been made.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- Quantification is information that can be evaluated subjectively compared to segmentation. It would be better to analyze the problems that may arise from these differences.
- When displaying experimental results, etc., more consistent and reliable information should be provided.
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

A good attempt was made by using data that can be constructed naturally in a clinical environment. However, it would have been better if I analyzed the results more technically and organized the results well. The frequent inconsistency is lowering the trust of the paper itself.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

3
Reviewer confidence

Very confident

Review #3

Please describe the contribution of the paper

The task this paper tries to solve is the segmentation and quantification of stenotic lesions in the coronary vasculature from coronary X-ray scans. The method they propose to solve this task is a “dual-branch multi-scale attention network”, which consists of: -A set of U-net like architectures which form a U-net architecture of their own in order to perform multi-scale segmentation -An attentive regression module which shares the encoder with the encoder part of the “high level U-net” and merges the features from the skip connections using two attentive modules.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

-The paper is well structured -The concepts in this paper are novel, interesting and could be applied in other areas -Evaluation of the method was performed exhaustively by also implementing and testing related work algorithms on the data set at hand -In general an overall performance boost can be observed compared to related work
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

-The figure describing the method is very convoluted and the signal flow is really hard to comprehend with arrows flying left and right and top and bottom everywhere. It is hard to assess the method from it paired with the description. Since Figure 1 is mainly motivational and only partially cited in the text itself, it would help the paper to have a better comprehensible figure of the overall workflow.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The data set used in this study is private and the architecture proposed is pretty advanced which might hinder reproducibility without publicly releasing code.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

-As described in S4 please rework Fig. 2 -Please comment on whether the cross-validation was performed patient-wise -A stenosis does not necessarily always lead to myocarial infarction. Please consider inserting a “may” in the sentence “The serious absence of blood flow in the main coronary vessels jeopardizes a large mass of myocardium even leads to myocardial infarction” -There are some blank spaces missing especially before brackets throughout the document -Some of the references do not follow the guidelines regarding the amount of authors that should be included
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper presents some interesting new concepts, which are well evaluated. A drawback is the description of the method, especially with the used figure. However, I see no problem with fixing this for a camera ready version.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

7
Reviewer confidence

Somewhat confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

This is an interesting manuscript, with a novel and potentially valuable contribution to the field that has some flaws limiting its value. Clarity has been addressed as a weak point by the reviewers, specifically because some of the claims are not fully supported by evidence. No sufficient details for the learning process are provided, this should be properly addressed. Also, the clarity of the figures is not great, sometimes being cluttered and making it difficult to map the figures and the methodology.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

6

Author Feedback

-Contribution The meta-reviewer praises that our work is “interesting” and has “a novel and potentially valuable contribution to the field”. R5 also agrees that “the concepts are novel, interesting and could be applied in other areas”.

R1 misinterprets our main contributions as residual block and attention. However, our main contributions are as follows: (1) A clinical tool to provide the morphological structure and stenosis indices of main coronary vessels simultaneously. The information indicates the lesion severity visually and quantitatively to help the diagnosis and treatment. Its importance is verified in many medical literatures. R3 also commends that our method “can help the clinician determine whether to perform coronary artery revascularization and interventional stents”. (2) The first attempt to fuse the segmentation and quantification in the X-ray coronary angiography analysis. Our method utilizes their mutual promotion. The segmentation helps the location of stenosis lesions to boost the quantitative accuracy. Meanwhile, the quantification forms a constraint to improve the segmentation performance. R3 praises our work as “a good attempt” and comments that “the networks that simultaneously perform different purposes have been studied relatively little”.

Moreover, the residual block and attention are the effective implementation of the above thoughtful ideas. First, the residual block learns the local details and global contrast information by utilizing multi-scale and multi-level features. Second, the attention mechanisms focus on small lesion features by weighting the relationship of the separated regions. R3 and R5 praise them: “A function that provides attention to various scales to the model” (R3); “An attentive regression module merges the features using two attentive modules” (R5).

-Validation R1 asserts that “many claims are not supported by experiments”. However, R5 praises that the experiment has performed “exhaustive evaluation” and it can support many key claims. First, the stability is presented through the patient-wise cross-validation on abundant samples (1893 images from 529 subjects). Second, the validity is shown by separate evaluation on three kinds of vessels (LAD, LCX, and RCA). Third, the method superiority is demonstrated in comparison with 5 segmentation methods and 4 quantification methods. Fourth, the consistency is shown in the analysis of Bland-Altman plots. The result figures and tables jointly support our claims. R3 praises that our method “provides stable performance”. R5 also commends that “the interesting new concepts are well evaluated”.

R3 improperly comments that “the learning process lacks details” in the experiment. (1) R3 doubts the fairness of the comparison experiment. However, the comparison is fair. The contrast methods are implemented as described in their papers. Their environment and data are the same as ours. The specific deployment is described detailedly in Section 3.1. (2) R3 misunderstands that our results are the worst in Table 1. However, our results are the best or very close to the best ones. “The worst performance” (R3) does not conform to the results. (3) R3 doubts that “not significant difference in segmentation” is meaningless. However, the results are clinically meaningful. First, other methods more or less segment out the non-main vessel regions. This harms the clinical analysis and operations. Second, the comparison results are statistically significant because P-value is 0.037.

-Method R1 misunderstands that “Attentive quantification is not clearly written”. However, R5 praises that our paper “is well structured”. We have described it adequately in Section 2.3, including implementation purpose, module function, fundamental formulas, etc.

R5 feels the figure is a little unclear but also praises that he “sees no problem with fixing this for a camera-ready version”. We have improved the image quality and details.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The response of the authors t the AC and reviewers comments is, somehow, bittersweet. They do provide additional explanations to some of the pitfalls and questions of the reviewers, but still trying to highlight disagreement between reviewers, by trying to put them “against each other” in some way. The aim of the review process is improve and enhance the quality of scientific work, the huge review effort should not be neglected by the authors and, even if accepted or rejected, the reviewers work will certainly be valuable to the authors to improve further improve their work. The paper has merit, yet is not crystal clear that deserves publication in MICCAI.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

10

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The paper proposes a method for joint segmentation and quantification of the main coronary vessel. The method is interesting. There are concerns related to the claims which are not supported by experiments, method description, and comparison with conventional methods (more info is needed). Most of the concerns have been addressed in the rebuttal.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

7

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

While the topic is clinically relevant and the paper is well-motivated, the main criticism of the paper are regarding the lack of evaluation of the individual main contributions of the paper, the methodology not being supported by experiments and lack of clear performance improvement against existing methods in the results, missing details regarding validation experiments, and organization and consistency of the results. While some of the concerns regarding the main contributions of the paper are addressed by the rebuttal, other concerns regarding validation experiments and results remain.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Reject
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

17

back to top

Joint Segmentation and Quantification of Main Coronary Vessels Using Dual-branch Multi-scale Attention Network