Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Shiyu Mi, Qiqi Bao, Zhanghong Wei, Fan Xu, Wenming Yang

Abstract

Stroke is one of the leading causes of death around the world. Segmenting atherosclerotic plaques in carotid arteries from ultrasound images is of great value for preventing and treating ischemic stroke, yet still challenging due to the ambiguous boundary of plaque and intense noise in ultrasound. In this paper, we introduce a new approach for carotid plaque segmentation, namely Multi-Branch Feature Fusion Network (MBFF-Net). Inspired by the prior knowledge that carotid plaques generally grow in carotid artery walls (CAWs), we design a Multi-Branch Feature Fusion (MBFF) module with three branches. Specifically, the first two branches are well-designed to extract plaque features of multiple scales and different contexts, and the other branch is to exploit the prior information of CAWs. In addition, a boundary preserving structure is applied to alleviate the ambiguity of plaque boundary. With the proposed MBFF and the novel structure, our model is capable of extracting discriminative features of plaques and integrating the location information of CAWs for better segmentation. Experiments on the clinical dataset demonstrate that our model outperforms state-of-the-art methods. Code is available at https://github.com/mishiyu/MBFF.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87240-3_30

SharedIt: https://rdcu.be/cyl54

Link to the code repository

https://github.com/mishiyu/MBFF

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The authors proposed a Multi-Branch Feature Fusion Network (MBFF-Net) for segmenting atherosclerotic plaques in carotid arteries from ultrasound images. The multi-branch design incorporates multiple scale features, different contexts through dilated convolutions, and used prior segmentation of artery wall as the guidance. Also a boundary preserving structure was used to enhance the plaque boundary. The method claimed to achieve state-of-the-art performance.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Fusion of multiple features, either from different resolutions or human prior knowledge about plaque locations Ablation study for proving the effectiveness of each model 430 images from different ultrasound machines (made public?)
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Comparing with weak existing methods on segmentation No analysis on learned feature map for model interpretability
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Code is available in Github
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

The segment region is very thin, is that necessary to use 4 times of down sample to rescale the feature map from 128128 to 88? Additional ablation study using fewer scales might be considered. No further analysis on feature map. It is hard to know interpret the behavior of these feature extractor. For example, whether the model is learning from the correct region of interest from the image patch. The boundary preserving module is unclear. Please define what is plaque boundary and what is the difference between plaque region. How to extract the plaque boundary from the shallowest layer? What is “the first layer”? In figure 1, it is also helpful to define which part belongs to boundary preserving. Orange lines? MSF was generated from various scales of SSFs, then combined in the MBFF with SSF of each scale for segmentation. Finally the results in various scales were averaged. Is the segmentation result from low resolution equally good as the result from high resolution? If yes, why need 4 MBFF in each scale? If not, is that risky directly averaging good and bad results? Comparison methods are all older than 2019. Considering the multi-scale feature map is used, comparisons should be focused on methods using similar structures. But only PSPNet (2017) has similar multi-scale design.
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

A minor innovation on plaque segmentation taking consideration of multi-scale features and wall segmentation. Experiements are sound but could be more comprehensive.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

This paper presents a method called “Multi-Branch Fusion Network” to do carotid plaque segmentation on ultrasound images which also overcomes the ambiguous boundary of the carotid plaque. The proposed method combines multi-scale, multi-context features of the carotid plaques on ultrasound images which produces robust segmentation and outperforms existing state-of-art methods.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The proposed method shows a framework of extracting carotid plaques from the ultrasound images which could be easily applied in the real world applications. The input of the method is the carotid longitudinal section images, and the output will be the segmented plaque.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The proposed nerual network is more like a combination of different components, and the inference time using the model might be long.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Overall, the paper has a high reproducibility.
1. The authors shared their codes on GitHub, and based on the reproducibility checklist that the authors also prepared “readme” in their GitHub repository to help the reader
2. The authors describes details of their training/testing and experimenting in “Experiments” section.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
Thanks the authors for their nice work. 1.Fig.2 gives examples of the IM detection results which will be used as a initial step for further carotid plaque detection. But I do see that in the example images that there are measurements existed. 1.1I am wondering whether the images used in the training for IMD-CNN contain the measurements? 1.2New datasets have been collected for the presented study, are the measurements included in the new datasets as well? 1.3From Fig2, it seems that the False Positive is quite large, do you think this will have large impact on the following plaque segmentation network?
1. On Page6, section 3 “Dataset”, the authors describe the preprocessing steps for the training images. Specifically, the original images are cropped into 256x256 ones as the input for the network. My concern here are (1) Do you crop the images around the plaques? (2) How to deal with the cases that contain several plaques? (3) How many cropped images will be generated from the original image(size 768x576) (4) are there augmentation applied during the training?
2. Line 3 of Section 3 “Evaluation Metircs”, Page6, Please fix the typo of “Prediction” which seems to be “Precision”
3. Table1 lists the ablation results of different components which shows the efficiency of the proposed components. It will be very interesting to see how the results improved through example images, especially how the boundary preserving performs since boundary ambiguous has been known as a challenge for the plaque segmentation or similar tasks.
4. The networks seem to be very large, but it is comforting that each epoch only takes around 40seconds. Could you give the estimation of the parameters to be trained?
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- Overall, the paper is well written. Methods are clearly described. Several experiments were conducted to show the effectiveness of the network components. The results from the experiments are well organized and fairly explained.
- The presented framework may easily be used in real applications to facilitate the automated diagnosis.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Review #3

Please describe the contribution of the paper

MBFF-Net is proposed for automatic carotid plaques segmentation in CLS ultrasound images. Proposed model is well-designed based on pathological knowledge of plaques and achieves discriminative feature extraction and segmentation performance.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

This work is an incremental work of the authors previous works. The authors from their previous work (IMD-CNN) use the segmentation mask as a prior knowledge for the current work. The novelty lies in MBFF module which use prior information, fusing the features of multi-scales and multi-contexts. The paper also presents a boundary preserving structure to enhance the plaque boundary, which diminishes the influence of blurred boundaries of plaque in ultrasound images.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Not applicable
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The training and testing split have been defined clearly so that there will be no bias in the evaluation.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

• The authors have mentioned about dice score and IOU, are they not the same?
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The proposed work is clinically significant and the overall paper is technically strong.
What is the ranking of this paper in your review stack?

3
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

This paper presents a method for segmenting atherosclerotic plaques in carotid arteries in ultrasound images. The method uses a multi-branch feature fusion network for the segmentation of carotid plaque, which utilizes multi-scale and multi-context features. The results are promising. The reviewers are all positive about this paper. After reading the comments raised by the reviewers, my recommendation is given in Q3.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

4

Author Feedback

Dear Reviewers, Thank you for your time and effort in reviewing our paper. We are pleased that you accept our work and point out some problems to help us improve the quality of our work. Motivated by your comments, we are trying to solve some problems you mentioned. However, we also find some misunderstandings. Thus, we make the following explanation: Ultrasound acquisition device generates ultrasound images surrounded by irrelevant parameter information. The resolution of the original images is 768x576. We crop the ultrasound images from the original images and then rescale them to the size of 256x256. Note that ultrasound images are completely preserved and the number of images remains the same. In addition, the dataset used to detect carotid artery walls shown in Fig. 2 is different from that used in this work. Thank you again for your suggestions.

back to top

MBFF-Net: Multi-Branch Feature Fusion Network for Carotid Plaque Segmentation in Ultrasound