Paper Info Reviews Meta-review Author Feedback Post-Rebuttal Meta-reviews

Authors

Rohit Jena, Sumedha Singla, Kayhan Batmanghelich

Abstract

Vessel segmentation is an essential task in many clinical applications. Although supervised methods have achieved state-of-art performance, acquiring expert annotation is laborious and mostly limited for two-dimensional datasets with a small sample size. On the contrary, unsupervised methods rely on handcrafted features to detect tube-like structures such as vessels. However, those methods require complex pipelines involving several hyper-parameters and design choices rendering the procedure sensitive, dataset-specific, and not generalizable. We propose a self-supervised method with a limited number of hyper-parameters that is generalizable across modalities. Our method uses tube-like structure properties, such as connectivity, profile consistency, and bifurcation, to introduce inductive bias into a learning algorithm. To model those properties, we generate a vector field that we refer to as a flow. Our experiments on various public datasets in 2D and 3D show that our method performs better than unsupervised methods while learning useful transferable features from unlabeled data. Unlike generic self-supervised methods, the learned features learn vessel-relevant features that are transferable for supervised approaches, which is essential when the number of annotated data is limited.

SharedIt: https://rdcu.be/cyl2t

N/A

Reviews

Review #1

• Please describe the contribution of the paper

This paper proposes a novel self-supervised training method that is designed for vascular structure segmentation. The method is applicable for many tubular structures. The efficacy was validated with several kinds of open datasets including 2D retinal vessel, 3D CT / MRI. The performance comparisons with the conventional methods show that the proposed method can achieves higher scores in DICE metrics.

• Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Novel formulation: The method models vessels with a few parameters of template image, radius, flow direction, and so on. The U-net like neural networks are trained to predict those parameters without labels by minimizing the similarity loss (Normalized cross correlation) between the training image and the predicted image.

Segmentation performance: The advantage of the method is that the method can achieve a high DICE score without labels, since it can model tubular structure’s thickness or extent, compared to the conventional methods. A wide range of vascular applications regarding volumetry purposes would benefit from this research.

• Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

It is not clear that the method is also applicable for diseased vessels with different appearances, for instance clogged vessels with soft / hard plaques. The method seems to fail segmenting such vessels, because it uses a template image representing an ideal vessel (disk/tube). Centerline detection performance is not improved significantly.

• Please rate the clarity and organization of this paper

Good

• Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The idea is clearly given in the paper. The reproducibility seems relatively high. Furthermore, the authors show wills to provide the codes in public.

• Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

The approach is unique, but its potential is limited by using a few parameters and single template image. It is interesting to provide the authors’s opinions about using multiples template images, or trainable (parameterized) template.

Probably accept (7)

• Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper proposes a novel and general approach for vascular structure segmentation. A wide range of vascular applications would benefit from this work.

• What is the ranking of this paper in your review stack?

1

• Number of papers in your stack

5

• Reviewer confidence

Very confident

Review #2

• Please describe the contribution of the paper

Propose a self-supervised method for vessel segmentation taking consideration of tube-like structure properties such as connectivity, profi le consistency, and bifurcation. It is generalizable across modalities.

• Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Claimed to be first unsupervised deep learning method that takes a raw image as input and outputs per-pixel vessel statistics as output. A clever way to incorporate vessel properties in the design but not directly as hand crafted features. Comprehensive evaluation on multiple datasets, both 2D and 3D, with multiple modalities.

• Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Unclear descriptions on key components, causing confusions for readers. Possible design flaw for the loss. Unclear value of this method compared with existing supervised methods which have much better performance

• Please rate the clarity and organization of this paper

Satisfactory

• Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Public dataset. No link for their code provided in the manuscript.

• Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

borderline accept (6)

• Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Interesting and explorative ideas, but lack of clear explainations and maybe flaws in design and experiments.

• What is the ranking of this paper in your review stack?

2

• Number of papers in your stack

5

• Reviewer confidence

Confident but not absolutely certain

Review #3

• Please describe the contribution of the paper

This article proposes an unsupervised vessel enhancement filter using self-supervised learning. This approach is based on geometric properties of the vessels such as profile consistency, connectivity and bifurcation. This approach provides three very interesting features to characterize blood vessels in the image: the vesselness (probability of a pixel to belong to a vessel), the direction of the vessels and the radius of putative vessels at each point. Extensive experiments have been conducted both in 2D and 3D images and comparisons with classic vessel enhancement strategies have been performed.

• Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

I thank the authors for this interesting article. Unsupervised vessel enhancement is of great interest for vascular-related clinical applications and, to the best of my knowledge, it is indeed the first unsupervised deep learning-based approach in the literature. The article is well written and the method is clearly explained. I also appreciated the extensive experiments that have been conducted on several datasets.

• Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

My main concern regards how the authors set the parameters of the compared methods. This can dramatically affect the comparison results. I am surprised by the results obtained for Frangi in Figure 3 (d). The Frangi filter usually yields many false positives due to border issues or close curvilinear structures. However, in the results shown on Figure 3 (d), the errors seem to come from a large overestimation of the size of the structures, as well as from detections of large structures that are not curvilinear at all. I am not used to seeing that in Frangi results. In Figure 4 in the supplementary materials, Frangi detects bright structures whereas the structures of interest are dark. Normally by choosing the sign of the eigenvalues and the contrast parameter correctly this should be prevented. The optimization strategy for setting the parameters of the compared methods should be explained to convince the reader that a fair comparison was performed.

• Please rate the clarity and organization of this paper

Very Good

• Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

I think the method could be reproduced, however the authors did not provide all the hyperparameters they used (see detailed comments)

• Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
• Regarding the profile consistency, the authors should explain in more details what their resizing transform $\Phi(u,r)$ is, as it seems that this is a crucial point of the method. This transform seems to be position-dependent, as the value of r can be different for every pixel. How is it applied in practice ? How is it included in the UNet architecture ?
• In Equation 2, I do not understand why the authors specify $\dot{q}_p = u(p)$. It seems that this relation is not used in the equation nor later in the paper. If this is important, please clarify, otherwise please remove.
• It is not clear for me why $b_1(p) = b_2(p) = -u(p)$. Why is there a minus sign ? The associated sentence is also not very clear. Please rephrase it: “Note that, in the absence of bifurcation, $b_1 (p) = b_2 (p) = −u(p)$ minimizes the same loss as $L_m$ . in the opposite direction of the vessel flow”.
• The authors should specify the value of the loss hyperparameters $\lambda_1$ and $\lambda_2$ they used. I assume that the results highly depend on these parameters. Therefore, a discussion on how to set them would be interesting.
• The authors used a batch size of 1. Is this because of memory issues ? Or did they obtain the best results with this value ? Did they observe training instability ?
• The authors should consider performing cross validation to improve the statistical significance of their results.

accept (8)

• Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The method is interesting and new. Even though I am not convinced that the comparison with other methods is totally fair, I still think the approach is interesting for the community.

• What is the ranking of this paper in your review stack?

1

• Number of papers in your stack

3

• Reviewer confidence

Confident but not absolutely certain

Primary Meta-Review

• Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

This paper presents a method for vessel enhancement in medical images. The method uses self-supervised learning and is developed based on the geometric properties of the vessels. Overall, the method is interesting. The method has been compared with the closely related methods. The reviewers are all positive about the paper. After reading the comments raised by the reviewers, my recommendation is given in Q3.

• What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

3

Author Feedback

We thank the reviewers for their detailed feedback. Here are some common concerns: R1: “It is not clear that the method is also applicable for diseased vessels…” R2: “Assuming vessel profile as a unit disk, …”

A: This is a valid limitation of our method. There is only so much we can detect using a single template, our idea behind the paper is to show that vessel properties can be “baked” into a neural network using novel loss functions without ground truth. Further work can use sparse amounts of labeled anomalies which can be used to extend our method to increase its detection for vessel segments that do not conform to the template.

R1:

“The approach is unique, but its potential…” A: we certainly considered experimenting with different templates, or alternatively, using a richer set of parameters (for non-rigid deformation of the template) for each pixel instead of a rigid scaling and rotation of the template, thus allowing a richer set of vessels. However, without ground truth data, the networks with so many parameters are susceptible to overfitting by “hallucinating” other regions of the image as vessels. Using a sparse amount of ground truth to tackle this problem can be the basis for future work.

R2: “There are three directions at a bifurcation … interchangeable during prediction.”

A: There is no “preference” towards “u” being the larger branch. Consider three points each on the 3 splits near the bifurcation, each point should have its corresponding split as u and the other two branches as b1 and b2. This is done to extend the “template” at the bifurcation locations because a single template gives a poor response due to the branches not overlapping with the bifurcation.

“Profiles at opposite directions …”

A: We do this by construction. In 2D, we output half-angles \theta_1 \in [0, \pi/2] and \theta_2 \in [-\pi/2, 0] and rotate -u by these angles, making b1 and b2 face opposite to u. In the case of 3D, we multiply b1 and b2 by the sign of dot products <b1, -u> and <b2, -u>.

“Fig 1 is the most important figure in this paper, but it is misleading …”

A: The figure is illustrative. 1) In principle, any encoder-decoder architecture which outputs maps of the same spatial dimensions can be used 2) We used this diagram rather than a per-pixel heatmap to show what we’re trying to predict. A per-pixel prediction will be the most accurate where it matches the template (centerline of the artery) resulting in the maximal vesselness at those locations and gradually falling off. A suitable threshold on this vesselness gives a segmentation. 3) Yes, the outputs are of the same size, we used subsampling and quiver plots to illustrate the different outputs we’re predicting (radius, arrows for directions).

“Fig 3 (a) why no false … in the design?” A: Adding another color made the figure cluttery, so we stuck with TPs and FNs, the both of which make up the ground truth. Any of the unsupervised methods do not have any method to penalize FPs. To select a threshold, we use a small validation set and find the threshold which maximizes the dice score on the validation set.

“The section of “efficacy of the representation” … fig 3.” A: Due to space constraints, we were unable to include all qualitative results.

R3: “My main concern regards how the authors set … A: We use a standard implementation of the Frangi filter [1]. To convert the continuous-valued output to a binary threshold, we find the best threshold using a small validation set on which we maximize the dice score. For Figure 4, Frangi detects regions which are dark and surrounded by bright regions, which looks like a ridge (see FPs on left side of Figures 4(c)).

• “The authors should specify the value of the loss hyperparameters…” A: In our case we set $\lambda_1 = 0.5, \lambda_2 = 1$. Due to space constraints, we didn’t include an analysis on hyperparameter setting.

[1] https://scikit-image.org/docs/dev/api/skimage.filters.html#skimage.filters.frangi