Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Dewei Hu, Can Cui, Hao Li, Kathleen E. Larson, Yuankai K. Tao, Ipek Oguz

Abstract

Optical coherence tomography (OCT) is a non-invasive imaging technique widely used for ophthalmology. It can be extended to OCT angiography (OCT-A), which reveals the retinal vasculature with improved contrast. Several recent deep learning algorithms have yielded promising vascular segmentation results; however, 3D volumetric retinal vessel segmentation remains difficult due to the lack of manual annotations for training. To tackle this challenge, we propose a learning-based method that depends only on the supervision provided by a self-synthesized modality named local intensity fusion (LIF). LIF is a capillary-enhanced volume computed directly from the input OCT-A. We then construct the local intensity fusion encoder (LIFE) to map a given OCT-A volume and its LIF counterpart to a shared latent space. The latent space of LIFE has the same dimensions as the input data and it contains features common to both modalities. By binarizing this latent space, we obtain a volumetric vessel segmentation. Our method is evaluated in a human fovea OCT-A and three zebrafish OCT-A volumes with manual labels. It yields Dice score of 0.7736 and 0.8594 +/- 0.0275 on human and fish data respectively, a dramatic improvement over existing unsupervised vessel segmentation algorithms.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87193-2_49

SharedIt: https://rdcu.be/cyhMs

Link to the code repository

https://github.com/DeweiHu/LIFE

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The authors proposed a deep-learning-based method for 3D vessels segmentation from OCT-A volumes. The authors introduced the LIF ( a JIF variant) as performing fusion between the 2D en-face slices. The authors proposed a self-supervised method (LIFE) to produce a 3D OCT-A vessel segmentation.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

– The proposed method requires neither inter-volume registration nor extra image acquisitions to train the unsupervised-based vessels segmentation model by the benefits of the LIF.

– The proposed method extends the unsupervised-based vessel segmentation from 2D OCT-A images to 3D OCT-A volumes in a novel way.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

– The role of the CE-LIF in the LIFE is unclear.

– Lack of evidence or discussion about LIF can exploit local structural information to enhance small features.

– The segmentation output of LIFE is influenced by the thresholding method as the final results. Phantom vessels may appear again.

– Fig.1 very hard to follow. Confused about inputs and outputs for each network.
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

I am quite confident it will be easy to implement the method given the detailed descriptions.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Please explain the role of the CE-LIF in the LIFE more clear.

Please elucidate why LIF can exploit local structural information to enhance small features.

It might be useful to redo Fig.1.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

– The proposed method requires neither inter-volume registration nor extra image acquisitions to train the unsupervised-based vessels segmentation model by the benefits of the LIF.

– The proposed method extends the unsupervised-based vessel segmentation from 2D OCT-A images to 3D OCT-A volumes in a novel way.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

The paper proposes a self-supervised learning method for retinal vessel detection in OCT-A images in which the supervision is from adjacent slices of the OCT-A volume. The proposed method is compared with several unsupervised learning methods and shows improvements over prior works.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The proposed method is annotation-efficient. It does not require manually labeled OCT-A vessel maps nor multiple scans from different imaging devices.
2. The proposed method works on 3D OCT-A volume rather than a single depth-projection image.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. The writing needs improvement. Notations are not consistent and sometimes cause confusion.
2. The evaluation is limited and a bit weak.
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Overall the paper includes implementation details and dataset descriptions. It would be better to mention the evaluation metrics in the main text as well.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

One of the benefits of using LIF is that it does not require the registration between input OCT-A slices. However, as mentioned in Section 2.1, the Greedy software is used to perform registration. Please clarify this.

The writing needs improvement. Some notations are not consistent or defined, making it hard to understand the technical details. In Eq (2), what are Y and Y’? In the paragraph above, only a single OCT-A volume X and its LIF is used. The symbol Y refers to the target image in the JLF method.

Training loss is incomplete. Eq. (2) shows the regularization terms. What are the loss function of VAE used in this paper? In addition, the denoise network, Dn-Net, is supervised by LIF? What loss function is used to train Dn-Net? Why Dn-Net is supervised by LIF? It seems that the goal of Dn-Net is to generated LIF from the input image.

For LIF, how are the weights map calculated? This is the core component in the proposed method and thus has to be explained clearly.

The paper claims, in Section 2.3, that no supervised method is applicable because of the lack of labeled data. There are some public datasets for OCT-A vessel segmentation, such as the ROSE and the OCTA-500 datasets. While it is not completely fair to compare the self-supervised method with the supervised method, it is still worth including the results of supervised method. It will help understand what the gap is between those two types of learning approaches. The paper only tests the method on a very small dataset, which is not sufficient to support the conclusion.

For the baseline methods, please include more implementation details. For example, how do you perform K-means for this tasks.

(minor) Two rectangles in Fig. 2 are too small and are hard to see in the printed environment. Please consider adding the zoomed-in views.
Please state your overall opinion of the paper

borderline reject (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

While the overall idea of self-supervised learning is interesting, some techinical details are missing and the experiments are limited.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Review #3

Please describe the contribution of the paper

The proposed method presents an interesting learning framework of achieving retinal OCTA vessel enhancement and segmentation based on unlabeled data. Different techniques including image registration, speckle denoising and motion artifacts removal are integrated to improve the overall performance.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The proposed idea of using a self-synthesized modality to supervise the learning process is interesting.
2. Several interesting pre-processing techniques are employed to improve the OCTA image quality.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. The methodology presentation needs to be improved.
2. From methodology point of view, it is not clear what are the main novelties in the designed framework. It seems like the method is established by different existed modules. Thus, the authors should clarify the critical technical contributions.
3. The experimental validations is only limited to the regions around fovea.
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The reproducibility of this paper is restricted by the very limited details of several pre-processing steps, such as the motion artifacts removal and the registration settings, which make those steps difficult to reproduce.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
1. Besides the self-supervised part, why do you think LIF and LIFE is suitable for 3D OCTA vessel fusion and segmentation. Is there any motivation from the combination of technical and data aspects. This point should be better explained.
2. Section 2.1 is a bit confusing. For introduction purpose, I would suggest the authors to put those contents in the related work section. Otherwise, it will first give readers an intuition that those techniques would be directly used as the main steps of your whole framework. This makes the current presentation not straightforward.
3. The authors proposed a new approach instead of direct use of the LIF and CE-LIF. Would it be better to compare the performance of the proposed model with those baseline models, or they are not comparable? However I didn’t see that in the experimental section.
4. Section 2 is named as cross-modality feature extraction. However, the authors already indicated that fact that imaging same retina with different devices are not practical and challenges may exist between OCT devices for registration. That’s why the authors propose a single OCT-A volume and its LIF to be used in the new framework. Thus, I don’t think it is a cross-modality approach anymore. So the title of section 2 is not precise.
5. The preprocessing for motion artifact removal is an interesting part and thus is worthy of giving more details of how to efficiently remove motion artifacts.
6. Not sure if it is a good idea to use CE-LIF to supervise the R2U-Net. If CE-LIF already shows phantom vessels as false positives after incorrect fusion as illustrated in Fig. 2, how would it be a reliable label for supervising the network?
7. I understand the point for comparing with supervised learning is difficult at this moment. However, the authors are encouraged to compare with learning based approaches in their future work.
8. I am not so sure if unsupervised Frangi filter and OOF can perform so badly. From the presented results it seems that the multi-scale parameter tuning may be not sufficient.
Please state your overall opinion of the paper

borderline reject (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Technical and experimental contributions.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

4
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

This paper proposes a novel self-supervised method for OCTA vessel segmentation. There is consensus from the reviewers about the novelty of the proposed method. Most questions from the reviewers require clarification about the role for the CE-LIF and LIFE, loss function used in the VAE, and other aspects of the proposed method. The experiments were also limited to regions around fovea.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

4

Author Feedback

Thank you for the careful review and the praise on the novelty of our approach. We appreciate that even R3 who asked for some clarifications has ranked our paper as #1. LIF vs. CE-LIF (R1, R3) Due to severe noise, capillaries are hardly visible (Fig.2a, box 2). Although small in radius, these typically cross multiple en-face slices. Fusing these consecutive en-face slices (LIF) exploits volumetric data to reduce the speckle and enhance small features (Fig.2b). LIF improves vessel connectivity, but the background intensity is also increased (Sec.2.1). To emphasize the contrast of vessels and background, we introduce CE-LIF. How Dn-Net is trained (R2) The main purpose of the Dn-Net is to fill in ‘noise holes’ within vessels. Without Dn-Net, the latent image extracted from the original OCT-A is too grainy. LIF has very smooth vessels, so it can be used to train Dn-Net. However, this is not equivalent to generating LIF from the noisy input. LIF is fused by 2R+1 en-face slices {Xi−R, …, Xi+R} and contains more information (including phantom vessels) than a single slice input Xi. We train Dn-Net for only 15 epochs to avoid overfitting. Hence, the Dn-Net is just used to smooth the vessels without introducing phantom vessels. Role of CE-LIF (R3) The main idea of LIFE (Sec.2.2) is to extract the common structure of two different modalities of the same object. The output of the network is the latent image, not the synthetic image. Although CE-LIF may contain phantom vessels, the Dn-Net image doesn’t share these, so they will not exist in the latent image. We regard the CE-LIF as a pseudo-modality, hence the term ’cross-modality’. Loss functions (R2) Eq. 2 is the loss function for both Dn-Net and LIFE, however the hyperparameters a and b are tuned differently. Our model doesn’t need a sophisticated loss function to constrain the synthetic output, because our target is the latent image. Baseline Methods (R2, R3) To our best knowledge, this is the first unsupervised learning attempt for 3D OCT-A segmentation. Existing supervised OCT-A methods use 2D en-face projection images. All available annotated data, including ROSE and OCTA-500 mentioned by R2, are 2D projections as well. These are not useful for our 3D model. A possible solution is to train a 2D model on a public dataset and test it on our data slice by slice. However, given the inevitable differences between datasets, this would require a domain adaptation step, rendering it well beyond the scope of this study. Hessian-based algorithms like the Frangi filter are sensitive to shape. Vessels in OCT-A are not cylindrical: due to the tailing effect caused by OCT-A, they appear flat along the depth direction. This effect is especially severe for larger vessels, causing excessive false positives for the Frangi filter. We thank R3 for the suggestion to use LIF and CE-LIF as additional baselines. The corresponding Dice scores on the human data are 0.5293 and 0.4892, well below LIFE (0.7736). We will include this in the camera-ready paper. Experiments/Dataset (R2, R3) We use the fovea as it contains a mix of large and small vessels. We also use zebrafish eyes, which are similar to the human optic nerve head (ONH) and contain only large vessels. Our model performs well on both datasets. Implementation details (R2, R3) LIFE is free of registration between 3D volumes from different machines, unlike Liu et al. 2020. Greedy is used to register adjacent 2D Bscans of a single volume, to create the LIF image. This is a faster and much easier task than inter-volume registration. The weight map is determined by the local similarity between atlases and reference image (Sec.2.1). This is detailed in the cited JLF paper (Wang et al. 2012); we omit the derivation for brevity. The artifact removal is described in Sec.2.3: we simply match the histogram of ill-registered B-scan (Fig.3b) to its neighbor without artifact (Fig.3a).

Minor comments Thanks, we will address these in the camera-ready version.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This paper proposes a novel self-supervised method for OCTA vessel segmentation. While the method could be interesting, the experiments are very limited with only four human OCTA volumes (three for training and one for testing). This greatly dampens the confidence on the performance of the proposed method.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Reject
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

11

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This work proposes a method for 3D segmentation of fovea vessels and capillaries from OCT-A volumes that requires neither manual annotation nor multiple image acquisitions to train. Overall, this work is interesting, and the task is relatively unexplored. I recommend for accepting this paper so as to bring more insightful research to this community. However, this paper also suffers from several issues, such as Fig.1 is hard to follow, and the methodology part needs to be improved. In addition, although the following work may be published after the submission deadline of MICCAI, it should be cited and discussed in your final version as it is very relevant.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

5

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The idea behind this unsupervised approach for vessel segmentation in these large and complicated image modality is novel and of interest. I don’t find the fact that only area around the fovea was evaluated as a problem as this is where most of the tiny, difficult to segment vessels reside. My main concern was the lack of ablation study but this has been covered by the rebuttal and it seems feasible to be included in the final version.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

4

back to top

LIFE: A Generalizable Autodidactic Pipeline for 3D OCT-A Vessel Segmentation