Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Yi Zhou, Jiang Li, Meng Wang, Weifang Zhu, Yuanyuan Peng, Zhongyue Chen, Lianyu Wang, Tingting Wang, Chenpu Yao, Ting Wang, Xinjian Chen

Abstract

Raw optical coherence tomography (OCT) images typically are of low quality because speckle noise blurs retinal structures, severely compromising visual quality and degrading performances of subsequent image analysis tasks. In this paper, we propose a novel end-to-end cross-domain denoising framework for speckle noise suppression. We utilize high quality ground truth datasets produced by several commercial OCT scanners for training, and apply the trained model to datasets collected by our in-house OCT scanner for denoising. Our model uses the high-resolution network (HRNet) as backbone, which maintains high-resolution representations during the entire learning process to restore high fidelity images. In addition, we develop a hierarchical adversarial learning strategy for domain adaption to align distribution shift among datasets collected by different scanners. Experimental results show that the proposed model outperformed all the competing state-of-the-art methods. As compared to the best of our previous method, the proposed model improved the signal to noise ratio (SNR) metric by a huge margin of 18.13dB and only required 25ms for denoising one image in testing phase, achieving the real-time processing capability for the in-house OCT scanner.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87231-1_36

SharedIt: https://rdcu.be/cyhVj

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

A novel end-to-end learning framework for optical coherence tomography (OCT) images speckle noise reduction has been proposed.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The proposed framework got significantly better results than state of the art OCT images speckle noise suppression methods and offer real-time processing capabilities. The proposed model can be trained and tested with OCT images collected by different scanners.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

I don’t see any weaknesses of this paper.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

I think that reproducibility checklist for this paper has been properly fulfilled.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

This is a well written paper. The proposed hierarchical adversarial learning strategy for domain adaption is very interesting.

A good ablation study has been performed. The shown qualitative and quantitative results obtained using the proposed method outperform state of the art requiring very short processing time.
Please state your overall opinion of the paper

strong accept (9)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Major factors that led me to propose this paper to be accepted has been its novelty and its good experimental section.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

4
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

The authors propose a novel end-to-end cross domain denoising framework for speckle noise suppression in OCT images. The backbone architecture is HRNet that preserves high resolution information across different layers to preserve information. They also add adversarial learning strategy for domain adaption to align distribution shift among datasets collected by different scanners.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

A very well written paper with adequate literature review, standard evaluation metrics and enough experimentation to validate the model. My only concern is the dataset used which isn’t publicly available and therefore not reproducible.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. Dataset does not seem to available to public for reproduction.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Might be difficult due to unavailability of dataset.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Please see weakness and strength.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

I think the architecture is quite novel. My only concern is dataset availability for reproducing the result.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

4
Reviewer confidence

Very confident

Review #3

Please describe the contribution of the paper

This paper proposes an end-to-end deep domain adaptation framework for speckle noise reduction from OCT images. The proposed method used “high- resolution network (HRNet)” during the denoising process and it also used a hierarchical adversarial learning to tackle the domain shift issue of data taken from different scanner.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The paper addresses a very clinically relevant problem which is domain shift. Often end-to-end models tend to be brittle. They fail to perform good enough with data taken from different camera or scanner settings. The paper is clearly written The paper clearly mentions several proposed novelties. The result section shows excellent improvement in performance.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The authors do not mention why they used a particular method (citation 16 in the paper), can you please mention why this particular method you found can perform well? Have you tried any other method? Regarding the dataset section, the paper does not mention the original dimension of the images. The images from 4 scanner- are they similar in dimension ? How many images from each scanner you used? In the ablation study section the table caption mentions : “on a disease OCT image”. Does that mean the result of the table is from only one image? If so, where do the mean and SD come from? Would you please elaborate? For the result section, are these the result only from the in house scanner? Would you please mention that in the table caption to help the reader’s to understand. Also “We also set aside three disease raw images and six normal raw images from the in-house scanner for testing.” , isn’t (3+6) = 9 is a very small number to conclude something? Is there any reason you did not consider to have more images? Was there any specific reason why disease images were introduced?
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Upon release of the code, the reproducibility can be verified
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

The reviewer believes that the problem addressed by the paper is scientifically very relevant. Domain shift is a common problem which hinders the clinical practice of many AI based algorithms and the papers aims to address that problem.

Are the authors considering making both the datasets public? It would be a very good resource for the research community as currently we do not have many publicly available noisy-denoisy paired OCT images. As the authors claim to use clinically generated datasets, it will be a great resource for wider research community. Please note that this does not have any impact in the review decision. Other comments mentioned clearly in the strength and the weakness section of the papers.
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The relevant weakness and comments are mentioned before but those are mostly minor and can be addressed quickly. The reviewer believes that the problems addressed in the paper is very clinically relevant and important. The paper can be accepted borderline given that the comments given in the weakness are addressed properly.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

4
Reviewer confidence

Confident but not absolutely certain

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

This is a strong paper that proposes a novel solution for OCT denoising and achieved superior performance. There is only minor concern about whether the dataset will be publicly shared.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

1

Author Feedback

Meta-Reviews: Response: Thank you for the encouraging comments. We will revise the manuscript accordingly.

Reviewer #1 Response: Thank you for the comments.

Reviewer #2 Response: Thanks for your review. For your concern regarding the publicity of our datasets, unfortunately, we won’t release them at this moment due to certain commercial reasons, but it may change in the future.

Reviewer #3 Response: Thank you for the constructive suggestions. Q1: The authors do not mention why they used a particular method (citation 16 in the paper), … Have you tried any other method? A1: This is a very good comment. The overall objective of our work is to leverage existing datasets with ground truth for OCT speckle noise reduction in datasets where ground truth data are difficult or expensive to collect. Different datasets were collected by different OCT scanners where data distribution shifts are very common, and a domain adaptation process is needed. In our previous work (Alg2, accepted for publication), we utilized the CycleGAN model to perform domain adaptation at image level and achieved satisfactory results. Method in [16] is a very good domain adaptation algorithm for image segmentation built upon the U-net model, where each layer was treated differently because different layers have different resolutions, thus the algorithm was named “Hierarchical”. Inspired by the method in [16], we redesigned the algorithm to fit to the high-resolution net (HRNet) for OCT speckle noise reduction. The new model presented in this paper obtained much better results than Alg2 and outperformed those competing state of the arts. Our results showed that layer by layer domain adaptation is superior to image level adaptation alone in deep models for OCT speckle noise reduction. In the camera-ready paper, we will reveal Alg2 and add this information.

Q2: Regarding the dataset section, … How many images from each scanner you used? A2: Due to space limitation, we did not present specifications of the datasets. The image sizes of Topcon DRI-1 Atlantis, Topcon 3D OCT 2000, Topcon 3D OCT 1000 and Zeiss Cirrus 4000 are 512×992, 512×885, 512×480 and 512×1024, respectively. The training set is composed of 256 images from Topcon DRI-1 Atlantis and 256 images from Topcon 3D OCT 2000. We will refer readers to Alg2 for details of the datasets in the camera-ready paper.

Q3: In the ablation study section the table caption mentions: … Would you please elaborate? A3: Thank you for pointing out this typo. The caption of Table 1 should be “Table 1. Ablation study performances on 9 images from in-house OCT scanner (mean ± standard).” Will be corrected.

Q4: For the result section, … Was there any specific reason why disease images were introduced? A4: Validation results presented in this paper were on images collected by the in-house scanner only. We will change the Table caption to accurately reflect this information. The chosen performance metrics require manual selection of regions of interest and labeling retinal layers on OCT images, as shown in Fig. 2(a) and Fig. 3(a). These manual operations are complex and labor intensive, we currently just completed 9 images from the in-house OCT scanner for validation. The number of ‘9’ is relatively small but the validation was based on regions and all algorithms were evaluated on the same regions. We believe the evaluation process is relatively fair. As part of our future work, more images will be labeled, and the proposed method will be applied to other publicly available OCT datasets for denoising. In addition, OCT scanner is purposed for disease diagnosis so evaluation on disease images is clinically more relevant.

Q5: Are the authors considering making both the datasets public? … Please note that this does not have any impact in the review decision. A5: Unfortunately, we won’t release them at this moment due to certain commercial reasons, but it may change in the future.

back to top

High-Resolution Hierarchical Adversarial Learning for OCT Speckle Noise Reduction