Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Hong Liu, Dong Wei, Donghuan Lu, Yuexiang Li, Kai Ma, Liansheng Wang, Yefeng Zheng

Abstract

Automated surface segmentation of retinal layer is important and challenging in analyzing optical coherence tomography (OCT). Recently, many deep learning based methods have been developed for this task and yield remarkable performance. However, due to large spatial gap and potential mismatch between the B-scans of OCT data, all of them are based on 2D segmentation of individual B-scans, which may loss the continuity information across the B-scans. In addition, 3D surface of the retina layers can provide more diagnostic information, which is crucial in quantitative image analysis. In this study, a novel framework based on hybrid 2D-3D convolutional neural networks (CNNs) is proposed to obtain continuous 3D retinal layer surfaces from OCT. The 2D features of individual B-scans are extracted by an encoder consisting of 2D convolutions. These 2D features are then used to produce the alignment displacement field and layer segmentation by two 3D decoders, which are coupled via a spatial transformer module. The entire framework is trained end-to-end. To the best of our knowledge, this is the first study that attempts 3D retinal layer segmentation in volumetric OCT images based on CNNs. Experiments on a publicly available dataset show that our framework achieves superior results to state-of-the-art 2D methods in terms of both layer segmentation accuracy and cross-B-scan 3D continuity, thus offering more clinical values than previous works.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87237-3_11

SharedIt: https://rdcu.be/cyl9Q

Link to the code repository

https://github.com/ccarliu/Retinal-OCT-LayerSeg.git

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The authors use 2D encoders to extract features from OCT B-scans. The 2D features are feed through 3D decoders which align the 2D features and generate the layer segmentation for the OCT data.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

It is interesting work. I have lots of clarifying questions that I go through below.

I think the idea of the quasi 3D framework is a nice addition to the field and has considerable benefits for OCT image processing.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

“Earlier explorations … [17, 2] methods.” There are levelset based methods [Carass2014, Liu2019, Novosel2015, Novosel2017] that I am not sure neatly fit into any of these three categories. Maybe, I draw an unnecessary distinction between “contour modeling” and “levelsets”. However, there is clearly a fourth category as both [2, 17] are graph based methods that use machine learning based features. Whereas [9] is a pure graph based method. Maybe this fourth category is “Hybrid Methods”.

“independent 2D images, despite … area of the eye”

This is all completely factual. However, what is being ignored is the fact that the inter B-scan distance is orders of magnitude bigger than the intra B-scan distance. That is the distance between B-scans is much larger than the distance between A-scans within any single B-scan. In such a scenario, it is not unreasonable (maybe even desirable) to regard B-scans as independent.

“[5] and [9] were the minority that attempted 3D OCT segmentation.” This is simply not true. [Carass2014, Novosel2015] are examples of 3D based methods that have been around for some time and have derivative works that build upon them [Novosel2017]. Unless I am mistaken [2, 17] are also 3D methods as they have graph connections between B-scans ensuring 3D smoothness.

“the surface intersects with each A-scan exactly once.” This is a common assumption in OCT work. However, it is only valid in macula imaging as in optic nerve head imaging the surface can bend back on itself around the nerve head. The authors should note this.

“Although it is feasible to add an alignment step while preprocessing, we believe that a comprehensive framework that couples the B-scan alignment and layer segmentation would mutually benefit each other” Maybe I am in the minority in thinking this about MICCAI, but it is a Scientific Conference, not a Religious One. So “believe” is not really important. Cold hard scientific facts are. You should demonstrate your belief with some experiments.

Equation 1, says “(b_i, b_j) \in \mathcal{N}{B}” which would imply any two B-scans of the image. But the text says “(b_i , b_j) denotes two adjacent B-Scans”. Would it not make more sense to just write “(b_i , b{i + 1})”. Or better yet B-scan “b” and its neighbor “b \pm 1”, why even introduce i and j?

Particularly, given that “b \in [1, N_B]” s introduced before i and j. Why the notation?

B-scan alignment is useful, but the reality is that A-scans are misaligned as well, due to eye (and patient) motion, tracking failure, “floaters” in the Aqueous humor, etc. Why not deal with that as well?

There is a disconnect in going from G_f to G_a. Do the authors run an entire volume through G_f and then do pairwise B-scans through G_a?

I have to assume that is the case. The lack of clarity here makes this confusing and the wasted space on extra (superfluous) notation could have been put to better use providing some clarity.

CONTINUED IN QUESTION 7.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

No comment.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Figure 2, the right-most image is described in the caption as “ours”. It is not clear to me if this is after the flattening to the Bruch’s membrane or after G_a? In either case there is an image missing, either the result after flattening or after G_a.

Figure 2 features three colours (yellow, green, and blue), we are not told what these refer to.

Table 1 lacks units, standard deviations, and any statistical analysis.

Figure 3 is confusing. There is a ground-truth (light blue) and then four other bars. The FCBR method (yellow) appears to be closer to the ground-truth than the proposed method (dark blue).

Yet the authors say “As shown in Fig. 3, surfaces segmented by our method has better cross-B-scan connectivity than those by FCBR [10] even with pre-alignment, as indicated by the more conspicuous spikes clustered around 0.” The spike may well be “conspicuous” but surely you want the results to match the “ground-truth”. Is “ground-truth” in Fig. 3, not really ground truth in the traditional (human) sense of the phrase?

The authors seem to allude to this with “human annotators work with one B-scan at a time”, in which case maybe it should not be called “ground-truth”?

References

Numbered references are from the authors paper.

[Carass2014] Carass et al., “Boundary classification driven multiple object deformable model segmentation of macular OCT”, Biomedical Optics Express, 5(4):1062–1074, 2014.

[Liu2019] Liu et al., “A layer boundary evolution method for macular OCT layer segmentation”, Biomedical Optics Express, 10(3):1064-1080, 2019.

[Novosel2015] Novosel et al., “Loosely coupled level sets for simultaneous 3D retinal layer segmentation in optical coherence tomography”, Medical Image Analysis, 26(1):146-158, 2015.

[Novosel2017] Novosel et al., “Joint segmentation of retinal layers and focal lesions in 3-D OCT data of topologically disrupted retinas”, IEEE Trans. Med. Imag. 36(6):1276-1286, 2017.

[Roy2017] Roy et al., “ReLayNet: Retinal Layer and Fluid Segmentation of Macular Optical Coherence Tomography using Fully Convolutional Network”, Biomedical Optics Express, 8(8):3627-3642, 2017.

Some typos the authors should correct: “DCNN” -> “CNN”

“superiority toward existing” -> “superiority over existing” OR “superiority with respect to existing”

“consisting 2D CNN” -> “consisting of 2D CNN”

“interested reader to [10]” -> “interested readers to [10]”

“Bruch’s memvbrane” -> “Bruch’s membrane”
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

See responses to questions 3, 4, 5, and 7.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

The authors proposed a new framework for simultaneous B-scan alignment and 3D retinal layer segmentation for OCT image, and two losses.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The authors novely combined the alignment and segmentation together.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. The experiments do not show the segmentation results in vision comparison. The proposed algorithm is only compared with FCBR. There is not ablation study in the part.
2. The flowchart in figure 1 is a little confused, such as the line in the Ga(3D).
3. There are some grammar errors in the paper.
Please rate the clarity and organization of this paper

Poor
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Fair
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
1. Please add some ablation study and comparison experiments.
2. Please add some segmentation images for visual comparison.
3. The flow chart should be revised.
Please state your overall opinion of the paper

probably reject (4)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The experiments and the explanation of the algorithms are not enough.
What is the ranking of this paper in your review stack?

3
Number of papers in your stack

4
Reviewer confidence

Confident but not absolutely certain

Review #3

Please describe the contribution of the paper

The authors propose an Optical Coherence Tomography (OCT) layer segmentation of the retina using a DCNN-based hybrid 2D-3D multi-task network that in addition to segmentation perform an alignment of slices (B-scans) and enforces a smoothness of layers between B-scans.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The multi-task approach of aligning scans and segmentation with smoothness constraints within one single model is interesting.

The authors nicely tackle major challenges of retinal OCT analysis: 1.) the highly anisotropic structure with large between slice distance, by using a 2D encoder to extract features from B-scans and 3D later-on to incorporate the relationship of features between slices.
2.) Motion between B-scans, by aligning b-scans guided by intensity values and smoothness of layers. The alignment is incorporated into the segmentation model by a spatial transformer module (STN) transforming feature maps of all scales.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. Evaluation dataset: The algorithm was evaluated on an intermediate AMD and healthy case dataset, where layers are relatively smooth, and neighboring B-scans are sufficient similar to allow an alignment via NCC. This assumption may not hold on more severe cases, such as late-stage AMD, and/or for a large slice-distance where B-scans are too different to allow a proper alignment.
2. Hyper-parameter tweaking: The smoothness parameter lambda has been tweaked in preliminary experiments. If the experiments were done on the same dataset, the results might be biased.
3. No comparison with other state-of-the-art layer segmentation algorithms.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

If the code is published, as hinted in the paper and stated in the checklist, reproducibility should be high. Model and experiments are well described and evaluated.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
1. It is unclear how the smoothness loss (L_SmoothS) is computed. Is the gradient function considering the 3D surface or 2D surface only. Is it considering the B-scan alignment as well? If not, you may not assume smoothness between B-scans due to motion.
Minor
1. Please provide information from which OCT device the scans are. There are significant differences in image properties and quality between scanner devices.
2. Typo: Page 6 center: Bruch’s memvbrane Suggestions for journal paper: ——
3. It would be interesting to see an evaluation on additional datasets with more severe diseases in the sense of disrupting layer structures, and furthermore on more layers.
4. Compare with other b-scan alignment methods, such as [1]
5. Add comparison with other layer segmentation algorithms.
[1]A. Montuoro et al. „Motion Artefact Correction in Retinal Optical Coherence Tomography Using Local Symmetry“, in MICCAI 2014, Proceedings, Part II, Cham, 2014, S. 130–137, doi: 10.1007/978-3-319-10470-6_17.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The approach of handling the high anisotropy, and the alignment of B-Scans in combination with enforcing smoothness is a nice approach. In particular parts of the developed method may also be applied on different tasks in OCT image analysis to better incorporate 3D information into their model and improve their results. The concept may also be used for other modalities with high anisotropy.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

The reviewers have recognized that the authors proposed a novel and interesting methodology to solve an important problem of simultaneously segmenting retinal layers and aligning the adjacent B-scans. Nevertheless there are a few items raised by the reviewers that could help improve the clarity of the paper.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

3

Author Feedback

N/A

back to top

Simultaneous Alignment and Surface Regression Using Hybrid 2D-3D Networks for 3D Coherent Layer Segmentation of Retina OCT Images