Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Dmitrii Lachinov, Philipp Seeböck, Julia Mai, Felix Goldbach, Ursula Schmidt-Erfurth, Hrvoje Bogunovic

Abstract

In medical imaging there are clinically relevant segmentation tasks where the output mask is a projection to a subset of input image dimensions. In this work, we propose a novel convolutional neural network architecture that can effectively learn to produce a lower-dimensional segmentation mask than the input image. The network restores encoded representation only in a subset of input spatial dimensions and keeps the representation unchanged in the others. The newly proposed projective skip-connections allow to link the encoder and decoder in a UNet-like structure. We evaluated the proposed method on two clinically relevant tasks in retinal Optical Coherence Tomography (OCT): geographic atrophy and retinal blood vessel segmentation. The proposed method outperformed the current state-of-the-art approaches on all the OCT datasets used, consisting of 3D volumes and corresponding 2D en-face masks. The proposed architecture fills the methodological gap between image classification and ND image segmentation.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87193-2_41

SharedIt: https://rdcu.be/cyhMj

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

This paper presents projective skip-connection for segmentation problems where the input is a high-dimensional data while the groud truth mask is a lower one. The results show some improvement and the paper is well organized and eay to follow.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The problem is interesting for MICCAI audience. The authors have compared their methods with other algorithms and the results show some improvement.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The authors did not show any samle results for visuliaztion.

The introduction part is not well written. The clinical background seems not quite relevant to the motivation of the proposed method. Since the method is to do M to N segmentation, it shall be a general one that fit for retinal and also other modalities.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The work is reproducible.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

I would suggest the authors to show some samples for visuliaztion. Moreover, the introduction part can be improved to reflect the motivation of the work.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Overall, the ideal seems interesting and shows some improvement.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

In some problems of medical image segmentation it is common that the input image data has more dimensions than the desired output segmentation map. In those cases, the deep network needs to combine the information along the reduced dimensions to produce the final predictions. This paper proposes a number of modifications to the U-Net architecture to make the network work with an arbitrary number of reducible dimensions. In particular, the feature maps are not upsampled over the reducible dimensions in the decoder part of the U-Net, and the skip connections from the encoder to the decoder perform average pooling with a proper stride so that the dimensions of the concatenated feature maps match. The paper shows experimentally how these two simple modifications outperform other baselines for projected segmentations.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

S1. The paper reads well, is concise and clear.

S2. The proposed method is sensible and very easy to implement, which greatly increases its potential usefulness in the field of medical segmentation.

S3. The experimental section is exhaustive, with plenty of details about the datasets, the training, and the evaluation procedure.

S4. Despite its simplicity, the experimental section shows a modest but consistent and statistical significant improvement over all baselines in all three datasets and under the two metrics considered.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

W1. The method is extremely simple. The proposed modifications to the U-Net are one of the most straightforward solutions for projected segmentation.

W2. I find some minor unclear details in the description of the baselines. How is the UNet 3D adapted to produce 2D results? A possibility could be running the UNet 3D unmodified and then applying GAP over the reducible dimensions. Is this what this baseline is doing? Also, what does “operating on OCT volume projection” means for baselines UNet 2D and UNet++?
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

I found no issues regarding the reproducibility.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

I suggest authors to clarify the unclear points I listed in weakness W2. Apart from that, I have no further complaints about the paper.
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

I am on the edge with this paper. The extreme simplicity of the method makes me doubt about its suitability for the conference. On the other hand, I found the experimental section quite interesting and convincing. I think that the paper could have some impact in the community given the ease of implementation and the improved performance compared to the baselines. That is why I am very slightly inclined to accept it.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

4
Reviewer confidence

Confident but not absolutely certain

Review #3

Please describe the contribution of the paper

This paper proposes a so-called projective skip-connection module to solve the problem of unmatching input output channels in U-Net shape model.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The details in experiments are shown explicitly. Besides the writing is clear and easy to understand.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The main contribution in this work is the projective skip-connection, which implemented by performing Global Average Pooling in the compressed reducible dimensions ((N-M) D in a NDMD segmentation) to reduce the feature tensor size to 1. The idea is not novelty enough.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The experiments shown in this paper is in great details thus it can be recreated probably though its dataset is private and the date is not open-source.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

The proposed module is too simple and the idea of projective skip-connections is more like an engineering setting in the segmentation.
Please state your overall opinion of the paper

reject (3)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This work is not novelty enough and is more like an engineering choice for specific application.
What is the ranking of this paper in your review stack?

5
Number of papers in your stack

6
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

The reviewers are consistent in the fact that the paper shows interesting results that are convincing. At the same time, the method is easy to implement and rather straight forward.

We ask the reviewers to comment on why their method is interesting to the broader MICCAI community and to clarify some of the smaller aspects raised.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

4

Author Feedback

Dear Area Chairs, Reviewers and Program Chairs,

We thank everyone involved for their thoughtful assessment of the paper. We strongly believe that the proposed method will be of interest to a broad audience of MICCAI because the proposed approach is suited to segmentation and regression of any lower-dimensional manifold of interest. Retina (OCT, OCT angiography) is ideally suited for this method due to its locally flat and layered structure. Similarly to retina, analysis of flat skin (OCT, Ultrasound) structures [1, 2] can benefit from the proposed method. Other potential application scenarios of the projective segmentation technique are segmenting a lumen wall from intravascular ultrasound (IVUS) pullback images (1D lines in 2D images) [3], finding a diaphragm in CT (2D surface in 3D volume) [4] or online tumor tracking during radiotherapy [5]. In fact, the method is suited to many problems that were traditionally addressed with a graph-based framework, where a boundary or a set of surfaces defining 2D or a 3D object of interest is segmented [6]. We will highlight this in the revised version of the Introduction. From a methodological perspective, the projective-skip connections may look simple, yet, they are not trivial. Addressing R5 concerns, we would like to emphasize that the projective-skip connections do not perform global average pooling, completely reducing the target dimension to 1 (used in the ablation experiment called 3D2D in the paper). In contrast, they reduce the encoder feature maps to the size of the bottleneck in “target” dimensions (dimensions that should be reduced). Fig. 2 provides an intuitive explanation. The GAP, however, is only performed at the very end of the network to produce a segmentation mask. We will revise the last paragraph of Section “2 Method” to clarify the principle of projective-skip connections and their role in producing compressed representations. At the same time, we show that our proposed approach provides consistently better results than the more sophisticated state-of-the-art. We are convinced that these results, paired with the possibility for a straightforward implementation, provides a large value for researchers and practitioners working on projective segmentation tasks. Addressing comment R2-W2, UNet 3D was adapted exactly as assumed by the reviewer. For creating the OCT projection, we used the algorithm introduced by Chen et al. [7]. We will clarify this fact and add the corresponding citation in the revised version of our manuscript. Addressing comment R1-W1, we decided to focus on presenting the quantitative performance evaluation due to the page limit. Nevertheless we provided qualitative results in Figure 1 and in the supplementary material (Figure S3). In conclusion, we want to emphasise that the topic of MD->ND segmentation is repeatedly gaining scientific attention, where two closest related works were selected as an oral presentation at MIDL 2019 [8], and for publication in IEEE TMI journal [9]. We build upon these methods, using them as baselines, showing a consistent improvement of our method with respect to them. Additionally, we provide methodological insights from the point of the receptive field and input image size in the supplementary material (Figures S1 and S2).

[1] Miosek et al., JU 2013, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4613587/ [2] Strivastava et al., BMOE 2018, https://dx.doi.org/10.1364%2FBOE.9.003590 [3] Sun et al TMI 2013, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3883441/ [4] Yalamanchili et al, IEEE ISBI 2010, https://doi.org/10.1109/ISBI.2010.5490132 [5] Murphy, SiRO 2004, https://doi.org/10.1053/j.semradonc.2003.10.005 [6] Li et al TPAMI 2006, https://ieeexplore.ieee.org/document/1542036 [7] Chen et al, TVST 2015, https://doi.org/10.1167/tvst.4.5.2 [8] Liefers et al, MIDL 2019, http://proceedings.mlr.press/v102/liefers19a.html [9] Li et al, IEEE TMI 2020, https://doi.org/10.1109/TMI.2020.2992244

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

After carefully reading the paper, the reviewer’s comments and the author’s rebuttal, the paper clearly has good performances in experiments that are conving. On the other hand the methodology novelty in this case is very limited. Indeed, had experiments considered more than a single application domain as the author’s highlight in the rebuttal, it would have much better impact. I would recommend showcasing this at a MICCAI satelite event instead.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Reject
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

8

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This paper introduces an interesting problem of M-dimensional space to N-dimensional space segmentation. The experimental design is thorough and the results are sound. The proposed method seems to introduce limited technical novelties. However, the proposed simple method introduces impressive improvement to several complicated approaches. All the reviewers’ concerns have been satisfactorily addressed. The discussion of design motivation of the projective-skip connection is also convincing. Overall, I believe this newly identified topic may attract new scientific attentions and good clinical translations.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

12

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

I read the paper and feel this paper is interesting, and the clinical impact is high, despite the technical novelty is limited. More illustrative results may be added in the final or its extended version.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

11

back to top

Projective Skip-Connections for Segmentation Along a Subset of Dimensions in Retinal OCT