Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Yanglan Ou, Ye Yuan, Xiaolei Huang, Kelvin Wong, John Volpi, James Z. Wang, Stephen T. C. Wong

Abstract

Diffusion-weighted (DW) magnetic resonance imaging is essential for the diagnosis and treatment of ischemic stroke. DW images (DWIs) are usually acquired in multi-slice settings where lesion areas in two consecutive 2D slices are highly discontinuous due to large slice thickness and sometimes even slice gaps. Therefore, although DWIs contain rich 3D information, they cannot be treated as regular 3D or 2D images. Instead, DWIs are somewhere in-between (or 2.5D) due to the volumetric nature but inter-slice discontinuities. Thus, it is not ideal to apply most existing segmentation methods as they are designed for either 2D or 3D images. To tackle this problem, we propose a new neural network architecture tailored for segmenting highly-discontinuous 2.5D data such as DWIs. Our network, termed LambdaUNet, extends UNet by replacing convolutional layers with our proposed Lambda+ layers. In particular, Lambda+ layers transform both intra-slice and inter-slice context around a pixel into linear functions, called lambdas, which are then applied to the pixel to produce informative 2.5D features. LambdaUNet is simple yet effective in combining sparse inter-slice information from adjacent slices while also capturing dense contextual features within a single slice. Experiments on a unique clinical dataset demonstrate that LambdaUNet outperforms existing 3D/2D image segmentation methods including recent variants of UNet. Code for LambdaUNet is available at https://github.com/YanglanOu/LambdaUNet.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87193-2_69

SharedIt: https://rdcu.be/cyhMN

Link to the code repository

https://github.com/YanglanOu/LambdaUNet

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

This work extend the recently proposed 2D Lambda layer to a 2.5D scenario, specifically tailored for 3D MR images with large intra slice spacing, e.g. diffusion-weighted (DW) MR images. The proposed Lambda+ layer is used in a UNet structured segmentation network, LambdaUNet. This network is used for stroke lesion segmentation in DW images.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The work proposes a simple, yet interesting extension of Lambda layers. The introduction of global and inter-slice Lambdas, in addition to local as in the original work, seems well motivated. Results on the tested dataset compared favorably to the chosen baselines. The manuscript is well written and organized.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Novelty is somewhat limited as this seems to mainly consist in the introduction of local and inter-slice Lambdas. Data used in the study is not public hence it is somewhat hard to gauge general performance. This reviewer appreciates the fact that this often the case with medical imaging datasets, nonetheless, there are some public datasets that would fulfill the main motivating characteristics of the DW images used here (large inter-slice distance). The method would have benefited from evaluation on some of these datasets, e.g. the ISLES or WMH Segmentation challenges. The choice of comparison methods fit well with the proposed method as they are all UNet-structured, however, there are other methods that have been shown to work well on lesion segementation in MR images with large inter-slice spacing [1,2], with [1] having a publicly available tool (pytorch) and [2] even being a UNet structured method (although, no code is publicly available).

[1] Konstantinos Kamnitsas, Christian Ledig, Virginia F.J. Newcombe, Joanna P. Simpson, Andrew D. Kane, David K. Menon, Daniel Rueckert, and Ben Glocker, “Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation”, Medical Image Analysis, 2016.

[2 ]White matter hyperintensity and stroke lesion segmentation and differentiation using convolutional neural networks. R Guerrero, C Qin, O Oktay, C Bowles, L Chen, R Joules, R Wolz, NeuroImage: Clinical. 2018
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Work is clearly described, with pseudo-code provided in the supplementary material. This would make the technical aspects fairly simple to reproduce, however, as the evaluations are done on a non-public dataset, experiments are not reproducible.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

See above comments.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper is well written, clear and results compare favorably to the chosen baseline methods. However, this reviewer believes that novel contributions are somewhat limited.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

2
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

The authors propose an architecture called LambdaUNet to solve the stroke lesion segmentation in DWI. They introduce a Lambda+ layer that (1) leverages long-range context for each pixel, and (2) extends the prior art into 2.5D for 3D volumes with large inter-slice discontinuities
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- reasonable design to use Lambda layers that provides long-range contexts in place of conventional convolution layers for tough problems like lesion segmentation
- comprehensive experiments and comparison with baseline approaches
- good overall performance achieved for stroke lesion segmentation
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- lack of clarity on explaining how the lambdas work in terms capturing long-range contexts
- the rationale for superiority of 2.5D design is not very convincing, there are indeed discontinuities across slices, but isn’t it more reliable to capture inter-slice relationship with a 3x3x3 kernel than only looking at one pixel on the two neighboring slices?
- lack of comparison with other 2.5D methods in medical imaging, some consider 2.5D as using multiple consecutive slices, some consider 2.5D as the axial, coronal, and sagittal planes crossing the voxel.
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors claim to release the code later. They also provide the pseudo code in the supplementary material.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
The authors highlight two points of this paper (1) using the lambdas for long range context, (2) using 2.5D for 3D volumes with large slice distance.

It feels like the authors would like to boost the overall novelty of this paper, they try very hard to validate (2), as they may consider it not enough to have (1) only, as it is adopted from another recent paper. But from my perspective, it would be already quite strong if they made it very clear how and why the lambdas capture the long-range context in this specific lesion segmentation problem, especially how do Q (query), K (key), and V (value) mean in the lamdba layer design, and maybe also add some discussion about the difference between the lambda layers and the transformers as both are known for getting long-range contexts since they mention TransUNet. On the other hand, (2) the 2.5D design is less convincing to me, when dealing with 3D volumes with large slice distances, the problem is often not about the discontinuity, but about the small number of axial slices, i.e., usually much smaller than the receptive field. It will be nice to see the statistical significance for the performance described in Table 1. It is also weird that the 3D UNet has high recall and low precision, but the LambdaUNet-3D has low recall and high precision. If they would really like to emphasize the 2.5D, they should also discuss and compare with other 2.5D designs, like using multiple consecutive axial slices, like using the axial, coronal, and sagittal planes crossing the voxel.

minor issue
- it is not clear whether the recall/precision/F1 score is based on pixels or lesion instances
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

reasonable design of using the lambda layers and good performance achieved for stroke lesion segmentation
What is the ranking of this paper in your review stack?

3
Number of papers in your stack

5
Reviewer confidence

Very confident

Review #3

Please describe the contribution of the paper

The author has proposed a segmentation network (LambdaUNet) for ischemic stroke lesion segmentation for 2.5D DWI images. The traditional UNets convolutional layers are replaced with Lambda+ layers which can capture both dense intra-slice and sparse inter-slice features.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The method is novel and technically strong. The proposed approach modifies the pre-existing Lambda layers to Lambda+ layers so that overfitting (3D) or loss of some information can be reduced (2D). The method computes the global, local and inter slice contextual information which is very important when dealing with 2.5D DWI scans.

Data split in case of training consists of equal class samples, hence no data imbalance issue is there. Validation & test samples are split using three-fold cross validation, hence there will be no bias induced during testing phase.

Qualitative results show that the segmentation mask from the proposed approach is approximately 86% (DSC) similar to the ground truth segmentation mask, which beats the state-of-art techniques.

The qualitative results show that performance gains range from 3.06% to 8.31% for average DSC. although there is trade-off between different statistical performance in table 1. Overall segmentation score (DSC) and F1 score using the proposed approach beats the state of the art techniques.

The paper is well written and easy to follow.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

There are no main weakness in the paper.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The author has provided supplementary section describing about implementation of lambda+ layers and ablation studies about effect on kernel sizes on the performance of the system.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Was there any pre-processing stages required during the implementation of the proposed approach?

Description about how data was evaluated by different observers can be very useful to understand inter or intra observer variabilities.

Using kernel size >7 can it have impact on results?
Please state your overall opinion of the paper

strong accept (9)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The proposed approach is clinically significant in terms of segmenting Ischemic stroke lesions in DWI images. The paper is technically sound, the author has implemented a novel idea of replacing the traditional UNets convolutional layers with Lamda+ layers which captures both dense intra-slice and sparse inter slice features.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
All reviewers agree that this work adapts the recently proposed idea of lambda layers in a convincing way to a relevant medical image analysis task. There were no major concerns about correctness or reproducibility, and the paper demonstrates a clear benefit in direct comparison to several competing methods. Despite this, there are a few comments in the reviews that should be addressed in the camera-ready version, and I have a few additional ones myself:
- I believe that the claim of being “the first 2.5D segmentation model designed specifically for 2.5D data” is untenable. For example, the approach in https://arxiv.org/pdf/1806.07146.pdf mixes 2D and 3D convolutions to account for the higher in-plane resolution.
- I would like to see more details of how the scores reported in Table 1 have been defined / computed. In particular, the reported F1 scores are not the harmonic mean of the reported precision and recall values. Moreover, if segmentations are given as binary masks, F1 and dice similarity coefficient should yield identical values.
- It did not become entirely clear to me whether the discussed “discontinuities” between slices arise from highly anisotropic voxels, gaps between slices, or both, and whether the authors consider their method to be most useful for one of these cases, or equally applicable to both. What were the slice thickness and gaps, respectively, in the data that was used for the reported experiments?
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

1

Author Feedback

N/A

back to top

LambdaUNet: 2.5D Stroke Lesion Segmentation of Diffusion-weighted MR Images