Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Federico Turella, Gustav Bredell, Alexander Okupnik, Sebastiano Caprara, Dimitri Graf, Reto Sutter, Ender Konukoglu

Abstract

Imaging plays a crucial role in treatment planning for lumbar spine related problems. Magnetic Resonance Imaging (MRI) in particular holds great potential for visualizing soft tissue and, to a lesser extent, the bones, thus enabling the construction of detailed patient-specific 3D anatomical models. One challenge in MRI of the lumbar spine is that the images are acquired with thick slices to shorten acquisitions in order to minimize patient discomfort as well as motion artifacts. In this work we investigate whether detailed 3D segmentation of the vertebrae can be obtained from thick-slice acquisitions. To this end, we extend a state-of-the-art segmentation algorithm with a simple segmentation reconstruction network, which aims to recover fine-scale shape details from segmentations obtained from thick-slice images. The overall method is evaluated on a paired dataset of MRI and corresponding Computed Tomography (CT) images for a number of subjects. Fine scale segmentations obtained from CT are compared with those reconstructed from thick-slice MRI. Results demonstrate that detailed 3D segmentations can be recovered to a great extent from thick-slice MRI acquisitions for the vertebral bodies and processes in the lumbar spine.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87193-2_65

SharedIt: https://rdcu.be/cyhMI

Link to the code repository

https://github.com/FedeTure/ReconNet

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The paper proposes a method for the detailed 3D segmentation and reconstruction of the vertebrae from thick MRI slices. The task is challenging because bone structures are missing (even in the ground truth) due to the slice thickness. However, the authors propose a method that can highly improve the vertebrae reconstruction from MRI thick slices, and they compared the results with the segmentation done on CT scans reducing the difference between the two modalities.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The paper is very well written and clear. Excellent the introduction where the authors state clearly the problem they want to solve, and the challenges linked to it. Examples and supplement material are presented to show the importance and the limitation of the vertebrae segmentation in MRI. Undergoing an MRI scan is much safer than CT, also it would be possible to visualise the soft tissues, but this modality has big limitation due to the slice thickness. The authors propose a novel approach, using paired CT images and MRI in order to train a reconstruction network. The method looks simple but it leads to a big improvement reducing the gap between CT and MRI segmentation. Good the evaluation section (both qualitative and quantitative) where the authors compared the results obtained each module developed versus the state-of-the-art work, and also versus CT segmentation.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The methodology is well described but it could be better illustrated. The authors provide a visualisation of the main steps involved in the process without providing details. I believe an illustration (a figure with a visualisation of the main blocks and steps) of the reconstruction network ReconNet would have been beneficial for the overall clarity of the method also to make easier the reproducibility process.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Some of the steps involved in the process are based on well know architectures, which are easy to reproduce. However, an illustration of the ReconNet (the proposed reconstruction network) would have been useful to reproduce the work since there are several variables and parameters involved. since no code have been made available, a pseudo code or a figure that illustrates the method would have improved the reproducibility.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
The paper proposed a novel approach to reconstruct detailed vertebrae from thick MRI slice. The proposed pipeline consists of three main steps: an initial segmentation using deep learning approach in literature, a reconstruction network for a high-quality segmentation and a VAE for the final refinement. The results show a clear improvement of each modules involved in the pipeline. The description in the methodology is clear but I would suggest to provide an illustration or an algorithm for the ReconNet. This would make easier to reproduce the work. There are also few corrections to address:
- There is a table 1 in the paper that has never been referenced or mentioned in the text. Please add the reference and add a brief description of the table in the text.
- Page 2, line 25: there is a repetition of the word “method”.
- When you mention the supplement material please refer to the specific section.
- Page 3 line 3: please check the punctuation, a full stop is missing.
- References index looks wrong: the first reference number is [16].
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper presents a great improvement for the 3D detailed reconstruction of vertebrae from MRI thick slice. The method could have an impact and could also be useful for other segmentation tasks facing similar issue. The results are well presented and are extensive. My only concern is about the methodology, which is well explained in the text but I think an illustration of the proposed ReconNet would have been beneficial for the reproducibility of the work. Also, in case of acceptance the authors need to address the minor corrections (typos and reference issues) that have been detailed above.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

4
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

In this paper, a segmentation reconstruction network named ReconNet was proposed to transform low-quality segmentations extracted from thick slice MRI to high-quality segmentations.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The work in this paper is a novel application, which is of great significance in clinical practice.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The innovation on methodology is insufficient. The proposed method is the combination of existing approches, which is short of innovation on methodology.
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The reproducibility is satisfactory. How the reconstructed vertebrae were registerd to the ground truth is not described in the paper.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
1. How did the author process the overlapped areas when the reconstructions of all vertebrae in the image were mapped into the original image space?
2. The intensities of T1-weighted and T2-weighted images are difference. Training the segmentation model with both T1 and T2 images is not a good solution.
Please state your overall opinion of the paper

reject (3)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The innovation on methodology is insufficient.
What is the ranking of this paper in your review stack?

3
Number of papers in your stack

5
Reviewer confidence

Very confident

Review #3

Please describe the contribution of the paper

The authors propose a method to segment lumbar vertebrae from thick-slice spine MR images (2D IFCNN) and refine the segmentation masks via subsequent super-resolution model (ReconNet) and statistical shape modelling (VAE). To develop and evaluate the method, the authors propose to use the available annotated MRI data , annotated CT data with derived low-res masks, and an annotated paired CT+MRI data. The results show a substantial improvement in accuracy (DSC) of the “reconstructed” masks over the plain segmentation results.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The manuscript presents the considered problem in an excellent way, provides a comprehensive review of the prior art, and justifies each step of the proposed method;
2. The proposed method is designed to make heavy use of the available and commonly acquired clinical MRI and CT data. Validation of MRI-focused methods for in vivo assessment of living tissues, in general, is a long-standing challenge. Here, the authors propose to use high-res CT data as a reference (which is one the best available clinical modalities for bone) in vertebrae segmentation task, assuming that there is no relative deformation of vertebrae between MRI and CT (which is true to a large extent);
3. The authors employ multiple distinct datasets to develop and evaluate the method, which, given the final results, somewhat supports the generalization and reproducibility of the method in the task;
4. The evaluation and analysis is done for each part of the method in a strict hold-out setting;
5. The authors analyze the impact of the proposed method on the potential downstream tasks (FEM-based).
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. Limited sample size and lack of subjects with moderate and severe vertebrae conditions;
2. Limited description of the datasets and the computational models used (i.e. number of layers, basic computational blocks, etc).
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
It is highly recommended that the authors add:
1. Detailed description of the the private dataset: scanner, coil, sequence, TR, TE, etc;
2. Brief descriptions of other used datasets (at least, distributions of age, sex, BMI);
3. Architectural details of the used models (number of layers, filters, etc).
The authors checked the questions related to analysis of statistical significance, yet it is not presented in the text. Even though the improvements are notable and, likely, significant, it is recommended to do statistical comparison (e.g. via Wilcoxon signed-rank test).
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
1. The manuscript would benefit from having a more detailed workflow diagram (e.g. with the datasets and validation schemes included). Regardless, the method is clearly presented in the text;
2. It is not clear how two MRI modalities (T1 and T2) from the private dataset were used in the study. Were the scans concatenated before feeding them into the network? Were independent T1 and T2 models trained and evaluated?
3. It is not clear whether/how the procedure of generating low-quality segmentations from CT data was designed/validated. Could the authors comment on how similar are the simulated low-res segmentations are to the ones obtained directly from MRI? Did the authors investigate whether sparse slice selection from CT volumes is better (i.e. more realistic) than averaging of groups of slices?
4. The authors use a weighting mask W to stress the importance of the near-surface voxels. What was W used in the study? Was it optimized as a hyper-parameter?
5. Given a small sample size of the MRI data used to train a segmentation model, it would have been appropriate to use a k-fold cross-validation setting (i.e. train an ensemble of diverse models);
6. Please, add the information on the distribtuion of subjects in training and testing subsets (age, sex, BMI) for all the considered datasets. Importantly, which subset the non-healthy subjects were assigned to?
7. Since the aim of the method is to obtain as accurate segmentations from MRI as possible, could the authors clarify why the scans were resampled to 1mm spacing (from 0.59mm/0.71mm), thus, losing the information. Even though the same preprocessing is done in some of the prior studies, it seems more reasonable to rescale to a lower (common) spacing (e.g. 0.5mm);
8. Supplemental Figures should be in inverse order according to their appearance in the text. Supplemental Figure 2 should have field-of-view aligned across the sub-plots for better visual correspondence. It is recommended also to highlight the areas of discrepancy referred by the authors.
Minor:
1. The authors somewhat discourage the use for CycleGAN for the considered problem for large ROIs. It is possible that CycleGAN could still find an application when applied on a vertebra-level (small ROI);
2. Term “reconstruction”, from my standpoint, is mainly reserved for obtaining the images from k-space (MRI) / sinogram (CT), etc. Perhaps, “refienment” could be a better wording;
3. “Latest space of 100 distributions”. Perhaps, “dimensionality of the latent space is 100”?
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- Clear introduction to and formulation of the problem. Reasonably wide review of the prior art;
- Extensive use of the modern computational blocks;
- Methodologically accurate development and evaluation of the proposed method;
- Clear clinical focus of the method, clinically-feasible implementation;
- Reasonably strong show-how on the potential value of thick-slice MRI for volumetric vertebrae assessment. I find this important, given the lately increasing interest in spine MRI.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

This work addresses the challenging problem of segmentation from thick slice MRI by incorporating information from paired higher resolution CT data. Reviewers agree that there is merit in studying new solutions for this problem. Some concerns were raised regarding the degree of innovation of the papers methodological contribution and some shortcomings and clarifications needed in the experimental evaluation (e.g. statistical significance of results). Authors are encouraged to identify and address the main issues of the reviews in their rebuttal, and provide a comment on how the final manuscript could incorporate these clarifications.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

4

Author Feedback

We would like to thank all the reviewers for the insightful remarks, we will try to address the main concerns with this rebuttal.

Both R1 and R3 critique the lack of details regarding the model architecture, especially that of ReconNet, which could impair reproducibility. We will make the code publicly available if accepted and augment Figure 1 by showing the computational blocks of the architectures instead of triangles.

The main concern of R2 is an insufficient innovation in methodology. While it is true that the singular tools existed prior to this work, we believe that their combination for the specific use presented, paired with the approach of combining CT and MRI datasets, is novel. In addition, it represents a significant improvement in closing the bone segmentation performance gap between the two modalities, as noted by the other reviewers.

R2 pointed at the problem of overlapping areas when the vertebrae are assembled back together. We observe an average voxel overlap of 0.39±0.45% per patient after vertebrae assembly. This work sees segmentation as an intermediate step towards biomechanical simulations and relies on existing solutions to address the overlap problem in that domain, specifically (DOI:10.3389/fbioe.2021.636953). For finite element models, the triangular faces of the meshes (superior and inferior vertebrae) that are overlapping are identified and the corresponding nodes are moved backward along the normal vector of these faces until the facet joint gap is created. Multi-structure modeling requires further work that we will address using the model proposed here as a building block.

R2 questions our use of both T1w and T2w modalities for training the segmentation network. We did so to help improve generalization motivated by the recent work (DOI: 10.1109/TMI.2020.2973595).

R3 asks to clarify the exact use of T1w and T2w images in the pipeline. Only one input image (either T1w or T2w) was used as the input, making the pipeline compatible with either modality.

We agree with R3’s observation that the test set is small and lacks severe malformations. However, it is a unique resource of paired CT and MRI images of the spine to the best of our knowledge. In this work, we aimed for reliable quantitative evaluation and were constrained to use the only publicly available dataset we could find, in which such cases are underrepresented.

Addressing the other comments of R3:

As suggested a Wilcoxon signed-rank test was done for 2D Iterative FCN vs. VAE and ReconNet Augmented vs. VAE (names corresponding to Table 2) resulting in both p-values < 0.001 and thus being significant.

We did not try other augmentation methods to simulate thick-slice MRI from CT ground truth, which is a direction worth exploring. Under visual inspection, ReconNet is trained with simulations that closely resemble the MRI segmentation, while the low-quality vertebrae fed to ReconNet Augmented appear to be more distorted than the MRI segmentation.

Information regarding weight patch W will be added, thanks for noting our omission.

The training set for the segmentation and reconstruction algorithm had a male/female ratio and mean patient age of 60%,50% and 50±17,55±18 years, respectively. The independent test set had a male/female ratio of 64% and a mean patient age of 57±12 years. No information on the BMI or diagnosis of the patients is available.

We agree that a resampling to 0.5mm would be a good idea. However, since the resolution bottleneck was the slice thickness, we do not expect a significant change in the presented results.

We appreciate the reviewer’s comment encouraging k-fold validation. Indeed this would have been a good strategy considering the small dataset. However, we note that the test set was an independent dataset that was not used during training in any part. We hope this demonstrates generalizability nevertheless.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The main concern of reviewer 2 is in its novelty. R1 and R3 find the proposed method original, and the author rebuttal again clarifies that the contribution lies in the unique combination of CT and thick slice MRI to establish a model that helps segmentations from thick slice MRI produce anatomically accurate vertebrae shapes suitable for biomechanical modeling. The approach seems to have merit and its result that narrows the gap of segmentation from thick slice MRI as compared with segmentation from CT is of interest. However, in case of acceptance, I would suggest that the limitations of the method are more prominently stated. In the end, super-resolution approaches like the presented is always hallucinating information from the training dataset, which makes truly personalized analysis questionable, due to the danger of introducing unwanted artifacts, or removing available pathologies.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

11

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This is a very well written paper and addresses an often overlooked problem in segmentation of vertebrae from thick MRI slices, by proposing a reconstruction network to map low resolution to higher resolution images, in order to improve the segmentation process. The topic is original, and while I agree with R2 the novelty is limited in terms of the contributions, the framework itself is new and the evaluation is thorough. The rebuttal answered several of the weaknesses raised by the reviewers, although the novelty justification could have been more elaborate. Still, I believe the paper is suitable for acceptance.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

7

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The main concern of the reviewers is insufficient innovation in methodology raised by reviewer 2. I agree with the authors that independently each part of their method is not innovative however, the method as a whole is interesting and the results are promising. My main concern is that k-fold cross-validation was not used and the method is evaluated on a small number of vertebrae. I also expected a baseline method in which MR segmentation is iteratively improved several times to meet the CT ground truth. Finally, I am not convinced that VAE is needed as the refinement step. Standard UNet should do the trick, or maybe even the proposed segmentation network simultaneously fine-tuned also to this task. Nevertheless, I believe that the work has the quality required for publication on MICCAI.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

4

back to top

High-resolution segmentation of lumbar vertebrae from conventional thick slice MRI