Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Chenchen Qin, Haoming Li, Yixun Liu, Hong Shang, Hanqi Pei, Xiaoning Wang, Yihao Chen, Jianbo Chang, Ming Feng, Renzhi Wang, Jianhua Yao

Abstract

Brain midline delineates the boundary between the two cerebral hemispheres of the human brain, which plays a significant role in guiding intracranial hemorrhage surgery. Large midline shift caused by hematomas remains an inherent challenge for delineation. However, most previous methods only handle normal brains and delineate the brain midline on 2D CT images. In this study, we propose a novel hemisphere-segmented framework (HSF) for generating smooth 3D brain midline especially when large hematoma shifts the midline. Our work has four highlights. First, we propose to formulate the brain midline delineation as a 3D hemisphere segmentation task, which recognizes the midline location via enriched anatomical features. Second, we employ a distance-weighted map for midline aware loss. Third, we introduce rectificative learning for the model to handle various head poses. Finally, considering the complexity of hematomas distribution in human brain, we build a classification model to automatically identify the situation when hematoma breaks into brain ventricles and formulate a midline correction strategy to locally adjust the midline according to the location and boundary of hematomas. To our best knowledge, it is the first study focusing on delineating the brain midline on 3D CT images of hematoma patients and handling the situation of ventricle break-in. Through extensive validations on a large in-house dataset, our method outperforms state-of-the-art methods in various evaluation metrics.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87240-3_49

SharedIt: https://rdcu.be/cyl6p

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

This paper presents a deep learning network designed to localize the brain midline for hematoma surgery planning.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

This is an important clinical area and a clear fit into MICCAI. The method involves training a network to segment the two hemispheres as well as hematoma, which permit detection of the midline. A weighting scheme focuses attention on the midline region during learning. The atrous spatial pyramid pooling module prior to the decoders is also used as input to a separate branch aimed at learning an affine normalization of the target image (the rectificative learning branch). Finally after an initial midline is estimated, a correction procedure is proposed for cases where the hematoma invades the midline. Results demonstrate better performance against several existing methods. Overall I think it will be an interesting contribution to the conference
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

I do have a few points of critique: (1) It is good to include the ablation study to see the effect of each proposed contribution, but I find the analysis uncomprehensive. The rectificative learning branch seems to have mixed effects – a reduction in Hausdorff distance errors (this is discussed) but also a worsening of average surface distance errors which is not discussed. It requires both the distance weighted map and the midline correction step to re-improve average errors beyond the baseline u-net. As such, I would have expected a more comprehensive ablation study to understand why rectificative learning worsens average errors and whether a network with the distance weight map and/or midline correction without the rectificative learning branch might perform better than the proposed method. (2) With many automated methods previously proposed to segment the midline/falx, this paper is very light on literature review (only 11 references). (3) There are minor grammatical issues throughout.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The method is clearly presented, so it appears reproducible.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

A more extensive literature review and grammar editing would improve the manuscript quality. A more in depth analysis of the results would also improve the paper quality.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Overall I think it is novel and will be an interesting contribution to the conference
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

5
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

The paper presents an automated method for determining the brain midline in patients where the midline is not a simple plane due to the presence of a large tumour. This method has two steps, an initial hemisphere (and hematoma) segmentation method which estimates the midline in more clear and salient regions, and a correction method for when the tumour interferes (i.e. breaks-in) with the midline, thus removing the salient edge between the hemispheres. An additional rectified learning step is used to assist in segmentation training (through multi-task learning) which affine registers the image to a standard template space. To improve segmentation, a distance-weighting scheme is used to make the first step more sensitive to errors close to the midline, a change that appears to yield the biggest improvement in performance even more than the addition of the second stage.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The ablation study is a clear strength of the paper, illustrating the improvement added by each element and showing a connection between the method presented and more traditional, well-understood methods such as U-Nets.

The paper is very clearly organised and implemented, so reproducibility is also a strength.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The main weakness with the paper is the lack of a clear motivation for the correction step outside of agreeing more closely with the annotator. The paper would be better if some brief explanation as to why accuracy around this region in particular is to important clinically.

Paired statistical testing would be warranted to show that the improvements are consistent and not due to random chance, especially given the size of the standard deviation and the noted high patient anatomical variability.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The paper appears to be methodologically reproducible. The methods are described in enough detail for someone to easily reproduce their method and create similar ones. Naturally, if the large CT dataset could be publicly released, that would very much improve the reproducibility of this particular experiment.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Aside from addressing the weaknesses detailed above, the paper would be improved through measuring the user variability of the ground truth midline delineations, especially in the break-in case. This is particularly important as the break-in case appears to have a high effect on the accuracy of the methods as well as on the salience of the midline. Again, supplementing this with information as to why this region in particular is so important would add a lot of motivation to the technical additions in the paper.

There are also some typographical issues to fix (e.g. “To address such issues, To address such issues,” on page 6, “We further conducted ablation study” (an ablation study) on page 7, defining DWM (distance-weighted map?) etc…).
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper is easy to follow, with a good ablation study that gives the reader some technical insights that are non-obvious. Although the methods are potentially more complex than need be, they are far from inaccessible. The paper’s language can be cleaned up with a thorough proof-reading, but it is structurally sound and the mistakes do not truly hinder the paper’s accessibility.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

5
Reviewer confidence

Very confident

Review #3

Please describe the contribution of the paper

A 3D midline model is utilized to segment the border between the hemispheres that is applicable for patients with midline shift too. A holistic approach is presented for computer aided diagnostics and surgery (correction) planning in case of haematomas. Besides the 3D matching midline model, the haematomas and potential ventricle break-ins get detected, classified and a strategy for correction is provided. The rectification of the input dataset leads to improved results. Proposed correction planning and landmarks are not tackled in the paper in a comprehensive and classification only in a quite shallow way. Approach itself seems sound, comparison to baseline models convincing.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Pragmatic approach based on probability & distance maps, and 3D shape rectification.

Related work section is very comprehensive and solid with utmost precision.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Naming this pipeline a hemisphere-segmented framework (HSF) is a bit exaggerated but it’s a nice and sound approach anyway.

It is unclear why “Rectificative Learning” is required within the deep learnin process. The pose alignment can be achieved a priori utilizing conventional registration approaches – so unclear, why this complex part is not handled in advance.

Description of the approach in a quite narrative way. Hardly any details regarding the implementation are provided, the optimizer and batch size is never mentioned – only the custom loss function is explained. Consequently, it is not reproducible. As well, the mentioned landmarks are never explained how they are detected to be used for the weighted distance map then.

Approach is nice and sound but 8 page format does not allow to provide all details as it seems.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

“challenging in-house dataset containing 300 3D brain CT images with hematoma.”  neither data nor algorithms available
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Fig.1: “Yellow line in (d)”  in the PDF provided it is clearly orange by the way. What means “Wang et al. Pisov et al. [6] proposed”  its Pisov et al., isn’t it? “(2) they resort”  not a good start for a sentence after a “.”

Abstract vs. Methodology: either 4 or 3 steps overall – it should be consistent Equation (2): either end with a “.” And/or consider a better begin of next sentence with variable w_i! What are the units in Fig.2 – scale of LUT [1.0;2.65]??

Unclear how the classification accuracy is calculated (3 class problem) On the testing dataset, the classification accuracy is about 91.4%. Of course one can imagine… but stating the error metric in an explicit way is always preferable Comparison to “U-Net, RLDN and CAR-Net” is fine, but “re-implemented and optimized parameters” seems to be a bit unclear… And as your approach follows the U-Net paradigm too with some hemisphere specific approaches and rectification – it is not a big surprise that you clearly outperform this baseline model then.

“When the midline correction strategy is employed, the mean HD and ASD decrease by 0.08mm and 0.05mm respectively.”  and same improvement if previously registering the 3D datasets?? Fig. 5 “Yellow arrow indicates the right midline correction..”  remove one “.” And still its rather orange color.
Please state your overall opinion of the paper

borderline reject (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Some inconsistencies and unclear aspects (why rectification instead of a priori registration), where are the landmarks (used), what parameters are utilized and lack of reproducibility.
What is the ranking of this paper in your review stack?

4
Number of papers in your stack

4
Reviewer confidence

Confident but not absolutely certain

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

Reviewers commented that the application and the methods outlined in the paper are highly suitable for MICCAI. One of the strengths was the ablation study.

One concern was the necessity of rectificative learning.

This is clearly an important application, especially as in the case of tumors/lesions/hematoma, standard methods for midline detection will fail.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

2

Author Feedback

Reviewer #1: Q1: It is good to include the ablation study to see the effect of each proposed contribution, but I find the analysis uncomprehensive. The rectificative learning branch seems to have mixed effects – a reduction in Hausdorff distance errors (this is discussed) but also a worsening of average surface distance errors which is not discussed. It requires both the distance weighted map and the midline correction step to re-improve average errors beyond the baseline u-net. As such, I would have expected a more comprehensive ablation study to understand why rectificative learning worsens average errors and whether a network with the distance weight map and/or midline correction without the rectificative learning branch might perform better than the proposed method? Answer1: Thank you for the question. We realize that the average surface distance errors are worse than the baseline when rectificative learning (RL) is equipped. RL module aims to reduce the effect of various head pose. We assume that introducing the RL loss only may lead to the attention reduction for midline area. Certainly, we need design more comprehensive experiments to prove that.

Q2:With many automated methods previously proposed to segment the midline/falx, this paper is very light on literature review (only 11 references) Answer2: Thank you. According to our literature investigation, most previous works adopt the conventional methods for brain midline delineation with limited validation. In this paper, we only compare our proposal with those deep-learning based methods proposed in the last few years(2014-2019), which is more comparable.

Reviewer #2: Q1: The main weakness with the paper is the lack of a clear motivation for the correction step outside of agreeing more closely with the annotator. The paper would be better if some brief explanation as to why accuracy around this region in particular is to important clinically. Answer1: Thank, you for the suggestion. We had prepared a picture to show the detail of correction step. However, for the lack of space, we have to delete it and explain that with a few sentences

Reviewer #3: Q1：It is unclear why “Rectificative Learning” is required within the deep learnin process. The pose alignment can be achieved a priori utilizing conventional registration approaches – so unclear, why this complex part is not handled in advance.. Answer1: Thank you for the question. The rectificative learning (RL) module aims to help the model (HSF) adapt various head pose input. The reason why we don’t preforming the alignment in advance is that we prefer to design an end-to-end model for midline delineation. Our proposed method can handle any head pose situation without alignment before.

back to top

3D Brain Midline Delineation for Hematoma Patients