Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Long Huo, Bin Cai, Pengpeng Liang, Zhiyong Sun, Chi Xiong, Chaoshi Niu, Bo Song, Erkang Cheng

Abstract

Spinal curvature estimation plays an important role in adolescent idiopathic scoliosis (AIS) evaluation and treatment. The Cobb angle is a well-established standard for spinal curvature estimation. Recent studies of Cobb angle estimation usually rely on detection of vertebra landmarks which requires complex post-processing of curvature calculation. Approaches directly regress the Cobb angles apply entire image or centerline segmentation results as the network input, which limits exploring the specific curve structure of the spine. In this paper, we propose a deep learning-based approach to simultaneously estimate spine centerline and spinal curvature with shared convolutional backbone. The spine centerline extraction is formulated as a row-wise classification task. To directly regress Cobb angles, we adopt curve graph convolution to exploit curve structure of the spine centerline. In addition, given a spine centerline, a Curve Feature Pooling (CFP) module is designed to aggregate features used as the input of Curve Graph Network (CGN) to regress the Cobb angles. We evaluate our method on the accurate automated spinal curvature estimation (AASCE) challenge 2019, and the proposed approach achieves promising results on both spine centerline extraction and Cobb angels estimation tasks.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87240-3_36

SharedIt: https://rdcu.be/cyl6a

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The authors proposed a deep learning method that estimates the Gobb angle for adolescent idiopathic scoliosis (AIS) evaluation from X-ray images.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The model is well described and well-validated using an available dataset.

The proposed method is proven to provide better Gobb angle estimation than that provided by state-of-the-art methods, thanks to the new block in the model that exploits geometric information with CGN.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The work sounds good. I do not have negative comments.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The model is well described and the dataset used is available. I think the reproducibility of the work is fair enough.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

I suggest to accept as is.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper is well written.

Technical details are listed clearly.

The validation is convincing and a comaprision with several recent methods is conducted.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

3
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

The paper proposes a method to estimate Cobb angle in spine AP view X-rays by combining in a novel way two ideas: I) detecting spine centreline with row-wise classification of global features using UltraFast method that was used in road lane detection, ii) regress spine curvature angles using 1D convolution on the detected spine centrelines which is inspired from DeepSnake that uses convolution on contours for instance segmentation.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Proposes a novel way to combine two existing ideas (applied to other applications) for Cobb angle estimation in spine.

The proposed method can take benefit of spine’s structure and vertebrae positioning by using similarity loss (Eq. 3) during centreline extraction, convolution on extracted centreline curve, and coordinates of the detected centreline as features in curve feature pooling.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The details regarding how the annotations used on the test/evaluation set is not clear which is important to assess the results of comparison to other methods. It seems that the results of other methods and proposed method shown in Table 1 are not directly comparable (more on detailed comments)

The description and presentation of the key steps (Sec 2.1 and Sec 2.2) could be substantially improved as it is currently difficult to read and details are missing (more on detailed comments).
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Test data annotation and how Cobb angle is estimated from landmarks are not clearly mentioned.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

How are row anchors predefined? I believe w and h of feature is much smaller than the input image width and height W & H. Which layer is used, and what are their sizes for the images used in the network? One pixel position in the feature map layer corresponds to a patch in the input image. How big/small is this patch? The receptive field and the layer which is used is not clearly written in the paper. We find partial answers once we arrive at Sec 3.2 (h=205, w=101 for specific image size), and then only realize that the number of row anchors gets decided once we choose feature layer. These must be explained clearly in methods section. Also is this h and w output or input of the centreline network?

It is not clear how the row-wise centreline classifier taking “global feature”; does it not depend on the receptive field of the features input to the classifier?

The caption of Fig 2, and its corresponding description in the main text could be improved, taking reference of the color (and numbers) of the specific example used in the figure which has 7 detected centrelines. Notations such as N are used without defining them, leaving readers to infer what they mean. This makes this section difficult to read.

The AASCE dataset has 98 images in the test set whose annotations are not public. The challenge ranking are based on the results in this test set which has a domain shift (size of the image, presence of skull and femur etc. Which is not present in training set). Most competing methods have reported good performance in validation set of 168 images but reduced performance on test set. The proposed method seems to have reported result on the easier 168 images, but the way results are presented in Table 1 is misleading. It is not clear if the proposed method actually outperforms the other reported methods unless all these methods compare results on the same test set.

The Cobb angles are completely described by the location of the centrelines. Thus, I am not clear why the features of the centreline that encodes appearance help in angle estimation. If centrelines are accurately found, should it not suffice to use only the coordinate information? The motivation or reasoning of why appearance information in features improve results of angle estimation when these features are added to coordinate information.

1st paragraph of Introduction motivates why Cobb angle estimation is important in Scoliosis, but it would be better to cite relevant clinical papers here to provide evidence to this need.

Sec 1 mentions that, “These regression networks take entire image or centreline segmentation masks as input, which limits exploring the specific curve structure of the spine”. The entire image and centreline segmentation (if properly segmented) both have complete information of curvature. So how does it limit “exploring” (rather exploiting?) the specific curve structure? Perhaps, what authors wanted to say was that those networks do not explicitly exploit the curve structure?

Typos: Abstract: “directly regress” -> “directly regressing” Sec1: “In clinical study” -> “In clinical studies”; “We category deep” -> “We categorize deep”; “extration task” -> “extraction task”; “a efficient” -> “an efficient” Sec2.1: “mathematically, the feature” -> “mathematically, a feature”; Sec2.2: “Totally, a feature” -> “Finally, a feature”?; Sec3.1: “ASSCE” -> “AASCE”
Please state your overall opinion of the paper

probably reject (4)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Although the proposed method provides a novel combination of the existing method suitably adapted to Cobb angle estimation problem, the method description must be improved to make clearer including the reasons for particular choices made such as which layer is used for row anchors. Similarly, in comparing against state-of-the-art, other methods have used an independent test set with a domain shift that is more difficult than the validation set used by the proposed method which is prone to overestimating the performance due to overfitting.
What is the ranking of this paper in your review stack?

3
Number of papers in your stack

5
Reviewer confidence

Very confident

Review #3

Please describe the contribution of the paper

A joint network introduced that can estimate both spinal centerline and Cobb angles directly without postprocessing. Curve graph network is proposed to use in order to explore curve structure of the spine.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Nicely written and easy follow.
- A novel method is introduced that allows to extract spinal centerline and estimate Cobb angles directly
- The proposed method is properly evaluated on public challenge data. Results are very promising, seems the proposed method could actually outperform the winner of the challenge.
- Ablation studies on input types for Cobb angle regression are shown
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

No major weaknesses
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

All information is provided in order to reproduce the results
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Very nicely written paper with very clear contributions and evaluations. Always happy to se ablation studies, it always makes the submission stronger.

I don’t have other comments. It’s a great work and enjoyed reading it.
Please state your overall opinion of the paper

strong accept (9)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- Very nicely written paper with very clear contributions and evaluations.
- Ablation studies are there too and results are convincing on both Cobb angle estimations and when compared against other participants of the AASCE challenge.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

6
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

Reviews are mixed for this paper. Two of the reviewers were quite positive, highlighting the overall clarity of the paper and proposed workflow, novelty of the technique and several experiments to demonstrate the Cobb angle regression outperforming other techniques.

However R#2 is more critical of the paper, particularly regarding how the tests/evaluations were performed with the annotations, and questions whether the results obtained with the proposed method are really comparable to other techniques since datasets are different. Some key aspects of the method are also poorly described according to R#2.

In the rebuttal phase, authors should address these outstanding issues with particular emphasis on the concerns raised by R#2.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

5

Author Feedback

We thank all the reviewers (R1, R2, R4 and meta-reviewer) for the constructive suggestions and the positive comments.

Q1 (R2 & Meta): Proposed method is really comparable to other techniques since the datasets are different?

A1: Thanks for raising this question. Our method is evaluated on the public dataset from AASCE challenge 2019. The dataset has 609 public X-ray images that are split into 481 training images and 128 validation images by the challenge organizer. There are also 98 blind testing images, but they are not available after the challenge. Therefore, our method is evaluated on the public 128 validation images. Specifically, the results of evaluation metrics SMAPE and Angles are reported in Table 1. For the metric SMAPE, top ranking methods are evaluated using the 98 blind testing images. Since we can not access the 98 blind testing images, we just report the results on the 128 validation images. For the evaluation metric Angles, the results of the winner method [5] using the 128 validation images are reported, and our method outperforms it with a notable margin. We will make the description more clear. In this paper, we present a novel method to estimate the Cobb angle by exploiting the specific spine centerline structure, and jointly output the spine centerline.

Q2 (R2): (1) Why appearance features improve angle estimation when added to coordinate information. (2) how does entire image/centerline segmentation limit exploiting the curve structure?

A2: Yes, we agree that given the accurate spine centerline, the Cobb angles can be completely obtained. However, finding the accurate spine itself is a challenging task. As shown in Table 2, directly using the estimated centerline can not provide accurate Cobb angle results. Instead of straightforward using the centerline to estimate the Cobb angle, some previous works utilize input image or spine segmentation mask as network input to predict the Cobb angles. We think that the entire image or spine centerline segmentation mask may have many irrelevant pixels besides the desired centerline context. For example, the entire image has massive irrelevant pixels of the spine. Therefore, features extracted from these regions are not discriminative enough to represent the specific curve structure of the spine centerline. To deal with the above problem, we propose to use a curve feature pooling module to extract the specific anatomical structure features defined by spine centerline locations. In addition, we empirically find that the extended feature improves the angle estimation result by aggregating deep features on centerline (CFP) and centerline coordinate (Coord).

Q3 (R2): How are row anchors predefined? How big/small is this patch?

A3: Thanks for the valuable comments, and we promise to improve the presentation of our method. The size of the input image is 1024x512x1 (HxWxC). We use resnet-34 as the backbone, and it provides feature maps with size of 32x6x512 (output of conv5) that is 1/32 of the input image. For centerline extraction, the feature maps from the backbone are further processed to produce the centerline output maps with size of 205x101 by several FC operators. Therefore, one pixel in the centerline output feature map layer corresponds to a patch of size around 5x5. The centerline output map size is chosen with similar settings of UltraFast [9]. h and w are used to indicate the size of the output of the centerline network. For the Cobb angle estimation task, we use the output of conv5 of Resnet-34 as the input feature map to the curve feature pooling module.

Q4 (R2): It is not clear how the row-wise classifier takes global feature.

A4: Thanks for raising this question. Like UltraFast [9], we use several FC operators after the Resnet backbone to compute the row-wise output, and these FC operators are able to construct a receptive field that covers the entire image. So, the row-wise classifier can leverage global features to predict spine centerline.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

Based on the information in the rebuttal, the major comments regarding the method’s comparison to other techniques presented in Table 1, anchor definition and method’s description, seem to have been addressed. Given the general enthusiasm about the method from the reviewers, I would recommend accept.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

8

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The authors presented an interesting direction for assessing the curvature of the spine and I share the enthusiasm with the two reviewers that the paper will attract great interest at the MICCAI conference. However, I agree with the third reviewer that the fairness of the comparison in Table 1 must be clarified which the authors promised in their rebuttal phase. I also agree with the third reviewer that more details on the method could be provided to make the manuscript self-explanatory. However, I consider this a minor criticism.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

1

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This submission proposes a network to jointly extract spline centerline based on row-wise classification and estimate cobb angle based on curve feature pooling and curve graph network. Overall, the manuscript is well-written, and the proposed network is novel. The authors addressed most of the concerns raised by Reviewer 2. The submission is recommended for acceptance, while clarification and improvements are encouraged to be made in the final version.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

1

back to top

Joint Spinal Centerline Extraction and Curvature Estimation with Row-wise Classification and Curve Graph Network