Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Christophe Chênes, Jérôme Schmid

Abstract

In many clinical applications, 3D reconstruction of patient-specific structures is of major interest. Despite great effort put in 2D-3D reconstruction, gold standard bone reconstruction obtained by segmentation on CT images is still mostly used – at the expense of exposing patients to significant ionizing radiation and increased health costs. State-of-the-art 2D-3D reconstruction methods are based on non-rigid registration of digitally reconstructed radiographs (DRR) – aiming at full automation – but with varying accuracy often exceeding clinical requirements. Conversely, contour-based approaches can lead to accurate results but strongly de-pend on the quality of extracted contours and have been left aside in recent years. In this study, we revisit a patient-specific 2D-3D reconstruction method for the proximal femur based on contours, image cues, and knowledge-based deformable models. 3D statistical shape models were built using 199 CT scans from THA patients that were used to generate pairs of high fidelity DRRs. Convolutional neural networks were trained using the DRRs to investigate automatic contour-ing. Experiments were conducted on the DRRs, and calibrated radiographs of a pelvis phantom and volunteers – with an analysis of the quality of contouring and its automatization. Using manual contours and DRR, the best reconstruction error was 1.02 mm. With state-of-the-art results for 2D-3D reconstruction of the prox-imal femur, we highlighted the relevance and challenges of using contour-driven reconstruction to yield patient-specific models.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87231-1_44

SharedIt: https://rdcu.be/cyhV1

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The paper presents the results of re-visiting deformable model segmentation where, basically, SSM models are combined with DL architectures to create contours. DRRs are created from CT image stacks of phantom and pre-clinical data, respectively.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

This paper is strong in summarizing the state-of-the-art, in combining the required methods and in creating the required “ground truth” data. Evaluation is done nicely.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The idea of this contribution is to add a deep learning step to the methods required. DL aims at replacing automated or manual creation of contours of the femurs. All other methods are, more or less, state of the art.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The results of this paper will potentially be reproducible.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

See above.
Please state your overall opinion of the paper

borderline reject (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The approach presented is an incremental improvement on existing approaches for femur segmentations.
What is the ranking of this paper in your review stack?

4
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Review #2

Please describe the contribution of the paper

This paper introduce a method for 3D reconstruction of the proximal femoral using 2D/3D registration of a template shape built from a statistical atlas. The method relies on 2D contour segmentation that are extracted with DL approach.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The paper is well written, easy to follow and results are applied to phantoms and real medical data. The method seems applicable and may be of interest for the radiologist community.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The choice of the netwok is not motivated, and it’s difficult to asses how much errors on the contour extraction will influence the final result.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Some details are provided on the data set used to train the network. However it’s not mentioned in the paper if the database will be made avalaible for the community. Very few information are given for the Image forces application which requiers to read the paper in [20]. In addition, the paper does not provide information about the sensitivity of the method with respect these parameters.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

The paper is easy to read, and results seems consistant with authors’ conclusions. A video may help to see visually the results and better understand the method.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Major points are the contribution of the paper that is clearly motivated, and the method seems to provide a consistent solution. The method is applied to real data, and limitation are correclty discussed.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

2
Reviewer confidence

Somewhat confident

Review #3

Please describe the contribution of the paper

This paper presents an approach to reconstruct a 3D model of the proximal femur from a set of 2D radiographs. This is promising for orthopedic surgery, where 3D models of the bone are required for surgical planning and/or intraoperative registration, and such models are still mainly obtained through segmentation of a preoperative CT scan, thus costly and irradiating for the patient. In the proposed approach, contours are first extracted from the X-ray images (either manually or automatically using a Deep Learning-based approach). A deformable model then uses this information to deform a template model and reconstruct the unknown bone shape and pose. The method was evaluated both on synthetic and real data, showing promising results when compared to other approaches found in the literature.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper is well-structured, clear and easy to read. The evaluation protocols are sound and well justified, and the evaluation results provide interesting insights for people working on similar applications.
- The main contribution of the paper is the use of a deformable model that takes into account clinical landmarks and contours for reconstruction a patient specific 3D shape from a template model.
- A quantitative evaluation of the 3D reconstruction performance, both on synthetic and real data (from a phantom and human subjects) is provided. This is really interesting for the reader because it shows that the proposed approach performs well both in data coming from a controlled setup (simulated data) and also from a realistic clinical setup.
- Even if the best results are obtained when manually extracting the contours of the bones in the X-ray, the paper provides an alternative and promising way to extract the contours automatically with an off the shelf deep learning model.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- It is mentioned in the paper that the dataset includes real pathological cases of hip joints. However, it is not discussed if the proposed approach is capable to reconstruct accurately a hip joint with pathologies or damage due to hip arthritis for instance. Indeed, osteophytes (bone deformations) are generally present in those cases and reconstructing their 3D shapes using only a couple of 2D X-ray projections can be challenging. This has been a challenge for similar approaches aiming to reconstruct the distal femur. It would have been interesting to present results when such challenging patient specific deformations are present.
- The performance of the deep learning-based approach for contour extraction is not evaluated directly. It is mentioned is section 3 that Dice similarity coefficient (DSC) was used to assess the 2D image segmentation accuracy, however this evaluation results are not in the paper. It would have been interesting to see in which cases the segmentation fails and then causes the deformable model to fail too.
- A potential issue with SSM is to know how the model can generalize well to a given population. It would have been interest to comment on this and to evaluated how well the SSIM is able to generalize properly to pathological or new unseen data.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

There is an entire section dedicated to implementation details and also the training/testing slits are provided. Hence the reproducibility of this paper is good.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
Please find below a list of questions and comments that I believe can be addressed:
- I would recommend adding a figure illustrating the complete pipeline since this can be useful for the reader to understand the whole method.
- The evaluation results are interesting, but I think more qualitative results could also be interesting for the reader, especially failure cases.
- Obtaining the reference 3D model (used as ground-truth) from segmentation and reconstruction from a CT scan or MRI, can be prone to error too (CT scan protocol, slice thickness, segmentation errors…). In the case of the phantom evaluation, wouldn’t it have been perhaps better to use a perfect CAD model of the phantom as ground-truth instead?
- I don’t really agree that CT segmentation is the only gold standard in Orthopedics. There are several computer-assisted orthopedic systems relying on bone morphing (palpation of bone surface with a tracked probe) to reconstruct the 3D model of the patient’s anatomy.
- What similarity measure was applied to find the closest SSIM?
- It is not clear how the force parameters are set and what is the impact on changing these parameters.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This paper provides an interesting approach for 2D-3D reconstruction using deformable models. The performances of the method are evaluated on synthetic and real data. I believe this can be a good contribution to Miccai.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

4
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

Two reviewers are in favor of accepting the paper. Please carefully summarize the merits and the critique by the reviewers in a rebuttal and pay attention to the points raised by the critical reviewer.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

2

Author Feedback

We would like to thank the reviewers for their constructive remarks, which highlighted the clarity and good organization of the paper. However, we could have improved it further with additional figures such as an overview of our approach, but we were limited by the lack of space. Similarly, the evaluation dataset including simulated and real clinical data was welcomed by the reviews. Finally, reviewers reported the promising transferability of our approach in clinical practice, as well as the interest of our approach for the MICCAI community.

Nevertheless, some valid criticisms and interrogations were expressed by the reviewers. We would like to take this opportunity to clarify misunderstandings and answer questions – by particularly expanding on the points raising major concerns.

DL-based 2D and reference 3D segmentations

Several criticisms focused on various facets of the segmentation, from the technical choices to the impact of 2D and 3D segmentation errors. At the technical level, the U-Net++ and RegNetY DL architectures were favored based on the reported improvements over SOA approaches [21][22] and due to publicly available implementations. In fact, while the DL performances were important, we also favor the reproducibility – our segmentation model trained on DRR will be made available. We also would like to respectfully point out that we did report the performance of the DL -based segmentation approach in the synthetic dataset results section, in which we wrote that the automatic segmentation had a “satisfactory 2D DSC error of 0.957 (CI at 95%:[0.95,0.965])”.

Regarding the impact of 2D errors, we acknowledged the relationship between contour errors and reconstruction accuracy, as we purposely “boosted” the influence of image gradients to mitigate contour segmentation errors of the DL approach. However, we indeed did not quantitatively assess this relationship, and we could adopt in future work the approach of Mahfouz et al. [16] which controlled the perturbation of reference contours to derive a relationship between 2D and 3D errors.

Similarly, reviewers are right to report that segmentation errors may exist in the reference 3D CT models. The suggestion to use a CAD model instead of the phantom is a good idea to get rid of the influence of CT segmentation errors. However, it should be pointed that our phantom includes real human bones without corresponding CAD model. Furthermore, we think that CAD-based machined props (e.g., 3D printed bones) or virtual CAD models would ultimately affect the realism of simulated radiographs. Our phantom is a good compromise between DRR and volunteers: not too pathological, good image quality and above all its structures remain static between shots in AP and LAT. Finally, we agree that CT reconstruction is not the only approach in orthopedics, but it is the reference for surgical planning. The text of the article will be amended to take this nuance into account.

** Pathological cases **

The use of a database including several pathological cases was welcomed by the reviewers but the lack of discussion on the impact of these pathological cases on the reconstruction results was criticized. Given the limited text space and allotted time, we provide some first insights on the impact of pathological cases. We asked 3 radiographers to visually assess the pathological “quality” (e.g., presence of significant osteophytes) of a random sample of 58 femurs from the THA dataset by classifying them into 3 categories: “good” (19 cases), “fair” (15) and “poor” (24). Although not all cases were evaluated, the analysis of the mean ASD for each category (0.98, 1.03 and 0.96 mm, respectively) using reference mapped contours, as well as the DSC of DL-based contours (0.96, 0.94 and 0.97), suggest that both our reconstruction approach and automatic segmentation appear to be robust to pathological variations. We updated the text to reflect these results.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The rebuttal fails to address the central question/criticism of the most negative reviewer, as specifically requested by the AC, which I would reformulate as what’s new technically in the paper. The positive reviews are not so enthusiastic and even critic the experiments. Overall, the paper is not technical novel, does not study a new problem or a new formulation, and has limited evaluation.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Reject
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

5

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The authors propose a method for femur segmentation from multi-view x-ray images, using a model based deep learning approach. The paper is well wrtiten and clear, and results show best-in-class performance.

There were a few issues raised by the reviewers, which the authors have properly addressed, hence I recommend acceptance.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

5

Meta-review #4

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This is an interesting manuscript. It has been argued by the reviewers that there is value in the methodology. To me, not being and expert in the topic, it is not clear the contribution WRT the SOTA, but it might very well be a limitation on my background. This is an interesting topic to MICCAI community.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

12

back to top

Revisiting contour-driven and knowledge-based deformable models: application to 2D-3D proximal femur reconstruction from X-ray images