Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Florian Kordon, Andreas Maier, Benedict Swartman, Maxim Privalov, Jan Siad El Barbari, Holger Kunze

Abstract

Reconstruction surgery of torn ligaments typically requires precise and anatomically correct fixation of the graft substitute on the bone surface. Several planning methodologies have been proposed that aim at standardizing the interventional procedure by localizing drill sites or defining the drill tunnel orientation with the help of anatomical landmarks. However, the practical implementation is limited by the often complex and time-consuming nature of the planning steps. For this reason, we propose an automatic solution for safe guide pin path planning based on bone contour extraction, axis detection, anatomical landmark detection, and geometrical construction. We evaluate our approach for the task of double-bundle posterior cruciate ligament reconstruction surgery on the lateral tibia using 38 clinical X-ray images. Our method achieves a median path angulation error of 0.37° and a median localization error of 0.96 mm for the ligament attachment center.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87202-1_54

SharedIt: https://rdcu.be/cyhQ9

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The authors developed a method to safe guide pin insertion in PCL reconstruction surgery
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The authors impressively integrate state-of-the-art methods into an application that satisfies a real clinical need.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

validation of the method can be improved. For example, what is the inter- and intra- observer variability? What accuracy is required from a clinical perspective?
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Clear and reproduceable.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

There are some typos in the text, please fix. I think I am missing the bigger picture. The 2D errors are part of a larger operation pipeline. Please describe it. Please add some segmented and augmented x-ray examples for qualitative impression.
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This is a nice integration of state of the art algorithms for a usefull application. The innovation level is just adequate for MICCAI, not ground breaking.
What is the ranking of this paper in your review stack?

4
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Review #2

Please describe the contribution of the paper

The method represents an important step towards the registration-free intraoperative planning of ligament reconstruction interventions.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Integration into the existing surgical workflow would be possible.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Intraopeative fluoroscopy would be employed if the proposed method would be used for surgical navigation. However, the evaluation was performed on conventional X-ray which may have considerably higher quality than fluoro-shots.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Based on the checklist, the reproducibility can be rated as sufficient.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- Although the paper contains drawings for explanation, a Figure showing real-world example data of each pipeline step would have been helpful.
- ROI / bone axis: How much of the anterior/posterior shaft part has to be on the fluoro-shot in order to calcualte a reliable bone axis?
- Section 2.3: What is the meaning of the “/” in “… feature extractor consists of 167/16 conventional X-ray images”?
- Results: There is a missing reference to a section in results/pin path planning.
- Results: How was the ground-truth pin path defined?
- Whats the total computation time of the approach?
Please state your overall opinion of the paper

borderline reject (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

A method which is intended to be intraoperatively used should be evalauted on intraoperative fluoroscopy images and not on coventional X-rays.
What is the ranking of this paper in your review stack?

4
Number of papers in your stack

5
Reviewer confidence

Very confident

Review #3

Please describe the contribution of the paper

The authors present a method for automating the path-planning step for safe insertion of a guide pin, specifically in PCL reconstruction surgery. The method consists of 2 stages: (a) A multi-task segmentation/extraction process to identify the tibial bone contour, regions of interest, and a specific landmark, followed by (b) Some geometric constraints to identify the correct trajectory of the guide pin. Of these, the learning task is based on prior work in [8], leaving section 2.2 as the main contribution of this paper.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Presumably, the ability to automate this procedure is an important task, as many reconstruction surgerys are performed each year. The method is novel, and clinically well motivated.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

This appears to be an extension of the work in [8], customised to work for the tibia and this type of surgery. However, the paper suffers from a lack of context, lack of explanation of what the clinical requirements are, and hence it is difficult to assess how well the method is really working.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The paper does not promise code or data. The geometric path planning method is described in detail, whereas the learning task is summarised substantially.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
The paper is clear enough, but lacks context. I appreciate space is limited, but can the following be commented upon?
1. Please specify the clinical requirement for angle and localisation error. Then in the discussion, please comment on performance relative to these benchmarks.
2. How important/frequent are these procedures? What might be the cost saving of automation? Please say something about the impact of such work, if successful. Is it just time-saving during pre-op planning, or would it facilitate a change of process intra-operatively. In the latter case, automation is essential, and so would increase the value of this work.
3. You have a 2 stage process, and provide some justification in the Discussion. You are justifying it by saying that various components can be re-used for other purposes. But in real-terms, keeping 2 stages separate is wrong if a 1-stage process ultimately outperforms the 2 stage. Ideally, I’d like to see a comparison, or at least a comment on the future direction of this work. Do you have evidence to say that a 2-stage process is ultimately better than a single (presumably ML based) approach?
4. Given that the main novelty of this paper, is the geometric path planning. Please can you spend more time analysing the results, rather than focussing on the segmentation?
Various minor edits:
1. “were proposed” -> “have been proposed”
2. “or requirements to pre-operative” -> rephrase, maybe: “or the requirements for pre-operative”, but I don’t understand what you are trying to say.
3. 3x3 filter structure, doesn’t look like a 3x3 matrix to me. Are they matrix entries, or coordinates of positive pixels…. in which case it’s a cross? Perhaps just say its a 3x3 cross?
4. Various things like the main contour, vectors etc are not labelled on Figure 1.
5. “therefor” -> “therefore”
6. In results, “(Subsec. ??)”, missing or invalid cross ref.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Overall, this is a good paper. The context could be explained a bit better, which would itself lead to better self-reflection, and analysis of the results. The results of the geometric path planning look a bit preliminary, and it is difficult to understand if the results are actually good or useful, or if there is some way to go before clinical benchmarks are met.
What is the ranking of this paper in your review stack?

4
Number of papers in your stack

4
Reviewer confidence

Somewhat confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

This MICCAI submission, titled “Automatic Path Planning for Safe Guide Pin Insertion in PCL Reconstruction Surgery” was reviewed by 3 senior reviewers in the CAI community. It should be noted that while the reviews were generally positive, this paper is consistently ranked at the bottom half of reviewers’ assignments, suggesting their recommendation may be overly generous.

R1 appreciates the quality of writing (although quite a few typographical errors and missing references are noted), and requested additional visual aid for qualitative accessment, R2 raised quite a few key questions, suggesting that key details are missing in the current form, R3 also raised a few key questions, focusing on the clinical context of the research.

Given their reviews and my personal reading of the manuscript, I am recommending the decision to invite authors to submit an rebuttal in response to authors’ questions. Please note that this is a request for rebuttal, and not a re-submission, thus authors are discouraged to provide “promises” of what the re-submission would be. Instead, the rebuttal should aim to clarify/answer any misconception/questions the reviewers raised.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

6

Author Feedback

Dear Chair, Thank you for the valuable comments that you and the reviewers have shared with us. We would like to discuss the most important points of criticism and suggestions for improvement. The goal of our development is to support the surgeon in the planning of a PCL reconstruction surgery. From discussions with our medical advisers, we know the importance to provide a tool that fits smoothly in the operation workflow, performs calculations they can follow, and provides means with which the automatic planning can be easily adapted. As we will discuss, the proposed method allows exactly for that. To address the major concerns of the reviewers R1 and R3, we want to clarify the clinical context of our work. While isolated tears of the PCL are rare injuries with an annual incidence of 0.65%-3% of all knee injuries, they often accompany concurrent ligament injuries and significantly decrease joint stability if left untreated. Recently, double-bundle reconstruction has become one of the primary techniques for surgical management: After identification of the anatomical graft attachment sites, a guide pin is drilled into the tibia and is subsequently over-reamed with direct arthroscopic guidance. After tunnel smoothing, the replacement graft is fixated with inference screws. During this procedure, a correct positioning of the pin and ultimately the drilling tunnel is crucial for an anatomical reconstruction and helps to prevent graft abrasion and premature cortex breakout with potential injury to the neurovascular bundle. Our method smoothly integrates into this key phase of the surgical workflow. Before insertion of the guide pin, an intra-operative fluoroscopic lateral projection is acquired and processed by the path planning algorithm. The planning is then overlayed on the X-ray image and provides visual guidance for a safe drilling path to the surgeon. Besides angulation and position of the path, the method also accounts for the drill width and incorporates a safety margin (cf. Johannsen 2013). We further want to address the question of R1 and R3 concerning the required accuracy from a clinical perspective: With an anatomical variance of the PCL tibial attachment site of 1.2mm w.r.t. the CGD reference landmark (cf. Anderson 2012), our results are well within the clinical acceptance range. We agree that this clinical context and the benefits of our method must be explained in more detail and will improve the introduction accordingly. As noted by R3, implementing a single-stage algorithm would have been easily achievable. However, a 2-stage approach allows the user to interactively adjust the location of the inferred anatomical features and enables modification of the path proposal in near-real-time (execution time for the geometric steps averages 0.5s on Intel Core i7-7820HQ). The ability to display all relevant anatomical structures also allows for better interpretability and overall acceptance of the method by the clinician. Additionally, the segmented bone can be used to constrain the registration of the planning on subsequent live images. In the context of intraoperative imaging, this is especially important since patient-device orientation changes slightly over time, and patient anatomy is typically not fixated. We will add this reasoning to the paper and thus improve the motivation of our approach. R2 and R3 both agreed that a weakness of our work is the lack of an inter-rater study. We observe great accuracy w.r.t. the ground truth, which however might be weakened when compared to human experts. Therefore, we seek to validate our results in a follow-up study with medical readers. Due to space limitations, we decided against integrating such an analysis in this paper. We thank the reviewers for their thoughtful critiques and believe to have successfully dispelled all concerns. We are confident that the proposed changes and further work on the reviewers’ minor comments will lead to a great paper to be presented at MICCAI.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

These authors have clarified the clinical context/motivation of the proposed work and the research topic is indeed fitting for MICCAI. As a CAI-based paper, it falls under the category of (https://miccai2021.org/en/REVIEWER-GUIDELINES.html) “2. Demonstration of clinical feasibility, even on a single subject/animal/phantom”. The method was described in sufficient detail hence reproducibility is not an issue as judged by all reviewers. To increase the significance/potential impact of this work, authors are strongly encouraged to release an open-source implementation of their work to further benefit the CAI community. After review the rebuttal, I recommend the decision to accept this manuscript. The clarifications made in the rebuttal would reflect minor changes in the narrative of the manuscript, hence the quality of the “revised” manuscript can be maintained.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

9

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The papers addresses a relevant clinical need by presenting a framework for automating safe guide pin path planning for PCL reconstructing surgery. The paper is well-written and well motivated. Some of the main concerns from reviewers around the validation results and context, including missing accuracy requirements and clinical context for the validation experiments, have been addressed by the rebuttal and should be incorporated in the final manuscript if accepted.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

7

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This AC would question what’s the clinical significance of this work. The technical novelty was still unclear.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Reject
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

10

back to top

Automatic Path Planning for Safe Guide Pin Insertion in PCL Reconstruction Surgery