Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Nicole Varble, Alvin Chen, Ayushi Sinha, Brian Lee, Quirina de Ruiter, Bradford J. Wood, Torre Bydlon

Abstract

Endobronchial biopsy is the preferred method for assessing lung lesions. However, navigation to pulmonary lesions and obtaining adequate tissue samples for diagnosis remains challenging. Utilizing information from high-resolution pre-procedural CT scans intra-procedurally could provide real-time guidance and confirmation during biopsy. An image registration algorithm was developed to automatically fuse thoracic 3D pre-operative CT images to 2D intra-procedural fluoroscopic images with a single 2D image or a limited C-arm sweep. A rigid intensity-based technique was applied and the CT image was iteratively transformed to minimize the sum of squared error between intraoperative fluoroscopy and closest forward projections. The registration errors were measured by computing the sum of squared difference and manually identified fiducial markers. In a swine model, error was minimized when using a CT with an inhalation breath hold (7.7±4.4mm) and when using an anterior-posterior positioning of the C-arm (3.7±2.4mm). Error increased marginally when the FOV was decreased (10.9±5.9mm) and was larger in peripheral (9.7±5.7mm) and distal (9.2±3.2mm) lung, compared to central (6.2±4.5mm) and proximal (7.6±5.9mm) lung. To determine the features that contribute most to registration, features were systematically masked and registration was performed. The largest error was seen when the spine was masked (52.5±27.6mm). When multiple images were used for registration, error converges (<5% change) when 50 images acquired in a 100-degree sweep were used. This work establishes a protocol and identifies sources of registration error for a reliable and automatic 2D-3D registration method that requires minimal changes to procedural workflow and equipment in the endobronchial suite.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87234-2_32

SharedIt: https://rdcu.be/cyl8s

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

This paper is about 3D/2D registration of X-ray images (one or a sequence) for bronchial biopsy guidance. Based on intensity-based registration using SSD, the 2D images are matched to DRR generated from the CT. The method is initialized at hand. Particle Swarn Optimization is combined with a quasi-Newton method for SSD minimization. Tests on swine using implanted fiducials are reported. Seems to be rigid registration but this is never explicitely stated.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The paper introduces properly the rationale for this work A method is implemented and tested on living (thus breathing) animals many experiments have been conducted
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Some important information about the method itself or its evaluation is missing. The registration method present no novelty (or the paper fails to transmit this) The results should be interpreted much more in depth
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The lack of some important information about the method and its evaluation makes the method hardly reproducible from this sole paper.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
Registration is probably rigid but this is not specified and MUST be. Abstract:
- The abstract mentions a single image 2D image (how is it possible to recover the 6 degrees of freedom from a single 2D image – even a projection). It mentions also a “limited C-arm sweep” as input. In the rest of the paper, the way a spatial sequence of images is handled is not explained properly. Introduction:
- Well motivated work.
- Information about the requirements (which accuracy is necessary for guidance of a needle) could be a useful input for interpreting the obtained results.
- Several references are not really directly related to the presented work (robotics, navigation); some of them might be suppressed. I would find more relevant to find references to recent papers using deep learning techniques for 2D/3D registration of X-ray images. This should probably be discussed also in the state of the art. Methods:
- 2.1: replace “was generation” by “was generated”
- Registration is probably rigid but this is not specified and MUST be.
- Regarding the case of image sequences, the computation of an average transform is not totally meaningful to me. Is it a time sequence or a spatial sequence? The average value would correspond to a time sequence without movement. In case of a spatial sequence (end of section 2.3, page 6), the process is not clear at all. Do you initialize the registration of one image with the result of the previous image in the sequence?
- In 2.3, it is written that the impact of the user initialization has been analyzed but it is not discussed in section 3. Moreover the corresponding graph in figure 3 is confusing; the color bars meaning seems wrong: shouldn’t it be the angles axes (x,y,z)?
- The input of the user for initializing the transform is not clear: does he input angle values ? won’t it be very difficult for non-standard positions (other than AP, lateral and mid-way)?
- Important information is missing about the coils: what are their shape and size? Do you locate their center/extrimities? And more importantly how accurate (repeatable) is their manual localization? These numbers are useful to interpret the results.
- How do you measure the registration error? Could you provide the detail of the computation? Results:
- It would be interesting to interpret the results: regarding the dependency to the obliquity of the image, does it show that some motions cannot be captured properly with some of the incidences? The orientation of coils (and their visibility) w.r.t. to the image angle certainly impacts the results.
- Regarding the high increase of error when masking spine: does it mean that spine contribution to the SSD (due to the contrast or size of the structure) is much higher that for the rest? Consequently, does it mean that you register the spine (and not the bronchial tree information)? Conclusions:
- Replace “breadth” by “breath”
Please state your overall opinion of the paper

probably reject (4)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The method is questionable and some important aspects are not discussed.
What is the ranking of this paper in your review stack?

3
Number of papers in your stack

5
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

This paper presents an automatic 2D-3D image registration workflow for pre-operative CT and intra-operative X-ray images, and analyzed the contributions of different sites’ features to image registration.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The authors presented a reliable and automatic 2D-3D image registration workflow for pre-operative CT and intra-operative X-ray images, which would be helpful in intra-operative guidance.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The novelties of this paper were not clear. It seemed the chief contribution of this paper was to develop a registration workflow, but without much theory novelty.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Not very clear.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

The authors should add some details of their registration procedures and the corresponding mathematical deduction. Besides, the visual comparison results should also be added for direct observation.
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This research might be useful in clinical applications, but the details of this work were not that clear.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

2
Reviewer confidence

Somewhat confident

Review #3

Please describe the contribution of the paper

The authors describe a protocol for assessing registration error between pre-intervention 3D CT of the chest and intra-procedureal 2D fluoroscopy for potential use in clinical practice for endobronchial lung lesion biopsy. While a number of technologies exist to aid guidance and increase diagnostic yield for endobronchial lung biopsies, the authors show using a non-rigid co-registration algorithm there is more accurate overlay (3-8 vs >9 mm) with larger FOVs, in central locations and using more fluoroscopy images.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The greatest strength of this paper is the methodology. All factors that might influence registration (and thus biopsy accuracy) are taken into account: respiratory state, amount of c-arm sweep and images used, target location (peripheral vs central, prox vs distal), and influence of different structures (by masking/ablating). The observation that heart and diaphragm masking led to translation errors and ribs and spine rotation errors was particularly interesting.

Another strength is the application in a large animal model that is as close to human anatomy as is feasible for lung interventions.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Details on the speed of the co-registration algorithm are missing and would be helpful in understanding the translatability.

The effect of the fiducial markers themselves on the registration algorithm are not discussed, and at least from the reviewer’s experience may play a role in accuracy itself.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Based on the methods description the experiments described appear as reproducible as can be expected from a CAI standpoint. It would be helpful to better understand specific methods used for forward projection of the 3D volume.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Determining a way to include fiducials that do not themselves influence co-registration is challenging. To start, picking a fiducial that is less conspicuous than a large coil may be helpful. Perhaps consider using breast biopsy markers.

Presentation of images such that the cranial direction of the animal is at the top of a figure is recommended (right side on the left side and vice versa, per radiology convention). It will avoid distracting readers who view medical imaging professionally.

Perhaps as a next step in validating this technique segmented airways can be provided with the fluoroscopy overlay showing the proposed biopsy paths, as mentioned in the paper’s introduction. Figures showing this augmented visualization process across multiple subjects with different target locations would be especially compelling.
Please state your overall opinion of the paper

strong accept (9)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper addresses an unmet clinical need, with large animal experiments and thus potential clinical translatability. The work thoroughly examines key factors that influence non-rigid co-registration performance in chest imaging.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

3
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

Two reviewers questioned the novelty of this paper. Some technical details were also missed. The computational time is important to clinical applications.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

7

Author Feedback

We appreciate the reviewers’ thoughtful commentary and guidance that will be used to improve and clarify points in the paper. Please find below our response which focuses on the major critiques and groups multiple reviewer comments in the same area. Given the opportunity to improve the paper, these and minor points can be incorporated.

Novelty: Reviewers stated “novelties of this paper were not clear,” specifically regarding the registration method. This work is novel because it presents a complete and fully functioning 2D-3D image registration pipeline with an accompanying analysis that details the contributing factors of error in an in vivo study. Achieving low registration error is challenging in soft tissue such as the lung and is further complicated by the presence of respiratory and cardiac motion. Identifying sources of error in this realistic experimental environment with a systematic approach is highly clinically relevant and produces results that have not been previously reported. Knowing which anatomical features are relevant to registration may impact procedure decisions such as patient positioning. Additionally, knowing where registration error is the greatest will enhance the clinician’s confidence and interpretation of an augmented 3D overlay.

Technical details: As R1 stated, “Some important information about the method itself or its evaluation is missing.” The following points can be clarified in the paper: a. Registration method: The registration was rigid and, when multiple images were used, was initialized with the result of the previous images in the sequence. b. Measuring error: Error was measured in 2 ways. First, to quantify the target registration error, 4 endobronchially placed coils were manually identified. A user marked the location of the coil on the 2D fluoroscopic and the registered image. When co-registered the Euclidean distance was measured between the pair of points. Then, the sum of the squared distances (SSD) was quantified to measure the difference in the overlaid image intensities. In a previous analysis SSD was a good surrogate for manual error measurement and is more efficient since it decreases the number of manual measurements required. c. Fiducial makers: The fiducial markers were stainless steel embolization coils (Cook, USA), size 0.035”x5cmx5mm (wire diameter, deployed coil mass diameter, length). Coils were evenly spaced laterally (2 on right side, 2 on left side) and coronally (2 proximal, 2 distal). Due to the size of the coils («1% of total volume), we do not expect that they play a role in the overall registration. However, we recognize that the coils are relatively radio-dense and an analysis where the coils are blocked from image registration could be warranted and may justify the use of different fiducials. d. Computational time: We appreciate this comment and recognize the clinical importance of rapid registration. In the current realization, a single 2D image can be registered to a 3D volume in <1 minute, which is acceptable to our clinical partners.

Results: The reviewers recommended that “the results should be interpreted much more in depth.” Specifically: a. Anatomical feature masks: Our results suggest that the spine contributes the most to image registration. We hypothesize that the rigidity of the spine during respiration and the high radiodensity make it the most relevant feature. In contrast, the bronchial tree is relatively opaque, hard to differentiate from surrounding features, and moves with respiratory and cardiac motion. b. Obliquity of image: We found that the lowest error occurs when the 2D image is taken at an oblique angle. This result was interesting and may be due to the unobstructed view of the spine (obstructed in the frontal plane by ribs/heart/lungs). We recognize that a swine model may not be representative as human arms are typically positioned at the side of the body. Further studies on humans could validate this finding.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This AC was satisfied with the rebuttal addressed many main questions from the reviewers.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

3

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This paper, titled “Determination of error in 3D CT to 2D fluoroscopy image registration for endobronchial guidance” received 3 polarizing reviews. It should be noted that the reviewers with clinical background is in favour of this submission:

R1 strength - strong motivation - validation weakness - missing detail - lack of novelty in the registration method - shallow interpretation R2 strength - clinical relevance weakness - novelty R3 strength - methodology - validation weakness - small dataset leading to questionable clinical significance

All agreed this submission is highly relevant to the MICCAI community. It is unfortunate the the meta-review from the primary AC was somewhat brief, leave little room for authors to provide a focused rebuttal. It should be noted that, based on the Reviewer Guidelines (https://miccai2021.org/en/REVIEWER-GUIDELINES.html), the requirement for novelty is not a necessary condition for CAI-based paper. The fact that it was rated highly by clinician suggested that this manuscript is highly relevant/practical; on the other hand, the fact it was rated poorly by engineers suggested that the clarity/quality of the writing is poor. As the Primary AC noted, quite a few technical details were missing in the original submission, but authors provided sufficient rebuttal in this regard. The true strength of this paper is the comprehensive validation on live animal model.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

7

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

Projecting CT and registering with XR (2D-3D) registration has been studied extensively and the contribution of the paper is incremental. Especially those with fiducial markers, 2D projection simulation, and rigid registration.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Reject
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

10

back to top

Determination of error in 3D CT to 2D fluoroscopy image registration for endobronchial guidance