Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Hengtao Guo, Xuanang Xu, Sheng Xu, Bradford J. Wood, Pingkun Yan

Abstract

Fusing intra-operative 2D transrectal ultrasound (TRUS) image with pre-operative 3D magnetic resonance (MR) volume to guide prostate biopsy can significantly increase the yield. However, such a multimodal 2D/3D registration problem is very challenging due to several significant obstacles such as dimensional mismatch, large modal appearance difference, and heavy computational load. In this paper, we propose an end-to-end frame-to-volume registration network (FVR-Net), which can efficiently bridge the previous research gaps by aligning a 2D TRUS frame with a 3D TRUS volume without requiring hardware tracking. The proposed FVR-Net utilizes a dual-branch feature extraction module to extract the information from TRUS frame and volume to estimate transformation parameters. To achieve efficient training and inference, we introduce a differentiable 2D slice sampling module which allows gradients backpropagating from an unsupervised image similarity loss for content correspondence learning. Our experiments demonstrate the proposed method’s superior efficiency for real-time interventional guidance with highly competitive registration accuracy.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87202-1_6

SharedIt: https://rdcu.be/cyhPN

Link to the code repository

https://github.com/DIAL-RPI/FVR-Net

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The paper proposes an end-to-end 2D-3D registration network to bridge the gap in real-time TRUS-guided interventions. The accurate registration of real-time 2D TRUS image and preoperative 3D TRUS volume is truly a challenging task.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The application is with clinical significance.
- This method is without hardware tracking.
- The paper is well-written, the structure of this paper is clear and the background and motivation is well-described.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The registration is still rigid registration.
- Ground truth DOF annotations are needed.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The reproducibility of the paper is satisfactory.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- Methodology:
- First paragraph, “We found the rigid registration can best suit in our application without loss of generality”. Does this mean non-rigid registration is unnecessary in this application?
- From the ablation study, it seems the supervised information of DOF annotations is extremely important in this method. How to make sure the DOF information is accurately obtained?
- Experiments:
- How do the baseline methods conduct 2D-3D registration?
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

See comment 3.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

4
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

The authors propose a deep learning method to perform 2D frame-to- 3D volume image registration (FVR-net) of transrectal ultrasound (TRUS) images. The network here estimates rigid transformation parameters by optimizing a similarity metric that combines ground-truth transformation measurements (from EM-tracking fusion device) and image similarity measure (intensity squared differences).
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

• Novel deep learning formulation to register 2D slices to a 3D volume in TRUS imaging. • Comprehensive set of ablation studies quantify and validate method design choices. • Comparison to 5 additional baseline methods is strong. • Large dataset for training (n=488) and testing (n=65). • While results do not significantly differ from the compared, conventional (non-deep learning) registration approachs, the proposed method performs in near real-time (and 80x speed up), which has high translational impact. • The paper is very well written.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

• The method address mono-modal registration of TRUS during the procedure, but does not address the fusion of pre-procedure 3D MRI to the 3D TRUS volume. • Sensitivity to parameter choices, e.g. R, is not presented. • Does not address potential non-rigid deformations during the procedure.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors have described their data resources and experimental setup in good detail. Code is to be shared.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Sec. 3: It is mentioned that rigid registration is optimal for this task. In the future, it would be great to provide quantitative results to support this conclusion. For example, another ablation study that tries to register the images using affine transformations. Non-rigid deformations do occur during the biopsy procedure, so this type of study would be interesting to see. Sec. 3.1: Are the two loss function L_trans and L_sim combined with equal weighting, e.g. L_total = L_trans + L_sim? It would interesting to see if results improved by performing a weighted linear combination.

Sec. 3.2: How sensitive is the method to the choice of R (the neighboring frame range)? This seems like it could be an important parameter choice for the method.

Table 1: It would be interesting to see the variance in the results shown. In particular showing mean +/- standard dev for the DistErr would be helpful to get a sense of worst-case performance.

Grammar/Typo:

Sec. 3.1, p4: “while completely ignore” -> “while completely ignoring”
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This is a well written paper with good experimental setup and validation of a novel 2D to 3D registration method based on deep learning. The authors provide a strong set of ablation experiments to highlight the contributions of various components of their method and provide comparison to conventional (non-deep learning) registration.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

4
Reviewer confidence

Very confident

Review #3

Please describe the contribution of the paper

This paper provides a 2D/3D registration method that can potentially run in a 3D biopsy guidance system. Existing iterative registration algorithms solve an optimization problem on-the-fly which could be slow if the initialization is too far from the solution. Methods such as the proposed algorithm run in O(1) regardless of initialization with comparable accuracy.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The clinical application is of interest to the MICCAI community.
2. The authors provide adequate validation and comparison to existing methods (Powel + NCC).
3. The approach is novel. To the best of my knowledge CNNs have not been used to solve 2D/3D slice-to-volume registration for freehand prostate biopsies.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

I do not think that this paper has a major weakness.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Given that the authors do not mention releasing data or source code in the manuscript, I do not think that the manuscript is reproducible.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
1. I think the authors should cite previous work in iterative registration. The authors have missed the following:
[*] Khallaghi et. al., “A 2D-3D registration framework for freehand TRUS-guided prostate biopsy”, MICCAI 2015.
1. The prostate is an elastic organ and undergoes deformations during the biopsy session. While I understand that the scope of this work is limited to compensating prostate motion, it is unclear how these methods could be extended to non-rigid registration. Iterative registration methods can be trivially extended by changing the transformation from rigid to a parametric deformation model. The authors should discuss this in their future work.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The focus of this paper is in solving prostate motion during a biopsy session and the authors provide adequate validation to show the efficacy of their method. Furthermore, the paper is well written and easy to follow.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

1
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

I agree with all three reviewers that this work interesting with sound approach and practically very important. I have no further comments but recommend early acceptance.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

1

Author Feedback

N/A

back to top

End-to-end Ultrasound Frame to Volume Registration