Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Andrei Svecic, Gilles Soulez, Frederic Monet, Raman Kashyap, Samuel Kadoury

Abstract

Percutaneous transluminal angioplasty (PTA) revascularization is a common minimally invasive treatment for occlusions in peripheral arteries, but it’s success in long occlusions is limited by technical challenges associated with crossing occluded vessels and lumen re-entry. Revascularization needs to be guided closely using ionizing imaging such as fluoroscopy, while intravascular guidewires lack the capability of characterizing physiological conditions near occlusions, such as blood flow. We propose a multimodal sensing framework to infer both three-dimensional shape and vascular flow from an optical fiber device using random optical gratings enhanced with ultraviolet exposure, allowing a fully-distributed strain sensor. A two-branch spatio-temporal neural network is proposed to process a generated optical signal trajectory from scattered wavelength distributions. A shape network is first used in combination with the pre-procedural 3D angiography image to track the 3D shape related to backscattered wavelength shift, while a flow velocity network trained on 4D-MRI measurements allows to extract vascular flow. A final refinement is performed to adjust the 3D-2D projection onto C-arm images, allowing to correct for slight deviations of the sensed shape. Synthetic and porcine experiments were performed in a controlled environment setting, enabling to measure the accuracy of the 3D shape tracking and flow measurements, with errors of 2.4+/-0.9mm and flow differences below 2cm/s, demonstrating the ability to provide anatomical and physiological properties during vascular procedures.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87202-1_24

SharedIt: https://rdcu.be/cyhQm

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The authors propose a technique for joint measurement of the 3D pose of a catheter within a vasculature, and of the blood flow within said vasculature. These quantities are measured in a continuous fashion along the instrument, making use of noise-based Bragg gratings. The measured back-scattered wavelength data is fed to a data-driven method, together with a 3D segmentation of the target vasculature from a preoperative image. The neural network is able to reconstruct the shape of the instrument, given the raw data and the constraint of the segmented anatomy. This knowledge is in turn used to estimate the blood flow within the same data-driven approach. The results are finally refined based on intra-operative 2D angiographic images.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The proposed method addresses an important clinical need, i.e. measurement of blood flow for guiding percutaneous transluminal angioplasty.

The approach allows to conduct this measurement continuously along the instrument, rather than point-based as when using fixed Bragg gratings at a few locations. The continuous distribution of the gratings allows to perform two tasks jointly, i.e. localization and physiological measurements, through a data-driven method.

The validation is rigorous and makes the case for the clinical applicability of the method.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

It is not clear to me if these noise-based gratings can be reproducibly created, or if each instance of the catheter will require individual calibration and training of the network. This may limit the applicability of the technique, or make it very expensive.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Sufficient - replicating the method requires a complicated hardware setup, but the authors provided the necessary information
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

In section 2.3, the “alpha” terms are never introduced. I guess they are learned parameters like “beta” later on. It would be nice to introduce the term explicitely.
Please state your overall opinion of the paper

strong accept (9)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This work provides an innovative and elegant solution to a strong clinical need.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Review #2

Please describe the contribution of the paper

This paper relies on a catheter equipped with 3 optical fibers with randomly placed Bragg gratings. It describes a method to both retrieve the catheter shape and surrounding flow measurements. The method is data-driven, vessel centerlines extracted from intra-operative X-ray imaging as projective shape constraints, and a segmented MRA volume both as 3D shape constraints (vessel centerlines), and for flow computation. 3 CNN are used: the first (ShapeNet) to recover the catheter shape, a second one (FlowNet) for the flow, which also takes the previous shape as input, and the shape is refined through RefineNet. Data are acquired on 18 porcine models to train the models, using 4D MRI as reference for blood flow measurements. Quantitative experiments are provided on phantom data, and also on 5 porcine models that compare the proposed approach with EM tracking and reconstruction with standard fibers. The proposed approach is shown to improve over both other methods, even though the accur
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

This is an impressive work with both hardware and algorithmic contributions. The evaluation involves both phantom and in vivo (porcine) data and compares with to competing solutions. This is a smart exploitation of works about event cameras to deal with asynchronous data from the optical fiber gratings.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Many different subjects are addressed in the paper and this is detrimental to its clarity. I have many questions and doubts about the method that may only stem from missing details. It is difficult to assess all steps of the proposed method.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The hardware part cannot be reproduced easily. Details are insufficient in the paper to confidently reproduce the algorithm section. The reproducibility checklist mentions that a link will be provided (hopefully in the final version if the paper is accepted).
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
Clarity: The text is well organized and clear. A lot of different things and topics are exposed through the paper, however which makes it difficult to read and understand the details. [19] is for example a prerequisite, and I am not convinced that the proposed transposition is as straightforward as the authors write.

Method: This paper relies on a catheter equipped with 3 optical fibers with randomly placed Bragg gratings. It describes a method to both retrieve the catheter shape and surrounding flow measurements. The method is data-driven, vessel centerlines extracted from intra-operative X-ray imaging as projective shape constraints, and a segmented MRA volume both as 3D shape constraints (vessel centerlines), and for flow computation. 3 CNN are used: the first (ShapeNet) to recover the catheter shape, a second one (FlowNet) for the flow, which also takes the previous shape as input, and the shape is refined through RefineNet. The method is largely inspired by [19] (pose recovery from event cameras) and uses the same kind of notations.

But many things are unclear: I understand that v_{i,h} is the position of the hth grating. But the notion of section N_H is unclear and so is the difference between P_l and v_{i,h}, with a very different notation for a 3D point. In [19], intensity images are only available for indices k, k+1, etc… C-arm images here play the role of intensity images. But then vessel centerlines are only available for indices i=0 and i=N and equation (2) is unclear: how p_{j,h} is determined for all j? I would assume the sum in equation (2) can only be computed with i=0 and i=N. On the contrary, I would understand why equation (3) would sum up over all i in [0,N], since you can ask for the shape to remain within the segmented MRA vessels at all intermediate time, but here it is only done for i=0 and i=N. Concerning this equation, I also ponder what the role of t’ might be. I understand in [19] it is used to correct for the motion of the character in the images, but its use is not so obvious in the paper. How is P_{f,l}^{3D} determined would also deserve some details (closest voxel in segmented MRA?). The \Psi function in FlowNet is undefined.

The refinement network seems to use the same L_cor and L_temp as in ShapeNet, and its addition is thereafter unclear. How does the result of FlowNet influence the input of RefineNet? I cannot see any impact. Then why adjust a different shape from the one produced by ShapeNet? Could it be that RefineNet generates a shape that better reprojects onto the X-ray images? Then why not use this shape as input to FlowNet? How do the authors explain the difference between these two shapes? Furthermore, in the first lines of page 6, it is unclear whether vessel borders or centerlines are used as constraints. Setting 2D-3D shape correspondence purely on a shortest distance criterion has also been long known as suboptimal in 2D-3D vessel registration work and might result in outliers.

Some details are provided for the various CNN used in the paper, but more details would be welcome, on the architecture as well as training (reference 3D shapes?). I assume that RefineNet uses the same network as ShapeNet?

Data are acquired on 18 porcine models to train the CNNs, using 4D MRI as reference for blood flow measurements. Quantitative experiments are provided on phantom data, and also on 5 porcine models that compare the proposed approach with EM tracking and reconstruction with standard fibers. The proposed approach is shown to improve over both other methods, even though the accuracy still is around 2.5 mm in 3D. The shape information is demonstrated to improve blood flow measurements.

A remaining question I have is about the capacity of the system to separate blood flow measurements from catheter motion when the interventionalist manipulates it. Or does this system requires that the catheter navigation is paused for the blood flow measurements to be accurate?

Overall, I would suggest that the authors try to separate this work into two publications, or expand it into a journal paper. The MICCAI conference format appears too tight for the method to be exposed and assessed with confidence. This is the reason why I consider this paper as borderline.

Typos:
- p. 5, two lines below equation 2: “the the inferred 3D point”
- please check for acronyms that need to be capitalized in the reference section (e.g. mri -> MRI, Uv -> UV, Cnn->CNN, etc…)
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The reported work is impressive and the evaluation sufficient for a conference paper. However, I feel that this work would benefit from an expansion either in two separate publications or a long journal paper. As such the text is too concise and details are missing so that it is difficult to assess its validity. That is why I consider this paper borderline.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

5
Reviewer confidence

Somewhat confident

Review #3

Please describe the contribution of the paper

The authors propose to work on a device able to track along the vessel and to measure flow. The paper is grounded the physical model of light scattering in optical fiber. The obtained signal is analyzed with neural networks.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

This paper describes a very complete set of experimentation with very encouraging results
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The main weakness is to convince that the available signal is sufficient to retrieve both the 3D shape and the flow. Some expertise in optical fiber and light back-scattering is required.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Limited. This paper relies on a complex experimental setup
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

The work is interesting. You propose the analysis of the optical fiber signal with neural networks. It is very challenging to assess the pertinence oft this approach without a bit more knowledge of the physics of the set up. It is not intuitive that the information can be recovered. It is intuitive to understand that any curvature of the guidewire can be noticed. It is much less intuitive to understand how it is oriented in space.
Please state your overall opinion of the paper

out of scope (1)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper content looks good and it reports a large set of valuable experimental work. However, I have a hard time to truly assess its content and I’m afraid that most of its content will be difficult for the average MICCAI readers. Indeed, I consider that this work can only be understood by a person familiar with optical fibers properties and the backscatter signal.
What is the ranking of this paper in your review stack?

5
Number of papers in your stack

5
Reviewer confidence

Somewhat confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper proposes the first use of a multimodal optical sensing device integrated with a catheter for arterial intervention, enabling inference of both 3D shape and vascular flow simultaneously in real-time using random optical gratings enhanced with ultraviolet exposure. The framework is validated in both phantom and in-vivo porcine experiments, achieving promising accuracies in 3D shape tracking and blood flow measurements. The topic is of clinical interest; the paper is well written and presents substantial contribution, both in terms of hardware and algorithms.

The main concern regarding the work is regarding the large scope of the paper, covering different topics and components, and as a result missing some necessary details in the methodology. The following points should be addressed in the rebuttal:
- Clarifications regarding the scope of the paper, and what modifications are reasonable within the scope of the rebuttal to improve on scope concerns highlighted by reviewer 2.
- Clarifications regarding missing details in methodology, including details of the shape and flow networks and refinement network
- Better Justification for choice of methodology, and its sufficiency for retrieving both catheter shape and surrounding flow, and associated limitations (e.g. capacity to separate blood flow from other catheter motion)
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

3

Author Feedback

We thank the reviewers and AC for their valuable comments. 1) Scope of paper by R2 and R3:

We agree that the combination of optics and imaging aspects may seem as too much for a MICCAI paper. However, it should be emphasized the background and more details on sensor fabrication and optics processing can be found in the journal paper [11; Monet et al. 19], which also presents the hardware setup. We will emphasize this in sections 2.1 and 2.2. The focus of this paper is primarily on the neural networks, and believe it to be suitable for MICCAI. We feel this topic can be of great value to the MICCAI community, particularly for the CAI field, as it can be made aware of new fiber optical sensors which are developed for interventional usage, and how these can be integrated in an image-guided therapy workflow. A background on optics in not required to understand the proposed framework. If space is required to add details on the network methodology, details of the sensor fabrication will be removed from 2.1, as these are in [11; Monet et al. 19].

2) Methods details:

N_H is meant to indicate the number of gratings (fiber sections) along the optical fiber triplet. While the sensor gratings are continuous, this discretization is needed to evaluate the loss functions in a discrete manner.

R2 also notes confusion in notation for the 3D points. The term P_l represents a 3D point from the reconstructed sensing data, while \textbf{P} represent a 3D point from the reference segmented MRA.

The difference between v_I,h and P_l is that the first represent the projected 3D point onto the 2D images, while the second is the 3D coordinate taken from the reconstructed shape.

We apologize for the confusion about the indices used in equations 2 and 3, we believe this was caused by a mistake in the notation [0,N], which may lead to believe only indices 0 and N are considered. In fact, we consider all frames between 0 and N, not just i=0 and i=N. Hence, we will change the formulation to [0,…,N] instead.

The symbol /Psi represents the temporal convolutional network model used previously for motion-based analysis and predictions made from time-varying features. This will be clarified in the revised paper.

t’ can be considered as a translation vector to align the 3D coordinates of the MRA with the sensing system. It allows to evaluate shapes in the same coordinate system.

P_{f,l}^{3D} represents the 3D point coordinate of the 3D centerline from the MRA, corresponding to the frame f and gratting l.

R2 is correct, RefineNet’s backbone is the same is ShapeNet.

While similar to ShapeNet, RefineNet uses instead the losses L_stab and L_proj, which are different as they use different inputs, compared to L_cor and L_temp.

Reviewer 2 is correct, we could have used the output of RefineNet as input to FlowNet. However based on experiments, it was shown that the adjustments brought by RefineNet did not bring any significant gain to the inferred flow values. We also observed the joint training between shape and flow networks was beneficial for convergence.

Vessel borders are used as constraints for the refinement, not the centerline. While we agree with R2 that the shortest distance criterion has been long known as suboptimal for 2D-3D adjustments, the added stability term in the refinement network allows to maintain the overall 3D shape, thus avoid outliers which may generated only from 2D adjustments.

The 3D shape reference in the MRA was not obtained using a neural network, but rather with a deformable surface mesh model described in [1; Badoual et al. 16]. The paper includes all parameter settings. We will clarify this in the manuscript.

3) System ability to separate blood flow measurements from catheter motion:

Reviewer 2 is correct, navigation is paused when performing blood flow measurements. This allows to disentangle the wavelength shift features related to shape and flow measures. This will be clarified in the revised paper.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The paper tackles a relevant clinical need, and demonstrates both hardware and algorithmic contributions, rigorous validation, and encouraging results. Main concerns regarding the scope of the paper have been addressed by the rebuttal, and details regarding methodology, networks and ability to separate blood flow from other catheter motion have been further clarified by the authors.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

4

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This AC did not disagree that this submission was still confusing and some technical details missed.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Reject
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

14

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The Authors provide substantial clarifications and proper responses to the comments brought up by the reviewers and AC. My understanding is that the methodology proposed is valuable and brings together existing tools into the solution to a problem no properly solved yet. The authors reply clearly and politely to the comments provided. The manuscript should be accepted.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

6

back to top

Multimodal Sensing Guidewire for C-arm Navigation with Random UV Enhanced Optical Sensors using Spatio-temporal Networks