Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Nazim Haouchine, Parikshit Juvekar, Xin Xiong, Jie Luo, Tina Kapur, Rose Du, Alexandra Golby, Sarah Frisken

Abstract

Digital Subtraction Angiography provides high resolution image sequences of blood flow through arteries and veins and is the gold standard for visualizing cerebrovascular anatomy for neurovascular interventions. However, acquisition frame rates are typically limited to 1-3 fps to reduce radiation exposure, and thus DSA sequences often suffer from stroboscopic effects. We present the first approach that permits generating high frame rate DSA sequences from low frame rate acquisitions eliminating these artifacts without increasing the patient’s exposure to radiation. Our approach synthesizes new intermediate frames using a phase-aware Convolutional Neural Network. This network accounts for the non-linear blood flow progression due to vessel geometry and initial velocity of the contrast agent. Our approach outperforms existing methods and was tested on several low frame rate DSA sequences of the human brain resulting in sequences of up to 17 fps with smooth and continuous contrast flow, free of flickering artifacts.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87231-1_17

SharedIt: https://rdcu.be/cyhUI

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The author introduces a deep-learning-based method to increase the fps of DSA, with phase decomposition and vessel-weighted reconstruction loss as innovative components.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The clinical application will be benefit from this technique. The results are promising and paper is well-organized.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

I don’t have major complains of this paper.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Good
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

I understand that limited by the pages, the author can’t present all the technical details, but if necessary, I would like to see some examples from the entropy maps, which supposes to show some small vessels while the binary mask doesn’t.

Another minor issue is in the Table 1, the decimal numbers should have the same precision across the row.
Please state your overall opinion of the paper

strong accept (9)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This work is novely solving a very challenging yet useful problem of DSA. The methods are well-crafted and fine-tuned for this specific task, the results are reasonably beautiful and the experiments settings are clear and convincing.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

3
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

The paper proposes a DSA (Digital Subtraction Angiography) sequence interpolation method. Different from prior works, this paper preserves blood coherence by first decomposing the sequences into different phases and then train a deep learning model with as input the original image and estimated volume.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The proposed method is novel and interesting in that it decomposes the vessels into different phases before feeding into the network.
- The experiments and ablation studies are adequate.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The results in Fig 5 are iteratively interpolated. Why is only one intermediate image shown instead of multiple images?
- I really appreciate it if the code will be released, which will be very helpful for relevant future research.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Code is not released.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

One minor typo: quotation mark in the third line in Page 2.
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Despite the potential weaknesses, the idea is interesting to me. Waiting for any feedbacks.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

3
Reviewer confidence

Confident but not absolutely certain

Review #3

Please describe the contribution of the paper

The authors proposed a learning-based method to generate high frame rate DSA sequences from acquisitions of low frame rate to eliminate flickering artifacts. The method first estimates the contrast agent volume for each image by decomposing the DSA sequence into three perfusion phases, which is used as an input later to control the contribution of input images. The main part of the method adopts a U-net that takes a pair of frames and the contrast volume as the input to generate the intermediate frame of the input frames. The network was trained with an entropy map generated by another network, in order to make the learning focus on regions of rich vessel information. The method allows generating high frame rate sequences by repeatedly taking the intermediate frames and original images as input. In the experiment, the method was shown to outperform baselines without the phase information or the entropy map, and has better and consistent performance than the state-of-the-art approaches.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper focuses on a new application, generating high frame rate DSA sequences from those of low frame rate without increasing radiation dose. This contribution potentially helps the clinicians to better interpret the dynamics of cerebrovascular blood flow.
- The authors have shown in the paper and in the supplemental video that the proposed method is able to substantially increase the frame rate of DSA from 1-3 fps to up to 17 fps with smooth contrast flow.
- The paper is well written in a clear and informative way. The supplemental video provides a good way to showcase the results.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- In the paper, the phase information is injected into the model (g) by adding a scalar number, the contrast volume v(i,j) of the input images I(i) and I(j), to the bottom level of the network, which is used to control the contribution of the two input images to the intermediate image. Although the results in Table 1 show that this operation improves the performance, there’s no convincing explanation in the paper to the following two points: – How does one scalar number (the contrast volume) help to better balance the contribution of two input images? How does the change of this scalar value influence the change of the contribution of the inputs? – The contrast volume is added at the bottom level of the network, whereas the information of the inputs has been merged in the earlier levels of the model. How does the contrast volume influence the contribution of the two input images to the final output?
- Although the task of this paper is interpolating DSA frames, the training of the model (g) relies on another model (e) to generate entropy maps, which requires per-pixel binary labels. This seems a substantial mount of annotation works compared to other related works mentioned in the paper. How many images need to be annotated to train the model e in order to generate reasonable entropy maps?
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors did not make the code or the dataset they used publicly available, but the description of the method and the experiments including parameter setting is relatively clear. It should be possible to reimplement the method and run the experiments on own datasets. One remark is that, the paper does not indicate the training/validation/testing split of the dataset, nor does the reproducibility checklist. Is there a particular reason? How many sequences and images were used for training and evaluation?
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- In section 2.1, it would be good to make the following points more clear – An image-based binary thresholding was applied on the outcomes of ICA. What value was set to the threshold? How was the value determined? – How exactly was the TDCs computed? Based on the intensity of the current image? How to compute the contrast volume using TDCs?
- In section 2.2, equation (1), for the perceptual loss, the operation phi is defined in the text (page 5) as the conv4_3 feature of an ImageNet pretrained VGG16 model. Would the features trained on ImageNet work for DSA images? It would be good to have an ablation study with and without the perceptual loss to see the effect.
- In section 2.3, the network g is based on the U-net architecture, but it has 6 hierarchies in the encoder and 5 hierarchies in the decoder. The numbers of hierarchies in the encoder and decoder are not balanced. Is there a mistake in the text? It would also be good to have a figure to illustrate the network architecture in the paper or in the supplemental material.
- In section 2.4, equation (2) has a e network on the right hand side. Does the inference operation of network g really need the entropy map? Please also check the notation carefully, especially the subscripts of I and theta, if there’s any mistake.
- In section 3, table 1, have the authors also checked the statistical significance for the methods? The differences between the last two methods are small. It would also be good to have a qualitative evaluation to show some example images for these methods in the paper or in the supplemental material.
- In section 3, the comparison with the state-of-the-art (SOTA) methods was evaluated on only 4 sequences. How many sequences were for evaluation in total? How were the 4 sequences selected? Additionally, how were the SOTA methods trained? What (hyper-)parameters did the authors used. It could be nice to put the details in the supplemental material for reproducibility.
- In Figure 4, the visual difference for the methods Our, OF and Lin seems not so clear. It would be good to point the difference more explicitly in the images.
- In the conclusion, the authors claim that “Our solution is applicable to different organs and procedures … there is no actual technical limitation for the use of our method in … any treatment of arterial and venous occlusions.” I find this sentence a bit over-claimed. The proposed method might be promising for other organs or procedures, but needs to be verified with evidences. For example, for cardiac interventions, in addition to contrast flow, there are respiratory and cardiac motion presented in the sequences, which adds more complexity. There’s no guarantee that the proposed method would work without any further adaption.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper presents a very interesting application on frame interpolation to generate high frame rate DSA sequences. The paper is in general well written and the results are nicely presented. Although I have a little doubt on using the phase information part in the method, overall, I lean to accept the paper. If the authors could nicely address my comments, I would recommend for an oral presentation.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

This is a clear, well structured, valuable and intermediate-to-high impact manuscript. It has been very positively assess by all reviewers, where the strongest points are in the novelty, the clinical application/clinical value and clarity of the text. Some comments and suggestions are provided by the reviewers that might be considered by the authors to improve the manuscript. The adaptation of the specific technique to the particular problem of intermediate frame estimation is very good and well described.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

1

Author Feedback

We are glad that reviewers found our work “novel solving a very challenging yet useful problem” and the method “well-crafted and fine-tuned” with “clear and convincing” results and that “the experiments and ablation studies are adequate”. The reviewers found that the paper is “well written in a clear and informative way”.

Reviewers gave very constructive comments. We will make the necessary correction and add the needed clarifications to the final version of the paper following:

We will address all typos in the paper

We will try to include figures of the entropy maps, if we have enough space

Fig 5 shows interpolation on various phases of the sequence (arterial, capillary and venous), we will include another row to show the iterative interpolation like in the video

The scalar number injected at the bottom of the autoencoder helps in adapting the interpolation w.r.t the phase. Indeed, during the Arterial phase the contrast is progressing very fast, it “jumps” from a frame to another and a large part of the vascular structure appears progressively whereas during the Capillary phase it’s almost linear and static. The last phase, the venous one, is the inverse of the Arterial phase where the contrast washes out, however the vascular tree is already visible because of traces left from the previous steps. The contrast volume can be seen as a way to adjust the “speed” of interpolation.

Because the scalar is added at the bottom, pointwise, as a 2D features array where features are the most dense and spatial dimensions are the least, it has a similar contribution as considering the scalar as an image and stacking it with the inputs, however computationally more expensive. Using this strategy, it is easier to “visualize” that the volume is propagated over each pixel, thus adding a linear contribution.

Indeed, the per-pixel binary segmentation needed to manually label DSA images. We used a patch-based approach to increase the dataset similar to Meng et al. 2019. We will add this reference and clarify.

Our dataset is composed of 32 DSA sequences for a total of 3216 DSA images. This is stated in the paper. The split is 75/25. We will add this information.

The TDCs are computed following Hong et al. 2018. We will add this reference and clarify.

The perceptual loss helps in removing some blur and make interpolated images sharper. Since Beta=0.01 it’s impact on the loss is relatively small.

We will add a figure of the architecture (as additional material) to describe the autoencoder and the pointwise plug of the scalar.

Yes, network g() does need the network e() to generate the entropy because it will be used in the optimization loss. However we agree that it’s more of an implementation detail than a formalization and we understand the confusion. Both formalizations are correct in our opinion.

The differences in Table 1 are indeed small, we provided qualitative evaluation to visually appreciate the difference. We will add more examples as additional material.

We used 4 sequences for evaluation against SOTA methods. We used the default parameters of the SOTA methods without retraining them since they are designed to work with slow motion video. The sequences

In the conclusion we wanted to point out that this work could work beyond neurovascular images. We are aware that adapting our method to organs with non-rigid motion (like the heart) will need some workaround. We will address this point in the final version.

back to top

Estimation of High Frame Rate Digital Subtraction Angiography Sequences at Low Radiation Dose