Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Mohammed S.M. Elbaz, Chris Malaisrie, Patrick McCarthy, Michael Markl

Abstract

4D Flow MRI has emerged as a new imaging technique for assessing 3D flow dynamics in the heart and great arteries (e.g., Aorta) in vivo. 4D Flow MRI provides in vivo voxel-wise mapping of 3D time-resolved three-directional velocity vector-field information. However, current techniques underutilize such comprehensive vector-field information by reducing it to aggregate or derivative scalar-field. Here we propose a new data-driven stochastic methodological approach to derive the unique 4D vector-field signature of the 3D flow dynamics. Our technique is based on stochastically encoding the profile of the underlying pair-wise vector-field associations comprising the entire 3D flow-field dynamics. The proposed technique consists of two stages: 1) The 4D Flow vector-field signature profile is constructed by stochastically encoding the probability density function of the co-associations of millions of pair-wise vectors over the entire 4D Flow MRI domain. 2) The Hemodynamic Signature Index (HSI) is computed as a measure of the degree of alteration in the 4D Flow signature between patients. The proposed technique was extensively evaluated in three in vivo 4D Flow MRI datasets of 106 scans, including 34 healthy controls, 57 bicuspid aortic valve (BAV) patients and 15 Rescan subjects. Results demonstrate our technique’s excellent robustness, reproducibility, and ability to quantify distinct signatures in BAV patients.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87240-3_21

SharedIt: https://rdcu.be/cyl5U

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The paper presents a novel method to characterise blood flow, as measured using 4D Flow MRI, introducing a new Hemodynamic Signature Index (HSI). The method is based on stochastic sampling of voxel pairs. Results show robustness to segmentation method and a significant difference in HSI value between control subjects and a group of patients with bicuspid aortic valve disease.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The method is novel, differing from most previous literature in that a conversion of the 3D vector field into a scalar field is not necessary. A good evaluation of robustness and efficiency has been performed, showing that the method can cope with segmentation errors. The paper is generally well written, with a good amount of mathematical detail. Figures are used appropriately.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

A number of methods have been developed in the area. The authors justify their development of a new method by stating that other methods are limited in that previous ones mostly transform a vector field into a scalar one, potentially losing information in the process. In my opinion, this is not a valid argument: any method that distils a vector field into a clinically useful metric is necessarily compressing the information in the original images. This is the case of this paper too, by calculating a single Index from the original datasets. Previous papers do this by trying to calculate biomechanically relevant parameters such as vorticity. In contrast, the method presented in this paper has no biological justification, or at least I have not obvious from the paper. This lack of biological grounding would be surmountable if the method was shown to have a clear clinical use. While the paper suggests the new index might be able to differentiate two patient groups, the clinical use is not clearly explained or demonstrated. There is also no validation that shows the new method being more accurate than already existing metrics.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

It would be hard for a different group to reproduce the work without access to the original code.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

I believe the work could be justified in one or two ways: either by showing that it is quantifying biophysically relevant flow properties, or by proving that it is clinically useful and superior to other, more easily explainable metrics.
Please state your overall opinion of the paper

probably reject (4)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The main factors are those described in the “weaknesses” section. In summary, there is no clear explanation on how this method is s contribution to the field, either for getting a better understanding of flow patterns or for clustering patients into groups in a way that is not possible with existing methods.
What is the ranking of this paper in your review stack?

3
Number of papers in your stack

5
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

This paper propose a novel way to process and analyse blood flow from 4D flow MRIs. The authors apply this new technique to bicuspid aortic valve detection from 4D Flow MRI data by proposing a complete pipeline, from segmentation to analysis.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The authors give a novel way to analyse the velocity field provided from 4D flow MRIs. Precisely, they propose to part from fluid dynamics operators and rather quantify to focus on pairwise angular dissimilarity between a selected subset of vectors within a patient’s flow vector field. To give a compact information they propose to have a patient signature represented as the estimated probability density function from the angular distances enabling a fast comparison between patients thanks to the earth mover’s distance. This distance is proposed as the new Hemodynamic Signature Index. The authors, additionally, propose to estimate the dimension of the subset of vectors from the segmentation as a proportion of total voxels in a segmentation. To ensure reproducibility of results, the authors show the methods robustness by quantifying the results variability in a Scan-Rescan scenario on part of the cohort (15 healthy subjects) as well as by simulating segmentation errors. Results are promising as they show robustness.
They show the use of the Hemodynamic Signature Index by comparing its value in a control cohort versus a bicuspid aortic valve patients cohort. Finally the computation and comparison of the novel index has linear complexity making it possible to use the Index in a real-time scenario.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

While the authors show that the method has low-complexity, they restrict the data to 9 timepoints taken at systole. It seems extending the technique to more time points and covering the whole heart cycle would constitute a better application. In addition to low-complexity, choosing to work on a random subset of the voxels further highlights the feasability, should the data be available. The choice of [1,3] voxel size erosion/dilation to look at variation in segmentation seems arbitrary, therefore it is hard to grasp the robustness to segmentation errors. A visualisation of the different masks or a precise quantification of variations such as a contour matching score would be more adequate. While the method and pipeline is thoroughly explained, the application to bicuspid aortic valve detection lacks a thorough analysis and comparison with other operators mentioned in the paper (i.e wall shear stress, kinetic energy, vorticity/vortex, energy loss).
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Every step from processing the 4D flow to extracting and analysing the proposed technique are clear and seem easily reproducible. The exact reproduction of the experiments needs access to the clinical data used in the paper however it can be done on any other 4D flow MRI dataset. There a two points which are very specific to this paper experiments: first the selected 9 time points at systole, more points on the complete heart cycle would not make this such an experiment specific variable. The second is the computation of the 𝛼 term, this element is optimal only for the present dataset and while it should not greatly slow down the process, it is still an important element to compute. The time taken for the convergence analysis should be precised.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

The new Hemodynamic Signature Index proposed is novel and well explained and the pipeline to extract and analyse blood flow is complete. Here are the points I would consider to improve other elements of the paper: There is a contradiction on the 𝛼 term, equation (3) suggests its range is [0,1], as if 𝛼 =1 then all voxels are in the subset. However when computing the optimal 𝛼 the initialisation is at 1 and the optimal 𝛼 at 30. Perhaps it is expressed in pecentage but I don’t see it anywhere. Additionally, it would be interesting to know the amount of time it took to compute, as well as the number of voxels in an average segmentation. As you show that the experiment is fast to compute, if the data is available, it would have more impact to see the analysis on a complete heart cycle with more time-points. You evoque existing indices in the introduction (wall shear stress, kinetic energy, vorticity/vortex, energy loss), can you compare your results with some of these indices? The variation in segmentations you provide to test robustness on segmentation errors is difficult to visualise. Can you provide a visualisation or contour matching score to better grasp the amount of variability between the segmentations ? Finally, on a cosmetic note : The first figure can be enlarged as we don’t see much of the information (especially for sub-figures 3 and 4). In section 2.1, 2) [t0,T] the zero is misplaced. In the equation (1) if X is a 1D vector there are missing commas (before and after elements). In equation (4),(6),(7),(8), and (9) there are missing commas where you state the range of indices.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

I recommend probably accept as the novel indicator to analyse blood flow is generalisable to many problems. The way it encodes changes in hemodynamics is fast to compute and the application highlights the capacity of such indicator.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Review #3

Please describe the contribution of the paper

The authors propose a stochastic methodology to derive the 4D velocity vector field signature from the 4D flow MRI data in 3D flow dynamics. The results are promising, the idea is interesting and the method has been tested for in vivo MRI datasets, in particular in the case of patients with a bicuspid aortic valve.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Using stochastic encoding of the probability distribution over the 4D velocity field in MRI data to accurately derive the flow three-directional velocity over time.
- The robustness of the approach is tested for several in vivo datasets of 106 4D Flow MRI scans, including those of patients with BAV.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- No validation of the method concerning the evaluation of the wall shear stress of the wall which represents a major limitation of the non-invasive 4D flow MRI technique and represents an important clinical indicator of surgical interventions in several cardiovascular applications.
- The incompressibility of blood flow is an important property (free-divergence). The authors should explain why the derived velocity satisfies this constraint and provide numerical evidence.
- The authors should comment on the effect of spatio-temporal resolution in 4D flow MRI on the accuracy of the results.
- Better depict the sensitivity of the results to segmentation errors and data noise.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Details on the method and the main parameters used are provided. The algorithm is not provided.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Address the major weaknesses of the paper.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The method is interesting and the results are promising.
What is the ranking of this paper in your review stack?

3
Number of papers in your stack

5
Reviewer confidence

Somewhat confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

The authors propose a novel method to extract information from 4D flow and propose that this is used as the new Hemodynamic Signature Index. All reviewers acknowledge the novelty and interest in the method, however authors should address reviewer’s concerns, particularly on the justification from a new method; the clinical applicability; and the impact of spatiotemporal resolution.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

4

Author Feedback

We thank the reviewers for unanimously recognizing the novelty, robustness and efficient real-time processing (3-seconds) of our new comprehensive stochastic 4D Flow signatures approach. We are also delighted that Reviewers 2 and 3 highlighted the wide range of potential applications of our novel technique for comprehensive standardized 4D flow quantification for different organs. We address the main comments and indicate where addressed in the paper. 1) Justification for a new method/ Clinical applicability (Reviewer#1): All the reviewers unanimously acknowledged that our proposed stochastic technique is the only technique that can directly quantify the dynamic three-directional velocity vector field and without any scalar or derivative transformation over the entire spatiotemporal volumetric space of interest (e.g. Aorta). One of the main raised questions is why such direct comprehensive analysis of the native complex 3D vector field data is needed compared to existing derivative methods. While the limitations of current techniques are addressed in the introduction (paragraph 1: p. 2), we reiterate few fundamentals and elaborate here. First, we emphasize that the velocity vector field data is the primary/basis flow field that embeds the full spectrum of measurable fluid dynamics (Fig 1 panel 4). This can be understood by the fluid dynamics definitions of all existing fluid dynamics metrics (e.g., vorticity, wall shear stress, energy loss, etc.), which are secondary parameters computed/derived from the underlying velocity vector field data. Thus, if a flow feature is not embedded in the velocity vector field, by definition, it cannot be captured by a derived flow metric in the first place. Conversely, a single derivative flow metric cannot capture the full flow dynamics spectrum embedded in the vector field dynamics. Clinically, flow changes are often multifaceted with a mixture of multiple interacting complex flow features simultaneously – especially in complex progressive stages where a diagnosis is most critical (paragraph1: p.2). For example, patients with progressive aortic valve stenosis and with aneurysm would present multiple complex composite flow alterations in the aorta simultaneously, e.g., flow jets, vortical flow, helical flow, flow-wall interaction (Ref. 6 for a review). As explained in the introduction, existing flow metrics are dedicated and can be useful in capturing the contribution of mainly a single flow feature (e.g., vorticity would capture vortical flow but not jets and vice versa). We address these critical limitations by our proposed comprehensive stochastic vector-filed signatures approach that quantifies the entire spatiotemporal multifaceted vector field with real-time processing. Experimentally, we extensively studied 106 in vivo 4D Flow MRI scans of real 91 patient data (not simulations). As already elegantly highlighted by Reviewers 2 and 3, our results show clinical feasibility, signifying our technique’s strong diagnostic potential in distinguishing BAV patients from healthy controls (sect. 7,8, 9). 2) Impact of spatiotemporal resolution (Reviewer#3): As follows from above, one of the inherent advantages of our stochastic mathematical design of the signature approach is that it works directly on the native velocity vector field without the need for derivative/gradient computations (sect. 2). Conversely, existing metrics (e.g., vorticity, energy loss, WSS), by definition, require error-prone resolution-dependent velocity gradient computation. Thus, our technique is inherently more robust to spatiotemporal resolutions than existing methods (sect. 5,6,8, 9). 3)On confusing 𝛼 range as [0,1] in Eqn. 3 (Reviewer#2): We want to clarify that 𝛼 range is Not [0,1]. As can be derived from Eqn. 2, 𝛼 range is (0, L-1]. In Eqn. 3, this results in maximal pairwise sampling η= L×(L-1) i.e., the deterministic pdf that we approximate stochastically in real-time processing (convergence in Fig.2; sect.4,8)

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The authors propose a novel method to extract flow information from 4D Flow MRI and to produce a novel biometric from it. In their rebuttal, the authors have focused on addressing the main criticisms, mostly coming from reviewer#1. I believe the rebuttal has clarified the main concerns: clinical applicability and usefulness, comparison to other methods and impact of temporal resolution. As a result I recommend acceptance.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

3

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The proposed method is novel to derive the unique 4D vector-field signature of the 3D flow dynamics. Experiments show the potential clinical usage of the proposed method. The rebuttal clarified the advantage of the method.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

7

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This paper presents a new method to characterize blood flow for the computation of a new index, namely the Hemodynamic Signature Index (HIS). The method, which is based on the earth mover’s distance, is novel as it is different from previous work, and this work is interesting and well-explained in the paper. I echo the concerns about the lack of biological background, and method application, e.g., bicuspid aortic valve detection. In the rebuttal, the concerns raised by the reviewers are satisfactorily addressed.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

3

back to top

Stochastic 4D Flow Vector-Field Signatures: A new approach for comprehensive 4D Flow MRI quantification