Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Mengwei Ren, Heejong Kim, Neel Dey, Guido Gerig

Abstract

Current deep learning approaches for diffusion MRI modeling circumvent the need for densely-sampled diffusion-weighted images (DWIs) by directly predicting microstructural indices from sparsely-sampled DWIs. However, they implicitly make unrealistic assumptions of static q-space sampling during training and reconstruction. Further, such approaches can restrict downstream usage of variably sampled DWIs for usages including the estimation of microstructural indices or tractography. We propose a generative adversarial translation framework for high-quality DWI synthesis with arbitrary q-space sampling given commonly acquired structural images (e.g., B0, T1, T2). Our translation network linearly modulates its internal representations conditioned on continuous q-space information, thus removing the need for fixed sampling schemes. Moreover, this approach enables downstream estimation of high-quality microstructural maps from arbitrarily subsampled DWIs, which may be particularly important in cases with sparsely sampled DWIs. Across several recent methodologies, the proposed approach yields improved DWI synthesis accuracy and fidelity with enhanced downstream utility as quantified by the accuracy of scalar microstructure indices estimated from the synthesized images. Code is available at https://github.com/mengweiren/q-space-conditioned-dwi-synthesis.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87234-2_50

SharedIt: https://rdcu.be/cyl8M

Link to the code repository

https://github.com/mengweiren/q-space-conditioned-dwi-synthesis

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The paper presents an image synthesis algorithm for generating diffusion weighted images from standard structural MRI scans (T1w, T2w). The unique aspect is that the diffusion weighting parameters can be arbitrarily specified at test time without retraining the model. In addition the generator can be conditioned on existing diffusion weighted scans enabling it to complete sparse or partial acquisitions.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The paper solves a long-standing limitation of non-model-based diffusion weighted image generation techniques.

Results are convincing.

Paper is well presented.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Only trained on health subjects so far and no test of generalisability to unseen content such as pathology.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Enough information is provided.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

This is a very nice solution to a long-standing limitation. Clearly presented and with convincing results. The weakness I highlight above is common among work on this topic and not a showstopper, but rather an area for future work, as the authors acknowledge.

My only additional comment is a minor presentational issue, which is that the authors say several times that “downstream usages such as fiber tractography and diffusion orientation distribution function (dODF) estimation are precluded with” other methods. I’m unsure what they mean by this and how their approach differs in this aspect - other methods that synthesise diffusion MRI according to a particular protocol can still feed into downstream processing.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Timely work solving a long-standing limitation in a popular topic. Possibly a bit niche for oral presentation, although I guess the general approach could have applicability in other quantitative imaging areas.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

3
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

In brief, the authors propose to utilize conditional generative adversarial networks to generate diffusion gradient volumes in in directions that are not present in the given sparse set once the model has been trained. This will allow for a better fit of the variety of microstructure methods and tractography methods that exists and require densely sampled gradient directions
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Clear well defined methodology for generating and upsampling the sparse set of gradient directions to densely sampled gradient directions.
- Highly useful for clincial DW-MRI acquisitions where only a sparse set of gradient directions are acquired due to scan time constraints.
- The work also shows applicability on quite a few microstructure methods and tractography both
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- It is a bit interesting that only 9 subjects have been used for training and 1 for validation. Given that the model is trained on 2D slices, more data could be beneficial specially that HCP is being used.
- Testing on HCP itself has become a bit idealistic. Evaluation on a secondary dataset will give a better insight as to how well this method holds up
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The experiments have not been repeated
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- Overall, I found this work to be commendable and the application evaluation towards different methods is really useful for the reader
- Found some minor typos such as ‘artefacts’
- Not for this body of work, but evaluation on a secondary validation dataset would be beneficial.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The approach is novel, useful and has been evaluated on a lot of existing methods. The only drawback being that the experiments have not been repeated
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Very confident

Review #3

Please describe the contribution of the paper

Deep learning based approaches have allowed to circumvent the requirement of densely-sampled diffusion-weighted images (DWIs) for the prediction of microstructural indices. Existing methods assume static q-space sampling scheme. This paper proposes a framework that allows arbitrary q-space sampling for deep learning based approaches. The proposed method was validated and compared with several other methods on brain imaging data. I have some major concerns about the significance of the proposed work, as well as the methodology. My detailed comments are given below.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The presentation of the overall framework is clear.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The significance of the proposed work is not well justified.
- The methodology is also questionable.
- The comparison in the experiment is not fair.
The details are given below.
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Public data was used, but some details in the experimental settings are missing. The reproduction of the experiment is difficult.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- For one single DWI dataset, the sampling scheme is generally fixed, and there is no need to address the problem of arbitrary sampling. In the experiment, the dataset with different sampling schemes was actually generated by the authors from a dataset with a fixed sampling scheme. It is not convincing that there is a need for the proposed method, and the significance of the proposed work is not justified.
- The proposed method synthesizes DWIs associated with different b-vectors using b0, T1-weighted, and T2-weighted images. However, b0, T1-weighted, and T2-weighted images are rotationally invariant, where DWIs are sensitive to the directional information of the underlying tissue. The directional information is not encoded in b0, T1-weighted, or T2-weighted images, and how is it possible to synthesize DWIs that encode the directional information? If the directional information purely comes from the training data, then the synthesis for directional information is based on prior information and may not well represent the actual data. The proposed methodology is problematic in this sense. Please clarify.
- How are the b0, T1-weighted, and T2-weighted images aligned before being sent into the network?
- In Fig. 2, does the generated DWI correspond to a b-vector that is included in the training scan or an arbitrary b-vector? The proposed method claims to address the problem of arbitrary q-space sampling. I would expect it to generate DWIs associated with arbitrary b-vectors. Please clarify.
- Is there a principled experiment for hyperparameter selection? If not, the selection is arbitrary and may be further optimized.
- Fig. 3 is a table not a figure.
- Even though q-DL cannot be applied directly with arbitrary q-space sampling, to make a fair comparison, q-space interpolation should be performed, so that q-space measurements corresponding to a fixed sampling scheme can be created for q-DL. This interpolation can be easily achieved with spherical harmonics basis functions. Currently, the comparison with q-DL is not fair.
- Quantitative results for dODF and tractography are missing, and it is difficult to determine the advantage of the proposed method with qualitative results only.
Please state your overall opinion of the paper

strong reject (2)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The significance of the work is not well justified. There are also concerns about the methodology. The experimental setting is not fairly designed.
What is the ranking of this paper in your review stack?

5
Number of papers in your stack

6
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

This manuscript proposes a convincing solution to a long-standing limitation of learning-based techniques for generating diffusion weighted images. I therefore tend to agree with the two reviewers who would like to see this work presented at MICCAI. In the rebuttal, I would hope to get an impression of how the authors would address the points brought up by R3, but also the issues mentioned by R1, in a camera-ready version of their paper.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

3

Author Feedback

We thank the ACs and reviewers for their valuable feedback which will be used to improve the submission. We are happy to see that R1&2 found the work to be well-executed and tackling a long-standing real-world task.

R3 || “For a single DWI dataset, the sampling scheme is fixed. No need to address arbitrary sampling” This is incorrect. While scanning protocols are static, practical DWI processing pipelines universally cause arbitrary sampling. DWI is subject to severe motion corruption & eddy currents whose correction involves the reorientation of gradients and/or their complete exclusion. As subject motion/eddies are impossible to predict, images can be degraded unpredictably and require methods for arbitrary directional restoration.

|| “Directional info absent in b0/T1/T2, how is directional DWI synthesis possible?” We hypothesize that the relationship between DWI and structural MRI conditioned on directional coordinates can be learned from training data, which is empirically validated as we find that the model generalizes to held-out randomly-sampled test subjects. We note that high-dimensional synthesis from easier-to-acquire data is not an unprecedented/unvalidated research direction. Examples in computer vision include image relighting, novel view synthesis, monocular depth estimation, and 3D shape from 2D image. Relevant to DWI, [Anctil, CDMRI20] translates T1 to DTI which is also directional in nature. Our method is more general as it predicts raw DWI instead of fitted-models such as DTI.

|| “Interpolation for qDL should be performed” qDL was developed for fixed gradient sampling which is unrealistic in practice and we included qDL experiments to demonstrate this inapplicability. While interpolation can be used, each subject in a dataset has a variable number of usable gradients. E.g., resampling to 128 gradients from 96 usable gradients from subject A will have lower interpolation error and different signal characteristics than resampling to 128 from 48 gradients from subject B. We therefore do not expect a qDL network to generalize to highly variable signal characteristics, without first validating stability to interpolation quality and substantial modification to the model and training strategies, which constitutes an entirely separate research project.

|| “No quantification on ODF+tractograms” We focus on quantitative benchmarking of scalar maps as they allow for direct comparison with current works. As there is no learning-based work that directly predicts raw DWI (with arbitrary q-space coordinates) which can be used for ODF/tractogram, we only qualitatively show downstream utility for tasks other than scalar maps.

|| “Reproducibility; hyperparameter selection” All experimental details and training scripts will be in the code release. Hyperparameters were tuned on the validation set for model selection prior to testing.

R1 || “Presentation issue: clarify wider downstream usage.” To clarify, current methods only regress predefined scalar maps [Goldkov, TMI16; Gibbons, MRM19] or coefficients of predefined models (eg, DTI) [Tian, NeuroImage20; Lin, MedPhys19]. Those cannot be used for generic DWI pipelines and thus have lower downstream utility. Methods that produce DWI signals (to our knowledge, only [qDL-R, Goldkov MICCAI15]) require fixed input/output sampling, whereas arbitrary sampling arises widely in DWI processing (see R3 response). If predefined sampling is violated (e.g., gradient exclusion due to motion), the method can no longer be used. As the proposed model can synthesize DWI at any specified q-space coordinate, the original gradient directions can be restored for use in any pipeline.

|| “Only demonstrated on healthy subjects, no tests on pathologies” We fully agree that healthy control evaluation does not yield complete insight into generalization to lesions with variable diffusion properties. As suggested, we will test the proposed methods on more diverse datasets in follow-up work.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

I was in favor of this paper before the rebuttal phase and I still am, despite the fact that the authors unfortunately decided to ignore my request to comment on what changes they are planning to make in their camera-ready version. Even though I agree with several points that the authors make in their rebuttal, I would still like to encourage them again to re-consider how they might further improve the presentation of their paper based on the reviewer feedback. In particular, I agree with R1 that authors should remove the claim that their method is the first to permit “downstream usages such as fiber tractography”. Improvements in tractography based on synthesized DWIs have been demonstrated previously, e.g., in [1].
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

2

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper presents an algorithm allowing to generate DWI from standard MRI scans. The diffusion weighting parameters can be arbitrarily specified at test time without retraining the model, also allowing to overcome limitations of conventional acquisitions with limited q-space coverage

The strengths highlighted by mostly 2 of the reviewers are
- the convincing results for an important long standing issue
- the fact that the paper is well written and very clear to follow.
Major weakness identified was the
- limited subject number (only healthy subjects and only 1 test case), the generability was thus not proven.
The rebuttal addresses this point well, confirming the need for follow-up work.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

2

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

I think the authors have done a good job in addressing the reviewers’ questions. I would like to recommend to accept.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

7

back to top

Q-space Conditioned Translation Networks for Directional Synthesis of Diffusion Weighted Images from Multi-modal Structural MRI