Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Davood Karimi, Lana Vasung, Fedel Machado-Rivas, Camilo Jaimes, Shadab Khan, Ali Gholipour

Abstract

Recent works have used deep learning for accurate parameter estimation in diffusion-weighted magnetic resonance imaging (DW-MRI). However, no prior study has addressed the fetal brain, mainly because obtaining reliable fetal DW-MRI data with accurate ground truth parameters is very challenging. To overcome this obstacle, we present a novel method that uses both fetal scans as well as high-quality pre-term newborn scans. We use the newborn scans to estimate accurate parameter maps. We then use these parameter maps to generate DW-MRI data that match the measurement scheme and noise distributions that are characteristic of fetal scans. To demonstrate the effectiveness and reliability of the proposed data generation pipeline, we use the generated data to train a convolutional neural network for estimating color fractional anisotropy. We show that the proposed machine learning pipeline is significantly superior to standard estimation methods in terms of accuracy and expert assessment of reconstruction quality. Our proposed methods can be adapted for estimating other diffusion parameters for fetal brain.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87234-2_46

SharedIt: https://rdcu.be/cyl8I

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

Deep learning has been applied to accurate parameter estimation from DW-MRI. However, no prior study has addressed the fetal brain, because the training data can be difficult to acquire. This work presents a method for accurately estimating parameter maps for the fetal brain. Newborn scans are first used to generated accurate parameter maps, and these maps are then used to generate DW-MRI data that matches the fetal scans. Experimental results show that the proposed pipeline is superior to standard estimation methods. The paper addresses a relevant and interesting problem, and there is some novelty in the proposed methodology. The evaluation is also reasonable. Some improvements and clarifications can still be made, though.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The problem being addressed is interesting and significant. How to perform deep learning based parameter estimation for fetal brain is indeed challenging.
- The proposed method has some novelty. It develops a new way to acquire training data.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- There are some typos and formatting issues.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Although data and code are not provided, the method description is clear. I expect the method to be reproducible.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- The authors describe that 600 images were generated, which were used for training, validation, and test. Does this mean that the quantitative evaluation in section 3.1 was based on simulation data? How many synthetic images were associated with one subject? Please clarify.
- Since the conventional method WLLS produces very noisy results, the neuroanatomist can easily judge that the CFA given by the proposed method has better quality. However, in future work, if a different learning-based method is developed and compared with the proposed method, how can we quantitatively and objectively compare their results? Please discuss if possible.
- The references can be combined and ordered. For example, on page 1 “[12] [28], [24]”->”[12,24,28]”. This applies to other references as well.
- There are some typos. For example, “pane” -> “panel” in the caption of Fig. 2; “propsoed” -> “proposed” in Fig. 5.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The proposed work addresses an interesting problem and the method has some novelty.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

6
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

Summary: Authors proposed a technique that uses both newborn and fetal scans. A convolutional neural network is used for estimating the fractional anisotropy. The proposed method outperforms the standard estimation techniques in terms of accuracy and precision.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Strength: the proposed method is compared with the standard techniques and it outperforms in terms of accuracy and precision. Weakness: It is great that the machine learning based approach outperforms the standard estimates, however it needs a set of training data that can be challenging sometimes. And also the quality of the training data will affect the quality of the CFA map. It is not clear to me among diffusion indices such as mean diffusivity, radial and axial diffusivity, the authors have used the CFA. All the parameters of the network should be reported and also the reason for selecting these parameters have t be provided.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Weakness: It is great that the machine learning based approach outperforms the standard estimates, however it needs a set of training data that can be challenging sometimes. And also the quality of the training data will affect the quality of the CFA map. It is not clear to me among diffusion indices such as mean diffusivity, radial and axial diffusivity, the authors have used the CFA. All the parameters of the network should be reported and also the reason for selecting these parameters have to be provided.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

It is reproducible.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Summary: Authors proposed a technique that uses both newborn and fetal scans. A convolutional neural network is used for estimating the fractional anisotropy. The proposed method outperforms the standard estimation techniques in terms of accuracy and precision. Strength: the proposed method is compared with the standard techniques and it outperforms in terms of accuracy and precision. Weakness: It is great that the machine learning based approach outperforms the standard estimates, however it needs a set of training data that can be challenging sometimes. And also the quality of the training data will affect the quality of the CFA map. It is not clear to me among diffusion indices such as mean diffusivity, radial and axial diffusivity, the authors have used the CFA. All the parameters of the network should be reported and also the reason for selecting these parameters have to be provided.
Please state your overall opinion of the paper

borderline reject (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Authors used a cNN to generate the CFA map of the fetal data. The quality of the estimated map depends on the training dataset.
What is the ranking of this paper in your review stack?

3
Number of papers in your stack

3
Reviewer confidence

Somewhat confident

Review #3

Please describe the contribution of the paper

The submission presents a novel diffusion weighted imaging parameter estimation with an application in fetal and newborn MRI. The effectiveness and reliability were tested and with the machine learning algorithm a better estimation can be achieved compared to standard estimation methods.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- A machine learning algorithm was used to perform diffusion estimation
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- A comparison against “classical” least squares estimation was made. This technique is used, but is fairly outdated. With novel acquisition schemes such as HARDI better estimation can be achieved.
- The color fractional anisotropy was reconstructed. It remains unclear why this parameter was used, since the “normal” FA is more widely used. It is believed that the first eigenvector was used to provide a color coding to the FA.
- How does the machine learning model preform against other diffusion estimation models?
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The code has been provided, data not.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

It remains unclear why only WLLS was used to compare against. This model is rather simple and currently more sophisticated techniques are available. In the data processing there are several remarks that need improvement. An anatomical reference, such as a T1 scan, can be used as a stable reference. Correction for eddy currents is missing. A score was used by an expert neuroanatomist. Was this a standardized score? And to what extend is the expert knowledgeable about DWI in this specific sample? In order to have a proper evaluation more scores from experts are required.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

That a machine learning algorithm was used to perform diffusion estimation
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

3
Reviewer confidence

Somewhat confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The authors present a method to estimate important quantitative parameters in fetal diffusion-weighted MRI. Strength: -This paper was well received by all the reviewers. They have highlighted the importance of the problem addresses - reliable quantitative maps can be a real challenge in fetal imaging due to the substantial motion present. -The pipeline presented does not rely on enough fetal data being present, but rather generates/simulates fetal data based on data from neonates, which is a good approach and has been highlighted as a new approach by the reviewers.
- The results have been explicitly mentioned as being very convincing. This is indeed the case, the results are very convincing!
Weakness:
- The major weakness mentioned refers to more transparency to be requested with respect to the other obtained parameters. Why was CFA chosen, whcih is an unusual parameter? Maybe the authors could comment on this.
- Better evaluation against less old methods could be included.
Minor weaknesses:
- Some clarification is required, such as the split of the 600 images, both in terms of training/(validation/testing as well as in terms of how many of these images belong to one fetal/neonatal head. -Some typos have been highlighted which need to be fixed, references to be combined -A question is raised on how to compare the quality obtained with different learning-based methods in the future. This can be added to the discussion.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

2

Author Feedback

The main concern raised by the reviewers is our choice of CFA among all possible DWI parameters. This is a good question. It is true that many different parameters could have been chosen. However, please note that our focus in this paper was to propose a novel and effective method for synthesizing reliable training data for fetal DWI. Nonetheless, we expect that our proposed pipeline should be equally applicable for “any” DWI parameter. We chose CFA only as an example to demonstrate the effectiveness of our proposed methods. CFA depends not only on diffusion anisotropy but also on the orientation of the major fascicle. Therefore, it is more challenging than FA or other scalar parameters that have been mentioned by some of the reviewers. Nonetheless, CFA, per se, was not the focus of our work. Our proposed pipeline should be able to generate reliable training data for other DWI parameters as well.

The other concern was with regard to our choice of competing method, i.e., WLLS. Please note that WLLS is the standard method for diffusion tensor estimation, and hence for CFA estimation. Therefore, the comment by one of the reviewers that this method is “outdated” is not correct. Rather, WLLS is the standard method that is widely used, and it is the default method in common software packages for DWI estimation (such as DIPY and MRTrix). One of the reviewers has mentioned “HARDI methods” as an alternative; we think this is a misunderstanding because HARDI refers to acquisition schemes and not to a parameter estimation method. Using HARDI measurements, one can estimate DWI parameters using methods such as spherical deconvolution. However for diffusion tensor estimation (and hence for CFA that we have considered in this paper), WLLS is the standard/default method.

back to top

Accurate parameter estimation in fetal diffusion-weighted MRI - learning from fetal and newborn data