Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Yilin Liu, Yong Chen, Pew-Thian Yap

Abstract

Magnetic resonance fingerprinting (MRF) is a relatively new multi-parametric quantitative imaging method that involves a two-step process: (i) reconstructing a series of time frames from highly-undersampled non-Cartesian spiral k-space data and (ii) pattern matching using the time frames to infer tissue properties, e.g., T1 and T2 relaxation times. In this paper, we introduce a novel end-to-end deep learning framework to seamlessly map the tissue properties directly from spiral k-space MRF data, thereby avoiding time-consuming processing such as the non-uniform fast Fourier transform (NUFFT) and the dictionary-based fingerprint matching. Our method directly consumes the non-Cartesian k-space data, performs adaptive density compensation, and predicts multiple tissue property maps in one forward pass. Experiments on both 2D and 3D MRF data demonstrate that comparable quantification accuracy to state-of-the-art methods can be accomplished within 0.5 second, which is 1,100 to 7,700 times faster than the original MRF framework. The proposed method is thus promising for facilitating the evaluation and adoption of MRF in clinical settings.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87231-1_16

SharedIt: https://rdcu.be/cyhUH

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

In this paper, the authors present a new method for real-time mapping of tissue properties for magnetic resonance fingerprinting. They show that the presented method performs on par with current state-of-the-art methods but up to 7000 times faster than current methods.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

1) The presented method allows end-to-end learning from the spiral k-space data to the tissue properties. Compared to other methods they do not need a transformation of the spiral k-space to the Cartesian k-space with NUFFT 2) Learned density compensation. 3) K-means as an alternative for gridding. 4) General Framework to easily change and use different network architectures that are not specifically developed for MRF.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

For the ground truth, no real acquired T1 and T2 maps are used. It is not clear how the separation of the dataset for training and evaluation is performed. On the patient level or the time series level.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

There is no information about the hyperparameters of the other methods used in the paper. The description of the used model is not detailed so implementation would be difficult (…) we extended an existing U-Net (…)
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Missing reference for the Adam optimizer

The definition of the index J in {1, 2, M x M} (Eq. 3) is not clear for me. Is j a continuous index (1-MM) or a point index ((1, 1), (1, 2), .. ((M, M))? If it is a point index I would change it to j in {M x M}.

In the result section (…) with processing speed 24 times faster than a CNN method (…). I assume you also use a CNN in the U-Net implementation. Better directly mentioned the method name. It is not clear how the data set is split into training, validation, and test. Is the splitting performed on the subject level or on the time point level?

Typos: Abstract 0.5 second -> seconds
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The general idea of this paper is very interesting. However, there are some issue with the evaluation of the method.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

6
Reviewer confidence

Confident but not absolutely certain

Review #2

Please describe the contribution of the paper

This paper proposes to lower the burden of the time-intensive inverse non-uniform fast Fourier transformation and dictionary matching in MR fingerprinting by operating in the k-space. Raw non-Cartesian k-space data is transformed via a neural network to the Cartesian k-space data of the parametric maps, and only a inverse fast Fourier transformation is finally applied. The method is evaluated on multiple datasets and compared to several baselines.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The paper is well written and easy to follow. The motivation to further accelerate the reconstruction of MRF makes sense and is clinically valid. The methodology is well described and the illustration are supportive and nicely drawn. The results look solid and includes a set of baselines as well an ablation of the gridding process.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- No variability of the results (standard deviations in Table 1 & 2)
- Short discussion. E.g., why is T1 worse than T2 (cf. Table 1) or why is the performance on the 3D dataset better than 2D dataset. Maybe one figure with qualitative results could be moved into the supplementary material to have more space for some discussion?
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The method is not reproducible as no public code seems to be provided (no (blinded) link or similar in paper). But the experimental setting is good described.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- Abstract “i.e., T1 and T2 relaxation times”: there are far more MR parameters that can be quantified. Consider replacing i.e. with e.g.
- Related work on deep learning MRF: consider at least citing the journal articles working on these topic: Song et al., Medical Physics, 2019. Balsiger et al., Medical Image Analysis, 2020. Golbabaee et al., Medical Image Analysis, 2021.
- Statement “employs a spiral k-space trajectory”: not only spiral, see review by McGivney et al., JMRI, 2019. Same for the first sentence in 2.1, “in MRF, only 1/48”, this applies only to the used MRF sequence in the paper. Please add reference to your sequence or rephrase “in the used MRF sequence”
- I suppose the DM results are with fewer time frames (25 and 50%) as the ground truth was also obtained with DM but on all time frames. Maybe indicate this in Sec. experimental setup.
- The idea of using fewer frames is not novel. Please reference Fang et al., TMI, 2019
- DFT not introduced
- Typo: “allows each each spiral”
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The method introduces a novel way to infer parametric maps in MR fingerprinting. By doing so, a significant increase in reconstruction speed is achieved compared to other methods. Both method and experiments seem solid and well executed.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Very confident

Review #3

Please describe the contribution of the paper

The authors propose an end-to-end deep learning method to generate tissue parameter maps from non-Cartesian MRF data. The proposed method directly takes input of non-Cartesian k-space, and performs adaptive density compensation, and produces multiple parametric maps in a single forward pass. The method is validated in both 2D and 3D MRF, showing comparable accuracy to conventional gridding reconstruction and dictionary matching methods, while operating much faster.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The proposed real-time MRF mapping method is very novel, as it proposes to perform gridding and tissue mapping in a single forward pass with a light-weight network, which may significantly simplify the MRF data processing procedure and thus reduce the computation time.
2. The adaptive density compensation by inputting the polar coordinates of each pixel is novel and is proved to benefit the mapping accuracy
3. The concept of k-space of parameter maps is interesting
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. No proton density map is reconstructed.
2. The proposed framework requires the ground truth T1 and T2 maps reconstructed by the dictionary matching (DM) method. So the mapping error in the DM method may influence the network’s performance.
3. The T1 quantification accuracy is lower than the DM method
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The methods are clearly presented and explained, so this paper has good reproducibility.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
1. Besides T1 and T2 maps, proton density (PD) map should also be reconstructed to facilitate image synthesis. Please clarity why PD map was not reconstructed.
2. The sliding window operation of stacking 48 consecutive spirals seem to introduce the contrast mixing and reduce the temporal resolution. Will it influence the mapping accuracy?
3. It is noted that the mirco-network has quite different output channels for T1 and T2. How does the number of output channels influence the mapping accuracy?
4. For the baseline method, is the adopted NUFFT CPU-based or GPU-based? Since the deep learning method runs on GPU, consider using GPU-based NUFFT for fair comparison of reconstruction time.
5. The authors are encouraged to discuss the achievable acceleration factor by the proposed method.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The proposed method in the paper is novel, which can dramatically simplify the tissue paramer mapping process of MRF data.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

All three reviewers concur in their assessment to accept the paper. Therefore, I follow this suggestion. Still, there are a couple of points mentioned by the reviewers that should be implemented, if possible in the final paper.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

3

Author Feedback

We thank the reviewers and AC for their time and constructive feedback! Please find below our clarifications on some key questions.

R1: Hyperparameters of the competing methods? How is the dataset split? Is j a continuous index or a point index?

The implementation of all competing methods (DM, SDM, Low-rank, SCQ) follows the settings in the original papers. Particularly, the learning rate of SCQ of 0.0002 and the batch size of 32 are the same as in the original paper (Fang et al., TMI, 2019). We did not find SCQ or our method very sensitive to changes of these hyperparameters.

As indicated in “Experimental Setup”, a leave-one-out cross validation scheme was employed, i.e., all the slices of a subject were held out for evaluation each time.

j is a continuous index, denoting each grid point/pixel in a k-space/image-space tissue map of size M x M.

R2: Why is T1 worse than T2? Why is the performance on the 3D dataset better than 2D dataset? Short discussion.

In Table 1 (2D MRF), T1 (MAE: 4.24%) is better than T2 (MAE: 7.09%). Similarly, in Table 2 (3D MRF), T1 (MAE: 9.14%) is better than T2 (MAE: 11.78%). Therefore the performance on the 3D dataset is not better than the 2D dataset.

The higher accuracy of T1 than T2 is consistent with previous findings (B.Zhao et al., TMI, 2016; Fang et al.,TMI, 2019). This is because, due to the sequence used in this study, the early portion of the MRF time frames, which are used for training, contain more information of T1 than T2. Hence, all methods are more accurate in T1 quantification. DM based methods exhibit significant artifacts in T2 as indicated by the arrows in Figure 3. Our method is more advantageous in T2 mapping. The accuracy of T1 of our method is also acceptable under clinical standards, especially considering the significant improvement in speed. As a proof of concept, we have utilized a plain U-Net, which can however be replaced with a more advanced CNN to improve accuracy.

R3: Why proton density (PD) map was not reconstructed? Will sliding-window operation influence the mapping accuracy? How does the number of output channels for T1 and T2 influence the mapping accuracy? The achievable acceleration factor?

In this paper we focused on quantifying T1 and T2 maps as they are critical in many clinical applications. The method can be readily applied to PD quantification and will be considered in future work.

Sliding-window operation indeed negatively influences the quantification accuracy in our experiments. However, we notice that this impact can be mitigated by inputting multi-coil k-space data, instead of coil-combined data. Using sliding windows is a trade-off that can greatly alleviate memory issues, making it possible to take in thousands of time frames at the same time to enable end-to-end MRF processing.

The number of output channels for T1 and T2 does make some difference: the performance is found to be slightly better with a greater number of features for T2 than for T1. This is consistent with the fact that the early portion of the time frames contain more information for T1 than for T2, as mentioned above in response to R2.

Based on our experiments, our method can achieve up to 16X acceleration for 3D MRF (4X along the temporal dimension and 4X along the partition direction).

back to top

Real-Time Mapping of Tissue Properties for Magnetic Resonance Fingerprinting