Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Qing Wu, Yuwei Li, Lan Xu, Ruiming Feng, Hongjiang Wei, Qing Yang, Boliang Yu, Xiaozhao Liu, Jingyi Yu, Yuyao Zhang

Abstract

For collecting high-quality high-resolution (HR) MR image, we propose a novel image reconstruction network named IREM, which is trained on multiple low-resolution (LR) MR images and achieve an arbitrary up-sampling rate for HR image reconstruction. In this work, we suppose the desired HR image as an implicit continuous function of the 3D image spatial coordinate, and the thick-slice LR images as several sparse discrete samplings of this function. Then the super-resolution (SR) task is to learn the continuous volumetric function from a limited observation using a fully-connected neural network combined with Fourier feature positional encoding. By simply minimizing the error between the network prediction and the acquired LR image intensity across each imaging plane, IREM is trained to represent a continuous model of the observed tissue anatomy. Experimental results indicate that IREM succeeds in representing high-frequency image features, and in real scene data collection, IREM reduces scan time and achieves high-quality high-resolution MR imaging in terms of SNR and local image detail.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87231-1_7

SharedIt: https://rdcu.be/cyhUy

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The paper proposes to use implicit neural representations to motion-correct, reconstruct and up-sample high resolution MRI images. Multiple anisotropic low-resolution sample volumes are acquired and a NeRF is trained. Experiments are done on 7T MRI adult data (n=7), 3T flair adult (n=2), and 4 T1 scans with high resolution reference.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- the paper tackles the important problem of fast imaging with a recently popular deep learning method. In theory infinity high up sampling should be possible.
- the paper is nicely written and solid in terms of method and evaluation.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- this is quite heavily based on the neural radiance field (NeRF paper) with some tweaks
- do the authors need to train one model per brain? This should be clearly stated and practical implications discussed and added to the paper.
- unlike motion correction combined super resolution as cited XXX, in this paper they do rigid registration initially, so it just aligns the LR slices in a canonical space, then they train the model like NeRF, where the known LR voxels corresponds to spatial location. Which limitation does this
- a point to bear in mind is the Nyquist sampling again, could there be a situation where (in the case of very thick slices), the SR network is likely to fail to learn maybe some “feature” in-between the thick slice scans(?)
- more subjects could have been studies and limitations discussed in more detail with more examples of failure cases and pitfalls.
minor: abstract: ‘an fully’ -> ‘a fully’
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

neither data nor code are available or promised to be made available thus reproducibility is very low.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

please see above. NIce paper. I have been waiting for somebody to apply NeRF or SIREN to this kind of problem.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This is one of those straight to the point papers and it has all the necessary comparisons for a conference paper. Good paper, good findings, solid presentation. should be accepted.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

4
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

In this work, a framework called IREM is proposed to perform super-resolution of MRI scans. The proposed framework is based on a fully connected neural network combined with a Fourier feature mapping. Initial experiments show promising results for the proposed approach.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

-Preliminary results are promising -The proposed network and its training are described in great details. -The paper is well structured and easy to follow
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

-3 small datasets are used -The novelty of the proposed approach is limited to the combination of 2 existing methods. -Simulated images used for the validation are obtained by a downsampling function and this is not necessarily the case in a real-world scenario. -Quality assessment is not done systematically and no evaluation from experts is provided. -Comparison with other method is also minimal
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Although the code and the data are not publicly available, the paper provides enough details to make the code reproducible.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

It is not clear to me what is the novelty of the proposed approach, in comparison with other existing solutions such as the work proposed in [20]. More specifically both the Fourier feature mapping and Fully-connected network seem methods already proposed in the literature.

Although the authors claim to use 3 datasets, all of them, contain a small amount of MRI scans.

One of the advantages of the method is that it seems that does not require a large amount of HR data set for training the network, as mentioned by the authors. However such small datasets and the large number of parameters in the network is subject to overfitting problems and poor generalizability. How the method overcomes this problem and according to what the authors claim that the method can be trained on small datasets?

From the paper seems that the authors cannot acquire pair of LR and HR images (from the paper “no actual GT can be built”). Why it is not possible to acquire HR and LR images of the same patient and register the image to have aligned pairs?

And if this pair acquisition is not possible, why the authors have not considered an unsupervised approach to perform super-resolution? (i.e. Adversarial training with cycle consistency for unsupervised super-resolution in endomicroscopy).

The conclusion presented in Fig.6 they are a bit vague. Who did the evaluation in Fig 6? Is there any clinical expert involved in this evaluation? Can the proposed approach create unrealistic super-resolved structures that would compromise clinical diagnosis?
Please state your overall opinion of the paper

borderline reject (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

-The authors should increase the number of MRI to validate the proposed method.

-The claim related to the use of only a small number of images required to train the proposed approach seems not realistic.

-Is not clear who did the quality assessment and whether clinical experts are involved in it. (Figure 5b referred in the text does not exist. I believe they refer to fig. 5).

-The proposed method is based on the combination of existing techniques.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Review #3

Please describe the contribution of the paper

This paper introduces an image reconstruction network named IREM that uses 3D image spatial coordinate to prediction image intensity. The network is trained using LR images. It can be used to predict HR images at arbitrary coordinates.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

1) The IREM method is able to predict HR images with an arbitrary up-sampling rate. 2) The network does of need for HR data for training. 3) It can provide equivalent level of image quality as realistic HR images and higher SNR.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

1 The main weakness of this paper is the need for a set of anisotropic images along different orientations which are not commonly available. 2 In practice, the scan time for three LR images are not significantly shorter than that of a HR image, as in the real data example of this paper. 3 No results are provided about the accuracy of the prediction results. The provided PSNR and SSIM measures do not reflect the prediction accuracy.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The data and code of this paper are not shared. But the network architecture and training parameters are provided. Thus, I think the results may be reproduced.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

1) The proposed method randomly selects one from the three LR scans as fixed image for data normalization. The trained networks may depend on the selected images. Adding more comments about the random selection would be helpful to justify the method. 2) In the training data, there are multiple LR image stacked at the same spatial coordinate. It is not clear their relationship with the true intensity. 3) The above question leads to a concern about the accuracy of the estimated intensity of HR images. The evaluation methods used the PSNR and SSIM metrices. It would be helpful to include more results on the prediction error, e.g., the normalized root means square error. Moreover, the definitions for PSNR and SSIM need to be explained.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

1) The proposed method is novel for super resolution reconstruction. 2) The method, such as the random selection of reference space, and the evaluation results, e.g. the prediction accuracy, have several problems need to be justified.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

1
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

The authors propose a super-resolution MRI approach based on existing deep neural network architectures. Although the novelty in the network architecture is limited, there is a consensus that its application to super-resolution MRI is novel and useful. Other issues raised by the reviewers, such as the dataset size and the extensiveness of the experiments, are expected to be addressed later in an extended version of the work.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

3

Author Feedback

N/A

back to top

IREM: High-Resolution Magnetic Resonance Image Reconstruction via Implicit Neural Representation