Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Chen Hu, Cheng Li, Haifeng Wang, Qiegen Liu, Hairong Zheng, Shanshan Wang

Abstract

Image reconstruction from undersampled k-space data plays an important role in accelerating the acquisition of MR data, and a lot of deep learning-based methods have been exploited recently. Despite the achieved inspiring results, the optimization of these methods commonly relies on the fully-sampled reference data, which are time-consuming and difficult to collect. To address this issue, we propose a novel self-supervised learning method. Specifically, during model optimization, two subsets are constructed by randomly selecting part of k-space data from the undersampled data and then fed into two parallel reconstruction networks to perform information recovery. Two reconstruction losses are defined on all the scanned data points to enhance the network’s capability of recovering the frequency information. Meanwhile, to constrain the learned unscanned data points of the network, a difference loss is designed to enforce consistency between the two parallel networks. In this way, the reconstruction model can be properly trained with only the undersampled data. During the model evaluation, the undersampled data are treated as the inputs and either of the two trained networks is expected to reconstruct the high-quality results. The proposed method is flexible and can be employed in any existing deep learning-based method. The effectiveness of the method is evaluated on an open brain MRI dataset. Experimental results demonstrate that the proposed self-supervised method can achieve competitive reconstruction performance compared to the corresponding supervised learning method at high acceleration rates (4 and 8). The code is publicly available at \url{https://github.com/chenhu96/Self-Supervised-MRI-Reconstruction}. \keywords{Image reconstruction \and Deep learning \and Self-supervised learning \and Parallel network.}

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87231-1_37

SharedIt: https://rdcu.be/cyhVk

Link to the code repository

https://github.com/chenhu96/Self-Supervised-MRI-Reconstruction

Link to the dataset(s)

http://brain-development.org/ixi-dataset/

Reviews

Review #1

Please describe the contribution of the paper

This paper proposes a novel self-supervised learning method for MRI reconstruction. Specifically, two subsets are constructed by randomly selecting part of k-space data from the undersampled data. A difference loss is designed to enforce consistency between the two parallel networks. Experimental results demonstrate that the proposed method can achieve competitive reconstruction performance.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The idea is interesting. The self-supervised reconstruction problem is important in real clinical practice.
2. The paper is well written and easy to follow.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. The self-supervised framework is not new. The ISTA-Net is also from other work. Therefore the novelty is incremental for me.
2. The proposed method only compare to UNet and SSDU, which is not sufficient. There are many other more advanced MRI reconstruction methods. Also, they haven’t mentioned recent self-supervised methods for MRI reconstruction. Such as “SELF-SUPERVISED PHYSICS-BASED DEEP LEARNING MRI RECONSTRUCTION WITHOUT FULLY-SAMPLED DATA”
3. Why using ISTA-Net as backbone rather than others? What’s the difference between ISTA-Net and ISTA-Net+?
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The author mentioned that they will release the code.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

The proposed method is not novel for me. It seems the only new thing is the Diffloss. And the experiment is not sufficient.
Please state your overall opinion of the paper

probably reject (4)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The novelty and experiments.
What is the ranking of this paper in your review stack?

4
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Review #2

Please describe the contribution of the paper

This paper proposes a novel self-supervised learning method for MRI image reconstruction. They randomly select part of k-space data to construct two subsets which are fed into an ISTA-Net to perform reconstruction. Two reconstruction losses are defined on all the scanned data points and a difference loss is designed on unscanned data points to enforce consistency. Therefore, the deep network can be trained on unlabeled data. Experimental results on IXI dataset demonstrate the effectiveness of the proposed self-supervised method and achieved competitive performance compared to the supervised learning method.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. It proposes a novel self-supervised learning method for MRI reconstruction which can be trained using only the under-sampled data.
2. It overcomes the problem that the deep learnings rely on fully sampled data, and is more suitable for the actual scene of MRI.
3. Extensive experiments on an open dataset prove that the method can achieve good performance of reconstruction.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

This paper does not validate the method on the real sampling masks such as Cartesian and non-Cartesian sampling, and real datasets (i.e., complex valued MR image).
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

This paper provides sufficient details about the method, dataset and experimental implementation.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
1. There’s something wrong in “where N is the number of undersampled training data”.
2. $f()$ is not reflected in the Eqn.(5),
3. How can this method be applied to non-Cartesian sampling cases?
4. How much time does it take to train the network?
5. How does this approach compare with advanced model-based approaches?
Please state your overall opinion of the paper

strong accept (9)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
1. It is a novel method for MRI reconstruction.
2. It has practical application value.
3. Extensive experiments to validate the method.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

4
Reviewer confidence

Very confident

Review #3

Please describe the contribution of the paper

The authors have focused on self supervised learning for MRI reconstruction. The authors have proposed a parallel training framework for MRI reconstruction. The two branches of the parallel network take images obtained from two different subsets of the undersampled mask. The authors use reconstruction loss for both the branches while they have also developed difference loss between the output of the branches. The authors have validated their method with a public dataset and they have also conducted ablative study.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Self supervision is an important avenue in MRI reconstruction, and is being targeted recently.

The paper is written well and authors have required knowledge on the area of reconstruction.

The reconstruction loss with undersampled mask and the difference loss is a novel contribution to the field of MRI reconstruction.

The choice of ISTA-Net as the base network for MRI reconstruction is appreciable.

The dataset selection and experiment design are done appropriately.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The method has been only evaluated with a gaussian mask. Cartesian mask is the more practical choice.

The loss function could have been verified with networks like Sigmanet, VN, VS-Net. Likewise, the method could have been evaluated for different datasets, particularly with fastMRI.

The qualitative comparison could be presented better.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Paper is reproducible.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Please check weakness.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The chosen problem is an important one to address and even though the solution is a simple modification of the existing method, it is appreciable. Experiments and results are sufficient, but more study could be conducted.
What is the ranking of this paper in your review stack?

3
Number of papers in your stack

6
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

This paper proposed a self-supervised MRI reconstruction method by constraining the consistency between two reconstructions using different random samplings in k-space. This idea is interesting, and the reviewers all feel positive on this basic idea. But they also have some important questions that should be addressed, especially the extensions to other masks instead of the Gaussian masks, and other network backbones and datasets.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

3

Author Feedback

Dear Area Chairs and Program Chairs,

Thank you very much for organizing the review process and sending us the reviewers’ comments. We appreciate all the reviewers for their valuable comments and suggestions. Thanks also to them for recognizing that our idea is interesting and the positive attitudes to our paper. The main concern lies in the experimental part, since the reviewers wanted us to evaluate our method with more undersampling masks, other base networks and different datasets.

We understand this concern. In our submission, we instantiated our method with the following settings: 2D random undersampling mask, ISTA-Net+, and the open-source IXI Dataset. We agree with the reviewers that different settings could be experimented with as a way to investigate the generalization capability of our method. Additional experiments with different settings (e.g. utilizing 1D random and 1D uniform undersampling masks, adopting a different base network MoDL, and evaluating with in vivo clinical T2-weighted 12-channel brain MR data) have already been conducted. Here, we want to clarify that the performance of our proposed self-supervised reconstruction method, like the supervised methods, will also be affected by the base network architecture and the quality of the dataset. Nevertheless, despite the slightly influenced performance, all the results confirm that our method can consistently achieve promising reconstruction results under these different settings. We are willing to present more comprehensive results to further validate the effectiveness of our proposed method. However, considering that this will make a lot of changes to the original submission and including all these results with corresponding discussions will substantially exceed the page limit regulation of MICCAI submission, we don’t intend to make the corresponding supplements in the manuscript. Instead, we will upload all the corresponding results on the public online platform with our code for references as stated in our original submission.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This paper proposed a self-supervised MRI reconstruction method by constraining the consistency between two reconstructions using different random samplings in sub-sampled k-space. This idea is interesting, but the AC and reviewers have concerns on more justifications using other sampling masks / datasets / backbones / sampling rates, etc. The authors promised to put them online with the provided codes. Overall, the basic idea is interesting, and the experiments preliminarily justified its effectiveness, thought not sufficiently. I think the paper can be accepted, but the authors are suggested to include some of the new promised results (more sampling rates, 1D sampling masks) in the final version of this paper.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

5

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The reviewers and original MR appreciated the idea, and taking a look at the paper I like it as well – it’s simple but could be effective, and creative ideas are always appreciated.

I find it peculiar that the authors answered that they don’t want to expand the paper – of course, adding a substantial amount of experiments is not the right thing at this phase, but at least a discussion of the reviewer concerns is warranted. For example, how might the idea be affected by different mask strategies, which would certainly be of concern in practice.

Nevertheless, the paper should be accepted as it will lead to interesting discussion.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

5

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

While the paper has some interesting novel direction, I find the justification and validation underwhelming.

The issue with the validation is raised by the other reviewers as well, and authors’ rebuttal touches this point but does not address it. The IXI dataset, as far as I can find is only magnitude images. In the presence of fastMRI dataset, I do not think it is reasonable to only use magnitude images for a reconstruction article. The reason why it is important to use complex data is because phase is crucially important. Not predicting the phase correctly will diminish the reconstruction accuracy substantially.

As for the justification, authors state that obtaining fully sampled datasets may be difficult in certain applications. While I agree with this statement, brain imaging is not one such scenario. I suggest authors to focus on such a scenario, e.g., cardiac imaging, to demonstrate their method.

Unfortunately, after reading the article, reviews, meta-reviews and the rebuttal, I cannot suggest acceptance of this article.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Reject
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

16

back to top

Self-Supervised Learning for MRI Reconstruction with a Parallel Network Training Framework