Back to top List of papers List of papers - by topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Liyan Sun, Hongyu Huang, Xinghao Ding, Yue Huang, Xiaoqing Liu, Yizhou Yu
Abstract
Magnetic Resonance Imaging (MRI) in a specific Region Of Interest (ROI) is valuable in detecting biomarkers for diagnosis, treatment, prognosis accurately. However, few existing methods study ROI in both data acquisition and image reconstruction when accelerating MRI by partial k-space measurements. Aiming at utilizing limited sampling resources efficiently on most relevant and desirable imaging contents in fast MRI, we propose a deep network framework called ROICSNet. With a learnable k-space sampler, an ad-hoc sampling pattern is adapted to a certain type of ROI organ. A cascaded Convolutional Neural Network (CNN) is used as the MR image reconstructor. By using a ROI prediction branch and a three-phase training strategy, the reconstructor is better guided onto the regions where radiologists hope to look closer. Experiments are performed on T1-modality abdominal MRI to demonstrate its state-of-the-art reconstruction accuracy compared with recent general and ROI-based fast MRI approaches. Our model achieves accurate imaging on fine details in ROI under a high accelerator factor and showed promise in real-world MRI application.
Link to paper
DOI: https://doi.org/10.1007/978-3-030-87231-1_10
SharedIt: https://rdcu.be/cyhUB
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The paper proposes a network called ROICSNet that jointly learns k-space sampling and image reconstruction towards ROI. The proposed method is compared to state-of-the-art CS-MRI approaches and has shown promising results in different regions of organs in MRI data.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- It is the first work that jointly learns k-space sampling and image reconstruction towards ROI. The problem is novel and important in clinical practice.
- The method is well described and easy to understand.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The training procedure is complicated and difficult to reprocodure without releasing the code.
- The experiment is not comprehensive. Lack of ablation study. It would be good to compare the results of using only Eq. (7), (8) and (9) using the proposed model to illustrate the importance of joint learning k-space sampling and image reconstruction towards ROI. The DC-CNN and ROIRecNet use pre-fixed under-sampling masks. The proposed method is using a learned mask. Did you compare the results of using the same mask? Such as DC-CNN and ROIRecNet use a learned mask and the propose method uses a pre-fixed mask. In this way, we can tell what improvement can a learned mask brings.
- The author mentioned that “In ROICSNet, we increases the number of RecBlocks from 1 to 3 and to 5”. I’m wondering if the compared methods also use a larger number of RecBlocks, will the difference still large. Since we seldom use “1” RecBlock in real practice.
- What’s Lsp in equation (8)?
- In the experiment, the authors compare the ROI PSNR. We can still see there are lots of artifacts in other regions. It seems it’s a tradeoff. Is this practical in real clinical applications? Will physicians accept such artifacts.
- What’s the conclusion? is the improvement of reconstruction from the ROI optimization or the learned undersampling mask?
Other problems:
- Is the dataset real k-space or simulated data? If it’s simulated data, it’s better to describe how to simulate them.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The training procedure is complicated and difficult to reprocodure without releasing the code.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- Provide more experiments to prove the significance of the proposed method.
- Draw a more clear conclusion depending on the experiment.
- Please state your overall opinion of the paper
borderline accept (6)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The problem is interesting and novel to the community. However, the experiment is not enough to prove the significance of the method. Also, the equation and dataset are not well illustrated.
- What is the ranking of this paper in your review stack?
2
- Number of papers in your stack
5
- Reviewer confidence
Confident but not absolutely certain
Review #2
- Please describe the contribution of the paper
The paper presents a method for accelerated MRI reconstruction that focuses on regions of interest. The ROI is presented using segmentation maps during training and the model learns to focus on these regions, instead of the background. This contrasts with prior methods that focus on reconstructing the entire image, and the resulting image can be affected by the background or regions that are unlikely to be interesting in clinical practice.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Simple method to incorporate RoIs in the reconstruction process.
- The resulting method is fast and obtains good results.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Limited novelty in the model.
- Experiments were performed on single-coil images while many modern MR scanners use multi-coil scanners. It would be good to extend this into the multicoil setting.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The dataset used is publicly available, but the code has not been released. However, the authors do specify the hyper parameters used to generate the results which should make their results reproducible.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- Present more experimental results. In particular, presenting SSIM scores would be good. SSIM has been found to be more reliable than PSNR (Knoll et al. 2020).
- Extend the method to the multi-coil setting and compare against recent SOTA methods.
- Please state your overall opinion of the paper
borderline reject (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The presented method is interesting and could be quite useful in practice if extended to the multi-coil setting. However, the limited novelty of the presented method leads to this rating.
- What is the ranking of this paper in your review stack?
3
- Number of papers in your stack
5
- Reviewer confidence
Confident but not absolutely certain
Review #3
- Please describe the contribution of the paper
The authors proposed a ROICSNet model for reconstructing high-quality MR images focusing on certain areas. The main contributions are 1) joint learning of k-space sampling and image reconstruction towards ROI; 2) introducing a ROI prediction branch under a multi-phase training scheme; and 3) showing superior results when comparing with other methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
It is novel to jointly learn both k-space sampling and image reconstruction towards ROI. Results of the proposed method looks promising when comparing with other methods.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
None
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
This paper can be easily reproduced since the authors described their method very clearly.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
The authors should consider adding comparisons of the number of parameters in different methods. It is hard to find differences in ROI regions among different results in figure 2. Please consider adding zoomed in subfigures.
- Please state your overall opinion of the paper
accept (8)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Some factors like the novelty of the proposed method, experiment design and result analysis.
- What is the ranking of this paper in your review stack?
1
- Number of papers in your stack
5
- Reviewer confidence
Very confident
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper focuses on learning k-space sampling for reconstruction considering region of interest. This task is meaning-full. The reviewers raised some questions on the ablation study, extension to multi-coil setting, etc. Moreover, the AC has question on learning line-based sampling pattern (instead of random pattern) in k-space, which is more easily applicable in real MRI.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).
4
Author Feedback
Q1: The Meta-Reviewer raised the question on the 1D line-based sampling pattern. R1: In ROI data-adaptive 1D Cartesian sampling, we learn the sampling probability of each line according to Eq. (3) with a little modification. Taking liver as ROI, we compare the proposed ROICSNet (33.89dB) with DC-CNN (28.62dB), ROIRecNet (30.96dB), LOUPE (31.21dB) and ROI-LOUPE (32.15dB) on 1D 20% Cartesian sampling pattern using the ROI-PSNR metric. We observe consistent results on other organs being ROI. The results demonstrate the ROICSNet maintains advantages over other state-of-the-art models in the line-based sampling pattern.
Q2: The Meta-Reviewer and Reviewer #1 raised the question on the ablation study. R2: The DC-CNN model uses a pre-fixed sampling mask for whole image reconstruction. The ROIRecNet further adds a ROI retraining using the loss in Eq. (9) with pre-fixed sampling mask and \lambda_{roiseg} and \lambda_{sp} being 0. The LOUPE jointly optimizes whole image reconstruction and adaptive sampling learning using the loss in Eq. (8). Compared to DC-CNN, the ROIRecNet proves the advantage of guiding the DC-CNN onto ROI and the LOUPE proves the advantage of joint learning k-space sampling and image reconstruction. The LOUPE-ROI adds the ROI retraining using the loss in Eq. (9) with \lambda_{roiseg} being 0, demonstrating the advantage of adapting the learnable sampling trajectory towards ROI. When the ROI prediction branch works (\lambda_{roiseg} is 0.01) in Eq. (9), the LOUPE-ROI further evolves into our ROICSNet. All the compared models share the identical network architecture except for above mentioned differences for fair comparison. According to Reviewer #1, we also increase the number of blocks from 1 to 3 for all the compared models. Under the 2D 20% sampling pattern, we compare the ROICSNet (44.99dB/0.967) with DC-CNN (29.84dB/0.904), ROIRecNet (32.07dB/0.895), LOUPE (35.64dB/0.865) and LOUPE-ROI (36.02/0.915) in the metrics (ROI-PSNR/ROI-SSIM). The results of ROICSNet reported here is different from the manuscript, because ROI prediction branch is applied in each block here and only in the last block in the manuscript. The results prove the consistency between the ROI-PSNR and ROI-SSIM and the advantages of our ROICSNet model still hold true when number of blocks increase. We also train 3-block DC-CNN and ROIRecNet using the sampling mask obtained from our 3-block ROICSNet, the resulting model performance is improved by 1.48dB and 8.66dB. This result proves the learned sampling masks from the ROICSNet capture the information of the underlying ROI organs in k-space. With a joint learning of k-space sensing and image reconstruction, the accuracy can be further improved.
Q3: The Meta-Reviewer and Reviewer #2 raised the question on the multi-coil setting. R3: Parallel CS-MRI model like VS-Net (MICCAI 2019) can be used as baseline deep reconstructor for our ROICSNet. The data consistency block in VS-Net admits analytical solution, which can be fitted into the learning of k-space sampling in Eq. (6). The ROI prediction branch can be incorporated into the denoiser block in the VS-Net. The resulting ROICSNet can be optimized using the 3-phase training scheme in the manuscript.
Q4: Responses to other concerns. R4: (1) For reproducibility, we release our code at https://anonymous.4open.science/r/ROICSNet_from_S_to_R-367F without invalidating anonymity. Related Codes will be released publicly soon. (3) L_sp in Eq. (8) promotes sparsity of sampling pattern. The explanation can be found in page 6. (4) Despite the proposed ROICSNet targets specific organs as ROI, the whole abdomen regions can also be treated as ROI at the expense of outer black backgrounds. Our experimental results show the ROICSNet still brings improvement and help radiologists better observe anatomical structures. (5) We use the publicized real k-space dataset in the format of DICOM. The detailed description can be found at https://chaos.grand-challenge.org/Data/.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper focuses on learning k-space sampling for reconstruction considering region of interest. This task is meaning-full. The reviewers raised some questions on the ablation study, extension to multi-coil setting, etc. Moreover, the AC has question on learning line-based sampling pattern (instead of random pattern) in k-space. In the responses, the authors provided results on 1D sampling pattern, and also clarified the concerns. Overall, it is an interesting work can be accepted, and the authors are suggested to incorporate these new results in responses to the final version of this work.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).
3
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper proposes to learn k-space sampling for reconstruction considering region of interest. The “first” claim is too bold, as there were papers in MICCAI 2020 sharing similar ideas. Nevertheless, this paper has its merit, and the idea is novel in general. Reviewers were mostly positive. There are confusing points in the paper, such as lack of displays of sampling masks.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).
4
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This work presents a learning k-space sampling for the reconstruction considering specific region of interest. The reviewers agreed on the limited novelty (joint k-space sampling and image reconstruction and ROI) of the approach but overall proceed to a favorable evaluation, as results are promising as regards SOA. Major points for discussion raised by the meta-reviewer are in my opinion very well addressed by the authors in the rebuttal. I did particularly appreciate the new added experiments on line sampling pattern (I hope it can be included in the final version). The authors also add more modifications according to R1 comments. I think that indeed multi-coil setting should be also included but probably not needed in a conference paper. I think that given the new experiments and some minor details on figures and clarifications as raised by the reviewers the paper could be accepted.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).
4