Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Junshen Xu, Esra Abaci Turk, P. Ellen Grant, Polina Golland, Elfar Adalsteinsson

Abstract

Fetal motion is unpredictable and rapid on the scale of conventional MR scan times. Therefore, dynamic fetal MRI, which aims at capturing fetal motion and dynamics of fetal function, is limited to fast imaging techniques with compromises in image quality and resolution. Super-resolution for dynamic fetal MRI is still a challenge, especially when multi-oriented stacks of image slices for oversampling are not available and high temporal resolution for recording the dynamics of the fetus or placenta is desired. Further, fetal motion makes it difficult to acquire high-resolution images for supervised learning methods. To address this problem, in this work, we propose STRESS (Spatio-Temporal Resolution Enhancement with Simulated Scans), a self-supervised super-resolution framework for dynamic fetal MRI with interleaved slice acquisitions. Our proposed method simulates an interleaved slice acquisition along the high-resolution axis on the originally acquired data to generate pairs of low- and high-resolution images. Then, it trains a super-resolution network by exploiting both spatial and temporal correlations in the MR time series, which is used to enhance the resolution of the original data. Evaluations on both simulated and in utero data show that our proposed method outperforms other self-supervised super-resolution methods and improves image quality, which is beneficial to other downstream tasks and evaluations.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87234-2_19

SharedIt: https://rdcu.be/cyl8e

Link to the code repository

https://github.com/daviddmc/STRESS

Link to the dataset(s)

http://crl.med.harvard.edu/research/fetal_brain_atlas/

Reviews

Review #1

Please describe the contribution of the paper

The authors propose a learning-based framework for increasing the resolution of fetal MRI acquired using an interleaved slicing scheme.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The method presented in the paper is highly relevant in fetal MRI, notably for modalities such as fMRI where the commonly used approach of acquiring scans in multiple orthogonal directions for increasing image resolution is not applicable. The method is methodologically sound, the presentation is very clear and the paper is well-written. The experiments and evaluation are detailed and the results favorable for the proposed method.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The main weakness that I see in the paper is the lack of discussion of the effects of motion on the method. While the authors do evaluate their method on simulated motion trajectories (the exact nature of these trajectories is unclear. are they real or simulated? how strong are they? how continuous/instantaneous are they?), it is unclear if the proposed super-resolution method requires a-priori motion correction of some sort and what the effects of failure in this would be.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

some data used in the paper (CRL fetal brain atlas) is publicly available. However, the motion trajectories used in the paper are not, and are not described in sufficient detail to allow for reproduction of the results. Also, no implementation is given.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

The paper is well-written, I only have minor specific comments. in 2.1., “stack, making inter-slice motion artifacts within each frame is milder.” , is not a proper sentence In 3.1., the nature of the simulated (?) motion should be described in more (quantitative and qualitative) detail In Figure 3., right panel, the meaning of the x-axis is not clear (s /mm ? slices / mm??)
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The proposed methods presents an important step in the processing of fetal MRI apart from the “standard” multi-view super-resolution approaches. Importantly, applications such as placental imaging and functional imaging would profit greatly from the proposed method and further research in that direction, which, together with the methodological and formal quality of the present paper lead me to propose its acceptance.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

4
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

The authors present a new method for generating high resolution MRI images from low resolution fetal MRI images. The novelty is designing an algorithm for interleaved EPI sequences and takes into account 4d or timeseries data.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

I have not seen a super resolution algorithm for fetal data designed for 4d data. I believe that is novel.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Main weaknesses are really the lack of statistical tests and the abbreviations (which make reading the paper hard).
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

I did not see if the code or data is publicly available. But I hope the authors make their trained model and code available open-source.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

I appreciate that the page limit for MICCAI can be challenging, but there are a few too many abbreviations in the introduction. It makes it very hard to read. For example, are LR, HR, SR, LR2 needed?

It would be useful to perform some level of statistical testing to compare STRESS to SMORE and the other comparison methods. From the error bars, it looks like there is a significant improvement but performing the tests is needed.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

I think the paper overall is well done. It has some flaws but that are not major.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Review #3

Please describe the contribution of the paper

This paper presents a SSR framework for dynamic fetal MRI with inter-leaved acquisition, named STRESS (Spatio-Temporal Resolution Enhancement with Simulated Scans).
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Authors evaluate the STRESS framework on both simulated and in utero data to demonstrate that it can not only enhance resolution of dynamic fetal imaging but also improve performance of downstream tasks.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Some parameters are not very clearly defined.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

I think it is not very easy to be reproduced, only if authors share the code.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
1. Your work introduced the concept of “ interleaved slice acquisition”, and the subsequent experimental data show that this approach also has good results. However, there is only a certain amount of textual description of this method and an abstract representation in fig. 1. As a reader, I could not understand the concept well. Therefore, can you add some explanations that are more relevant to the work of the article, such as adding some actual demonstrations about the process of slicing fetal MRI images, which can show more clearly the process of data processing by the method you used.
2. With regard to the “low SNR” proposed in this section of the article, can you clarify what criteria are used to judge this concept in the implementation of the method?
3. The article mentions the possibility of improving the performance of downstream tasks, can you be a little more specific to highlight the advantages of the method and the resulting image results in the article?
4. The flowchart for STRESS is the most critical schematic of the article. For the ABC three parts, in fact, is the main work and innovation point of the article, but did not highlight it. For part D, it is also redundant with the previous three parts of the content see, while taking up a large proportion of the entire structure of Fig.1. Fig.1 should have some structural problems and may not be intuitive enough for the reader. Is it possible to split ABC & D into two figure or use other ways to better reflect the structure and characteristics of STRESS.
5. For Fig.2 and Fig. 4, is it possible to appropriately increase the presentation of the data to make the experimental results more full and complete. Also, can you highlight some details of the presentation, such as partially zooming in a part of the image, so that the reader can compare the results of different models more intuitively.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The experimental result.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

4
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The authors describe a method to improve the resolution, motion-correction and downstream assessment for EPI based fetal data. Called STRESS, it consists of simulating an interleaved slice acquisition on the original data used to train a super-resolution network to enhance the original data. Evaluations are shown on simulated data and in-utero data (T2- weighted and EPI-based).

The reviewers emphasise the relevance of the studied challenge and the use of different types of data set for evaluation as strong points of this paper.

The limitations mentioned from all three reviewers include a lack of details in the methods, in the description of the data (eg described motion trajectories) and achieved results (eg lack of statistical tests) as well as too many abbreviations used. Various minor comments are made, eg on a better description of “low SNR” as described in the paragraph Blind-spot denoising.

Major comments:
- I agree with the attested lack of detail. The method is not described in a way which would allow the reader to easily reproduce it. The novelty is - as pointed out by one of the reviewers - not very well described, esp also not in Figure 1. The schematic Figure is hugely helpful, but the contributions need to be made clearer and the flow of data / HR/LR images etc pointed out better.
- Details about the used data and performed experiments are missing. The acquisition referred to in reference [6] is not EPI-based, but a T2wSSFSE sequence with interleaving of 2 or 4 as described in the reference. How does this alter the method, has this been evaluated? Or are you assuming these to be completely-motion free and are using these just like a high res fetal dataset? The parameters of the applied motion are not given and the description in 3.1 is somewhat superficial. It would really help to show the motion curves (eg the rotation and transformations applied) and to comment on how realistic they are. Also - this data including the transformations would be of interest for other researchers in the area, I would thus follow the recommendation of one of the reviewers to make this available to enhance the reproducibility of this work.
Additional minor comments:
- Please refer to the minor comments made by the reviewers.
- The motivation to improve fetal data is obvious, however, an additional sentence on the “downstream tasks” etc would be hugely helpful.
- Better description of the visual results, eg in Figure 2 and 4 would be good. Replacing “Visual results” by eg, “reformatted sagittal and coronal views” would help.
- Parts of the results (eg page 7 last paragraph when commenting on SMORE and SI results) are speculative and should not be given in the results section. Some unproven claims should be removed, eg “STRESS is able to generate MR time series with high image quality which is beneficial to downstream tasks”. This might well be the case, but needs to be linked to better evidence.
- Abbreviations in the Figures are note helpful, eg Fig. 3 has many and at least the most important ones relevant to appreciate the result should be outlined in more detail (eg PCK and PSNR/dB)
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

4

Author Feedback

First of all, I would thank the reviewers for their constructive feedback. In the rebuttal, we focus on the major concerns raised by reviewers.

Details of fetal motion trajectories (AC, R1) As mentioned in Sec. 3.1, we use three keypoints (two eyes and the midpoint of two shoulders) to model the fetal head pose at each frame. Let X be the vector between the two eyes, Y be the normal vector of the plane defined by these three keypoints, Z be the cross product of X and Y and O be the average of the three keypoints. X, Y, Z axes, and origin O define a coordinate of the fetal head. The transformation of this coordinate is the motion trajectory of the fetal head in the MR time series. The motion trajectories used in the experiment are extracted and sampled from real scans but no generated from random numbers, therefore, they are more similar to the fetal motion that occurs in clinical practice. Some statistics of the motion trajectories are as follows: Rotation: mean=3.10 degree/s, std=3.75 degree/s, max=59.7 degree/s; Translation: mean=2.40 mm/s, std=1.80 mm/s, max=21.36 mm/s. Our method doesn’t require preprocessing such as motion correction.

Details of the proposed method (AC) In this work, we propose STRESS, a self-supervised super-resolution framework for dynamic fetal MRI with interleaved slice acquisitions, which has the following key features: a) It is self-supervised, so doesn’t need ground truth data for training, which are usually unavailable in fetal imaging. Besides, the self-supervised manner makes it possible to optimize our model for a particular subject. b) It utilizes the characteristic of interleaved slice acquisition and doesn’t need multi-oriented image stacks for oversampling. c) It can be applied to 4D fetal dynamic MR data. d) Our method doesn’t require preprocessing such as motion correction. Given 4D fetal MR data that is interleaved along the z-axis, we interpolate the data along the z-axis and swap the x- and z-axes. Let x’, y’, z’ denote the new axes, then x’=z, y’=y, z’=x. First, we downsample swapped data along z’ axis using the same interleaved pattern (Fig. 1 B). Second, we extract y’-z’ planes from volumes before and after downsampling to form high-res and low-res pairs. And use these pairs to train a self-supervised super-resolution network (Fig. 1 C). Finally, we apply the trained network to the y-z planes of the original acquired data to perform super-resolution. Other details can be found in our released source code https://github.com/daviddmc/fetalSR. More detailed comments and tutorial scripts are coming soon. Besides, we will add more description on fig. 1 to make it clearer, thanks for the comment.

The use of CRL dataset (AC) Our method doesn’t make any assumption on the contrast of the MR images. In the experiment, we use the CRL atlas as a motion-free high-resolution dataset, which provides ground truth for evaluation. Besides, it covers a wide range of gestational ages, which helps demonstrate that our method can be generalized to different gestational ages.

Statistical tests (R2) Thanks for pointing it out. We performed paired t-test to compare our method with other baselines and the results will be added to the evaluation section.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

STRESS is a method to improve the resolution, motion-correction and downstream assessment for fetal data. It consists of simulating an interleaved slice acquisition on the original data used to train a super-resolution network to enhance the original data. Evaluations are shown on simulated data and in-utero data (T2- weighted and EPI-based).

As outlined in the original meta review, the main criticisms from the review stage were the lack of details in certain areas (eg on the motion parameters and the methods), a better explanation why the CRL data set was used, the need to add stats and some additional minor comments.

The authors give additional details on the fetal motion trajectories with many details including that they are extracted from real data and the ranges of rotations/translations. They also give more details on the method (which I hope also end up in the paper!) and the link to the well documented code repository.

They promise to add paired t-tests to compare the method with other baselines. Based on these comments, and especially the details given on the motion trajectories - which I hope they will make publicly available for other researchers in the area to benefit as well - I recommend accepting this paper. The explanation why CRL data was used is still not very convincing in my opinion and could still be made clearer!

All in all the paper can be interesting for the community, esp with the code & trajectories released.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

4

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This works presents a highly original and relevant contribution for fetal fMRI, with a novel SR network. Overall, the reviews were already very positive and also in agreement as regards the added value of this work. Most concerns were in my opinion minor as regards mostly some missing details and need for further discussions. I think authors add these relevant details on the “generated” motion and also of the method itself. Justification of a baseline experiment using T2WI make sense also to me. I do not see though in the rebuttal the results of the statistical analysis (is the proposed approach statistically significant better?), but authors claim that these are now computed and will be included.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

2

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

Reviewers thought that the method was well presented and novel. Concerns about clarity seem to be well addressed in the rebuttal. The authors indicate that code will be released.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

1

back to top

STRESS: Super-Resolution for Dynamic Fetal MRI using Self-Supervised Learning