Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Qiaoqiao Ding, Hui Ji, Hao Gao, Xiaoqun Zhang

Abstract

Image reconstruction in sparse view CT is a challenging ill-posed inverse problem, which aims at reconstructing a high-quality image from few and noisy measurements. As a prominent tool in the recent development of CT reconstruction, deep neural network (DNN) is mostly used as a denoising post-process or a regularization sub-module in some optimization unrolling method. As the problem of CT reconstruction essentially is about how to convert discrete Fourier transform in polar coordinates to its counterpart in Cartesian coordinates, this paper proposed to directly learn an interpolation scheme, modeled by a multi-scale DNN, for predicting 2D Fourier coefficients in Cartesian coordinates from the available ones in polar coordinates. The experiments showed that, in comparison to existing DNN-based solutions, the proposed DNN-based Fourier interpolation method not only provided the state-of-the-art performance， but also is much more computationally efficient.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87231-1_28

SharedIt: https://rdcu.be/cyhVb

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

This paper proposes a multi-scale DNN for sparse view CT reconstruction. The input projection is first converted to a 2D spectrum in polar coordinates. Then select the lower 1/8, 1/4, 1/2, and full-spectrum respectively to feed 4 independent processing branches. Each branch contains an “Interp” and CNN layer. “Interp” uses a “learnable adaptive weighting scheme.” The experiments showed that the proposed deep-learning-based multi-scale Fourier interpolation method not only provided state-of-the-art performance.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Very good reconstruction quality. Fast.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. Interpolation in spectral-domain is a challenging problem. As it is also indicated in the paper, spectral-domain is not smooth, normal interpolation doesn’t work. How the weights in eqn2 are trained, are they trained along with the CNN? The detail of the “Interp” block is not clear enough.
2. Why not interpolate in sinogram?
3. Why CNN is adopted follows the “Interp” block? why it is favorable for interpolating spectral data?
4. Training data is limited to prostate CT images.
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Reproducibility is low.
1. Weighting in “Interp” block is not known.
2. Code is not avaliable.
3. Details of training set is not known.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Give more details on “Interp” block
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Major factors are the reproducibility, quality of proposed method description.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

4
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

This paper proposed a multi-scale DNN for sparse view CT reconstruction. An interpolation scheme is directly learned to predict the complete set of 2D Fourier coefficients in Cartesian coordinates. The experiments showed the proposed method is computationally efficient and provides state-of-the-art performance.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The idea of using deep neural networks to perform multi-scale Fourier interpolation is interesting.

The experimental shows significant improvement. The efficiency against previous arts is demonstrated.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The hyperparameters are not well studied. For example, the performance using different numbers of K.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Code not provided.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Please refer to Block 4.
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The idea of using deep learning to do Fourier interpolation is interesting. One benefit is that the method avoids the projection or back-projection, which is computationally dense.

The experimental results show significant improvements against conventional methods.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Somewhat confident

Review #3

Please describe the contribution of the paper

This work proposes a neural-network-based Fourier-type reconstruction method for 2D CT reconstruction problems. They perform a learned regridding into 2D-Fourier space followed by a 2D Fourier reconstruction. This is performed multiple times in a multi-scale fashion. They evaluate their method against a number of other state-of-the-art methods in simulated parallel-beam and fan-beam scenarios.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Innovative approach of using a Fourier-type reconstruction for a learned reconstruction scheme
- Evaluation against many other competitive methods
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Extension to the practically relevant cone-beam geometry reconstruction difficult, except when using rebinning algorithms and not presented
- No standard deviations and tests for significant differences provided
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

A problem of this paper is that the experiments don’t report standard deviations of the methods output and don’t test the results for significance.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

The authors present a method following the less-explored Fourier-type of CT reconstruction algorithms. This is an interesting direction to explore. From the qualitative results figures I expected to see the groundtruth results but they are not shown. Please add those to judge how faithful the method recovers and preserves genuine detail. A specific comment about the fan-beam experiment: The authors state they used measurements from a “half-circle”. If that is correct this is unsufficient data for a fan-beam measurement since one needs 180 + fan_angle to reconstruct accurately from fan-beam data. If this is correct the methods try to compensate for the limited angle artifact and the sparse-view artifact simultaneously. Surprisingly this is not visible in the FBP images, but it may be obscured by the sparse-view artifacts.

There are some typos: Fig. 5: LAERN instead of LEARN in the figure subtexts Fig. 6: Biliner instead of Bilinear
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper presents a novel method in an underexplored direction which I find interesting. I do not believe the results are groundbreaking or already practically useful but may be an inspiration for the community.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

7
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

The reviewers of this paper concur in their assessment all three suggest acceptance of the paper. I follow this assessment and suggest that the authors implement the suggested changes and comments as much as possible in the final paper.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

7

Author Feedback

Rebuttal for “Learnable Multi-scale Fourier Interpolation for Sparse View CT Image Reconstruction” We thank the area chair and the reviewers for their careful reading and instructive comments. All comments will be taken into consideration in the revision for further improvement of the paper.

Response to Reviewer #1

“How the weights in eqn2 are trained, are they trained along with the CNN” The weights in eqn2 are trained by back-propagation along with the parameters in CNN.

“The detail of the “Interp” block is not clear enough?” We will elaborate it more in the revision.

“Why not interpolate in sinogram?” The main reason why the interpolation is done in FFT domain not sinogram comes from the concern on computational efficiency. For Fourier interpolation, the relating forward/backward operator is FFT which is more computationally efficient than the forward/backward operators involved in sinogram interpolation.

“Reproducibility is low and code is not available” Due to space limitation, only important details are covered in the main manuscript. Upon the acceptance of the paper, we will setup a Github project to put the well-documented code in public domain.

Response to Reviewer #2

“The hyperparameters are not well studied. For example, the performance using different numbers of K” We will study the sensitivity of K.

“Code not provided” Please refer to the response to Reviewer #1

Response to Reviewer #3

“Extension to the practically relevant cone-beam geometry reconstruction difficult, except when using rebinning algorithms and not presented” We believe the proposed method can be extended to 3D reconstruction. The proposed method originates from Fourier Slice Theorem, which interpolates the Fourier domain data from polar grid to rectangular grid. In the case of 3D reconstruction, we can make use of the Generalized Fourier Slice Theorem (GFST) [1][2] to define the interpolation domain, which enable us to extend our learnable interpolation network to solve 3D reconstruction. The proposed method can be extended to Helical CT reconstruction by incorporate with GFST directly. However, for cone-beam reconstruction, we have to deal with missing data problem and then adopt the proposed method.

[1] Zhao S, Yang K, Yang K. Fan beam image reconstruction with generalized Fourier slice theorem[J]. Journal of X-ray Science and Technology, 2014, 22(4): 415-436. [2] Zhao S R, Jiang D, Yang K, et al. Generalized Fourier slice theorem for cone-beam image reconstruction[J]. Journal of X-ray science and technology, 2015, 23(2): 157-188.

“I expected to see the ground-truth but they are not shown. Please add those to judge how faithful the method recovers and preserves genuine detail” Indeed, the ground-truth images are shown in Fig. 6 of the main manuscript.

back to top