Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Tao Wang, Wenjun Xia, Yongqiang Huang, Huaiqiang Sun, Yan Liu, Hu Chen, Jiliu Zhou, Yi Zhang

Abstract

Metal implants can heavily attenuate X-rays in computed tomography (CT) scans, leading to severe artifacts in reconstructed images, which significantly jeopardize image quality and negatively impact subsequent diagnoses and treatment planning. With the rapid development of deep learning in the field of medical imaging, several network models have been proposed for metal artifact reduction (MAR) in CT. Despite the encouraging results achieved by these methods, there is still much room to further improve performance. In this paper, a novel Dual-domain Adaptive-scaling Non-local Network (DAN-Net) is proposed for MAR. We correct the corrupted sinogram using adaptive scaling first to preserve more tissue and bone details as a more informative input. Then, an end-to-end dual-domain network is adopted to successively process the sinogram and its corresponding reconstructed image generated by the analytical reconstruction layer. In addition, to better suppress the existing artifacts and restrain the potential secondary artifacts caused by inaccurate results of the sinogram-domain network, a novel residual sinogram learning strategy and nonlocal module are leveraged in the proposed network model. In the experiments, the proposed DAN-Net demonstrates performance competitive with several state-of-the-art MAR methods in both qualitative and quantitative aspects. The code is available online: https://github.com/zjk1988/DAN-Net.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87231-1_24

SharedIt: https://rdcu.be/cyhU8

Link to the code repository

https://github.com/zjk1988/DAN-Net

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The authors proposed a novel network for dual domain MAR in CT. The two main contributions in the network are the adaptive scaling of the metal trace and non-local modules in the image domain network. The network was trained on simulated data. The network was compared to SOTA MAR algorithms, both qualitatively and quantitatively on simulated data as well as one real CT image. An ablation study was performed and showed the effects of the non-local modules and adaptive scaling.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

A very thorough evaluation and comparison to SOTA.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

No major weakness
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors provide the code. The parameters are clearly stated.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
I liked the detailed description of the SOTA. However, the part describing the critical limitations could be more clear.

The problem formulation should be more clear. The transformations in Eqn. 2 (line 2 to 3) is not understandable and the text does not help the understanding. S_v is never used again.

The proposed network is described well. however some questions remain:
- why is the non-local module not embedded after each each sampling step?
- why is the L_FBP included into the objective function? why is is valid to include? is is really necessary?
- fig. 3: Can’t see real differences between the methods. Where are the blue arrows pointing at?
While i liked the broad evaluation and large number of comparison methods, details on the configurations are missing. Please add them to ensure a fair comparison.

More details about clinical data would be nice. Please add some more discussion on the real data set. Where is the metal mask come from? Are all SOTA methods getting the same mask?

The method is shown for 256x256. Is it still computably feasible for real clinical image sizes?

Some minor errors:
- fig. 1: typo domain
- fig. 2: image numbering and caption does not match
- dataset: which unit for the CT values?
- fig. 4: caption: A2- G2
- title supressed due to length
Please state your overall opinion of the paper

strong accept (9)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

I believe the improvements to the dual domain learning make sense and improve the results for MAR. Especially the non-local Blocks seem to make a difference. The evaluation is solid. I value the work and think it should be shown at MICCAI.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

4
Reviewer confidence

Confident but not absolutely certain

Review #2

Please describe the contribution of the paper

The authors present a data-driven method for metal artifact reduction. A dual-domain adaptive-scaling network using residual sinogram learning strategy and non-local module is developed for metal artifact reduction (MAR) in CT.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

(1) The authors provide an adaptive-scaling strategy to simultaneously reduce the artifacts and preserve the detail structure near the metal. This might be a proper way to utilize the contaminated projection data which in most cases is discarded. (2) The idea of using non-local U-Net architecture to reduce the streak artifacts is practical, and it also shows better results. (3) Verification on both simulated datasets and clinical cases are provided， which show the clinical feasibility.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

(1) In this paper, the sinogram-domain network uses mask pyramid network (MPN) to modify the projection data within the metal trace. However, this method is not novel enough. The paper ““Generative mask pyramid network for CT/CBCT metal artifact reduction with joint projection-sinogram correction” has already used MPN to modify the projection data. (2) The idea of utilize the contaminated projection provide better details around the metal objects. However, it is also adopted in recently published paper, for example, cited paper [15], [25]. I back up authors’ idea that it is necessary to treat this contaminated data more carefully, but I believe a direct comparison with these State-of-the-Art methods will be more convincing. (3) The residual sinogram learning strategy is supposed to be effective, however, an ablation study is needed as well, especially if the authors regard it as a major contribution of this paper.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The code is submitted to Github, and the complete network structure is also supplemented in the material. The data used in this article comes from the public data set, and the training details and parameter settings are explained in the article. This article is with highly reproducibility.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

(1) ‘Adaptive scaling’ is kind of confusing, the coefficient λ is set to 0.4 experimentally for all the images according to my understanding. Also, is it a good choice to use a fixed coefficient without regard to the size of metal objects? (2) In the Poisson noise simulation, the incident beam X-ray was set to 2×10^7 photons. This is a relatively high dose, under which the simulated Poisson noise is weak with respect to the signal (that is, the SNR is high). Actually, the incident photons in clinical applications will not be set to exceed 10^7. The incident photons may need to be reduced to simulate a more reasonable Poisson noise.

Some minor problems might be fixed for a better paper: (1). Typo in 3.3. NMAR is mistakenly written as NAMR twice. (2). In Fig. 2, (A2)-(H2) are labeled as (A3)-(H3). Also, the specific window width is not mentioned. (3). Figure 3 is not obvious. Please try to adjust the display window or add markers.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The idea of this paper is not so refreshing, since algorithms based on dual domain neural network have already archived many successes in MAR problem. Extra experiments and comparisons are needed though authors’ algorithm actually performs well.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Review #3

Please describe the contribution of the paper

This paper introduces an adaptive scaling and global residual learning for sinogram inpainting using a proposed non-local U-Net.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

This paper is a slight modification of the dual domain network (DuDoNet) for CT metal artifact reduction. The slight modification mainly lies at the adaptive scaling. However, there is no significant improvement in methodology compared with DuDoNet.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. The authors introduced the non-local network modules into the U-Net to get a large receptive field. However, the U-Net, with its down-sampling/up-sampling operations and concatenations, is known to have a large receptive field. Therefore, I don’t think that adding such a non-local network module will make a difference.
2. The authors introduce an adaptive scaling for the sinogram inpainting. However, the scaling parameter is set to a fixed value 0.4 instead of being trained. Therefore, it lacks the adaptivity.
3. The “novel” residual sinogram learning is widely used/known now. So it is no longer novel.
4. Overall the method has no significant improvement from the DuDoNet.
5. The captions of subfigures are missing. It is difficult to find which image corresponds to which method.
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Yes
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Please describe the context of each subfigure in the caption; otherwise it is very difficult to know their meaning. For region-of-interest (ROI) patch display, please choose a small ROI patch with a large zoom-in factor. In Fig. 5, the ROI patches are almost the same as the original figures.
Please state your overall opinion of the paper

reject (3)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

No novelty in methodology.
What is the ranking of this paper in your review stack?

4
Number of papers in your stack

4
Reviewer confidence

Very confident

Review #4

Please describe the contribution of the paper

Adaptive scaled (a blend of LI and raw sinogram) sinogram and image are used as inputs to retain tissue details. Non-local U-Net improves the image enhancement performance. Experimental results on simulation and real data show decent MAR performance.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Adaptive scaling helps to retain edge information within metal trace in the sinogram data, and recover tissue structure near the metal in the reconstructed image.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Hyperparameter lambda is an adaptive scaling factor to compensate for the beam-hardening effect. Constant lambda may not work well when metal material changes. Domain gap exists between simulation and real data, lambda chosen is not applicable on real data.
- Limited performance on clinical data. In Fig. 4, the streak artifacts near the right bone are not suppressed completely, and DuDoNet performs better.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The dataset is public and the code is available.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- As mask projection is fed to sinogram network, the authors should compare with “Encoding metal mask projection for metal artifact reduction in computed tomography”.
- In the ablation study, adaptive scaling seems to be the main contribution. Comparing DAN-net with Ma-Dual-Net, it is not clear the performance boost comes from mask projection or adaptive scaling.
- The authors should present how lambda affects the network performance.
- Present cropped metal affected sinogram as B’ for better comparison.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Adaptive scaling recovers the details within metal trace and somehow improves the MAR performance.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

3
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

The paper obtains mixed reviews (3 positve, 1 negative). R1/R2/R5 are postive and R4 is negative, mainly due to his/her concern about lack of novelty in methodology. R4 believes that the proposed modification of DuDoNet is rather minor. However, R5, who the AC knows for sure is DuDoNet expert, expresses no such concern, instead s/he thinks that adaptive scaling does help retain edge information within metal trace. Here I choose to weigh R5’s opinion more and therefore downplay R4’s view.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

2

Author Feedback

Reviewer #4: The authors introduce an adaptive scaling for the sinogram inpainting. However, the scaling parameter is set to a fixed value 0.4 instead of being trained. Therefore, it lacks the adaptivity. R: For the details of adaptive scaling, please refer [27].

Reviewer #1: why is the non-local module not embedded after each sampling step? R: According to [29], in the deep layer of the network, the spatial size of feature maps was small and the non-local module cannot provide enough spatial information. If the non-local module was used at the first layer, an expensive computational cost will be paid. In addition, using multiple non-local modules can further improve the performance. As a result, we embedded non-local modules after the second and third down-sampling layers.

Reviewer #2: the incident beam X-ray was set to 2×10^7 photons. This is a relatively high dose, under which the simulated Poisson noise is weak with respect to the signal. R: We followed the same strategies used in [23], [25] and [28].

The main contributions of this paper and the differences between our method and DuDoNet are as follows: a) To suppress the artifacts and preserve tissue details efficiently, adaptive scaling is applied. b) Because of the fact that the metal has a much higher attenuation coefficient, the projection data inside and outside of the metal trace can be regarded as obeying two different data distributions. It is difficult to convert two different data distributions to a unified distribution for normal networks. To tackle this problem, a residual learning strategy that only modifies the metal trace region values of the adaptively scaled sinogram is used. c) To alleviate the new artifacts introduced in image domain enhancement and expand the receptive field, a novel nonlocal U-Net architecture that can capture long-range dependencies is proposed.

back to top

Dual-Domain Adaptive-Scaling Non-Local Network for CT Metal Artifact Reduction