Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Hangfan Liu, Tanweer Rashid, Jeffrey Ware, Paul Jensen, Thomas Austin, Ilya Nasrallah, Robert Bryan, Susan Heckbert, Mohamad Habes

Abstract

Deep learning for medical image analysis requires large quantities of high-quality imaging data for training purposes, which could be often less available due to ex-istence of heavy noise in particular imaging modalities. This issue is especially obvious in cerebral microbleed (CMB) detection, since CMBs are more discerna-ble on long echo time (TE) susceptibility weighted imaging (SWI) data, which are unfortunately much noisier than those with shorter TE. In this paper we present an effective unsupervised image denoising scheme with application to boosting the performance of deep learning based CMB detection. The proposed content-adaptive denoising technique uses the log-determinant of covariance matrices formed by highly correlated image contents retrieved from the input itself to im-plicitly and efficiently exploit sparsity in PCA domain. The numerical solution to the corresponding optimization problem comes down to an adaptive squeeze-and-shrink (ASAS) operation on the underlying PCA coefficients. Obviously, the ASAS denoising does not rely on any external dataset and could be better fit the input image data. Experiments on medical image datasets with synthetic Gaussian white noise demonstrate that the proposed ASAS scheme is highly competitive among state-of-the-art sparsity based approaches as well as deep learning based method. When applied to the deep learning based CMB detection on the real-world TE3 SWI dataset, the proposed ASAS denoising could improve the preci-sion by 18.03%, sensitivity by 7.64%, and increase the correlation between counts of ground truth and automated detection by 19.87%.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87231-1_26

SharedIt: https://rdcu.be/cyhLX

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

This paper proposes an unsupervised denoising technique combined with a deep CNN model for improved CMB detection. Experiments show that the proposed method (ASAS) is beneficial to the accuracy of CMB detection.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The writing is good. The proposed method seems reasonable.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The main weakness of this paper is the experiment. The reported experimental results are not convincing.
1. The noise in most medical images is not Gaussian.
2. Table 1. The performance of DnCNN is inferior to BM3D. These results are weird. Many literatures [1] have demonstrated that DnCNN is better than BM3D (especially for Gaussian noise).
3. Lack of comparisons with recent unsupervised denoising methods (e.g. Noise2Void [2], Self2Self [3]).
4. A denoised image will inevitably lose some image details. It is not clear why denoising can help detection.
[1]Kai Zhang. “Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising”. IEEE TIP, 2017. [2] Alexander Krull. “Noise2Void - Learning Denoising from Single Noisy Images”. CVPR 2019. [3] Yuhui Quan. “Self2self with dropout: Learning selfsupervised denoising from single image”. CVPR2020
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Satisfactory
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

See weaknesses.
Please state your overall opinion of the paper

borderline reject (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

My main concern about this paper is that the reported experiments cannot demonstrate the superiority of the proposed method.
What is the ranking of this paper in your review stack?

4
Number of papers in your stack

2
Reviewer confidence

Confident but not absolutely certain

Review #2

Please describe the contribution of the paper

An unsupervised adaptive squeeze-and-shrink (ASAS) denoising technique is proposed. Proposed technique has been successfully applied to long echo time susceptibility weighted imaging data. A deep learning model for improved cerebral micro-bleed (CMB) detection has been proposed.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Proposed ASAS denoising technique is novel, improves state of the art and it is unsupervised. Combination of the denoising technique and the deep neural model achieves a good performance in early cerebral micro-bleed detection.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

I don’t see any weaknesses of this paper except for a minor errata in an equation.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Enough information about used dataset for gaussian white noise removal and CMB Detection has been provided.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
Paper is well written, and interesting problem and a good and useful solution has been described and a good experimental section –showing good experimental results– included.

I only have detected a minor errata easy to solve:
- Page 4, subsection 2.2: Left hand side equation is wrong though right hand side equation is true. This is easy to solve.
Please state your overall opinion of the paper

strong accept (9)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

CMB detection requires long echo time (TE) susceptibility weighted imaging (SWI), producing noisy images where CMD is hard to detect. The proposed method could have very useful clinical applications.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

4
Reviewer confidence

Very confident

Review #3

Please describe the contribution of the paper
-Proposed a new method to denoise SWI method based on the idea of signal adaptive.
- Sparse representation is used to efficiently denoise SWI images.
- The proposed denoising method can great improve the segmentation performance.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

-The signal adaptive method in sparse representation is new to some extent. The squeeze and shrink design is not proposed before in sparse representation. It optimizes sparse representation of images without explicitly training the signal-adaptive dictionaries.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

-Limited improvement in comparison with existing method. The average PSNR/SSIM of proposed model only has marginal improvement than CSR which is proposed in 2011. -Lack of novelty in deep detection. Although the paper is titled for improving deep detection. The deep detection section (section 3) is not new.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

-The experimental setup is based on a dataset that is publicly available. -Code are not open to public yet.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

-The improvement of precision is overclaimed. The improvement of precision is 5%, rather than 18.03% while the improved precision is 0.3286. -It is recommended to compare proposed method with more recent work. In current manuscript, it is compared with BM3D (2007), CSR (2011), and DnCNN (2017). -It is not clear why there is a point with 0 precision and 0 recall in Fig. 5
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The major factor that led to the overall score of this paper is novelty of this method and the fact that it brings a better segmentation performance in real world dataset.
What is the ranking of this paper in your review stack?

4
Number of papers in your stack

8
Reviewer confidence

Somewhat confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

The authors propose a denoising technique, consisted of sparse representation and deep learning, for susceptibility-weighted MRI with the goal of improving cerebral microbleed detection. All reviewers agree that there is some merit to the work, but the decision of acceptance will be further motivated if the authors can better clarify the novelty of their work, the significance of their results, as well as the justification for the chosen Gaussian model for the noise (e.g. how about motion artifacts?).
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

7

Author Feedback

We thank the reviewers for their efforts in reviewing the paper.

Comment: Since noise in most medical images is not Gaussian, the chosen Gaussian model for the noise should be justified. (R1, Meta-R2) Answer: We did not assume the noise to be Gaussian distributed; in fact, the proposed method does not depend on the distribution of noise. We only used Gaussian white noise as the synthetic noise in Sec. 4 to evaluate the denoising performance, because it is the most widely used noise model for testing denoising methods. Please note that in Sec. 5 we used real-world clinical data, in which the noise distribution is unknown, but the proposed approach is still demonstrated effective, as shown by the visual comparison in Fig. 4 and the improved cerebral microbleed (CMB) detection performance in Table 2.

Comment: In Table 1, the performance of DnCNN [15] is inferior to BM3D, but literatures (e.g., [15]) have demonstrated that DnCNN is better than BM3D. (R1) Answer: Previous works usually test DnCNN on natural image datasets such as BSD68 [15], on which DnCNN indeed exhibits superior performance. However, such superiority partially relies on the closeness of the training data and the testing data. When the test set is dissimilar with the training set, the trained model may not fit the test data so well. The results in Table 1 indicate that DnCNN may not be well generalized to the medical image datasets like DX and MMM. In contrast, BM3D and the proposed ASAS do not rely on training data and are better at generalization.

Comment: Denoised images inevitably lose details. It is not clear why denoising can help detection. (R1) Answer: By pursuing optimal signal-adaptive sparsity, the proposed ASAS strives to preserve the fine details while removing noise. Since the image contents of interest are well preserved and the interference of noise is effectively suppressed (see Fig. 4), the proposed ASAS not only significantly reduces false positives, but also makes the lesions more discernable and thus increases true positives. Besides, when data size is limited, using data with higher quality could improve the performance of the deep learning model.

Comment: The improvement of precision should be 5%, rather than 18.03%. (R4) Answer: By saying “ASAS denoising improved the precision by 18.03%” we meant relative improvement, which is calculated by (P_denoised–P_original)/P_original=(0.3286-0.2784)/0.2784=18.03%, while the reviewer considers the absolute improvement, which is P_denoised–P_original=0.3286-0.2784=0.0502.

Comment: It’s not clear why there is a point with 0 precision and 0 sensitivity in Fig. 5. (R4) Answer: One of the subjects in the dataset doesn’t have any CMB, hence the corresponding true positive is 0 and both precision and sensitivity were trivially set to 0.

Comment: The author should better clarify the contributions of their work. (Meta-R2) Answer: We combined a novel unsupervised denoising scheme ASAS and a deep learning model to tackle the challenging problem of CMB detection. The proposed ASAS achieves signal-adaptive sparsity without explicit dictionary training. We revealed that ASAS essentially applies adaptive squeeze-and-shrink operations to the eigenvalues. When tested on real-world clinical data, ASAS significantly improved the performance of the deep neural network for CMB detection.

Comment: The compared methods are not the most recent ones. (R1, R4) Answer: We used BM3D [7], CSR [9] and DnCNN [15] for comparison, considering that BM3D is the most widely-used benchmark for denoising in recent years, CSR is a typical and successful sparsity based method and is in the same category as the proposed ASAS, and DnCNN is a representative deep learning based denoising method. Table 1 shows that ASAS is highly competitive.

Comment: In Sec. 2.2: The left side equation is wrong though the right side one is true. This is easy to solve. (R2) Answer: We thank the reviewer for the kind reminder. We’ll correct the typo.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The authors have well responded to the reviewers’ concerns, and I think the paper is suitable for MICCAI.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

7

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

Authors have addressed major concerns from reviewers. This paper provides a novel unsupervised idea for image denoising that improves microbleeds detection.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

12

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This paper proposed Adaptive Squeeze-and-Shrink Image Denoising method, together with deep network, for detection of cerebral microbleeds. It is shown that the denoising can improve the detection task. The reviewers mostly feel positive and have questions on the non-realistic Gaussian noises, comparison with DnCNN, precision of improvement, etc. The response clearly clarified these points. One remaining question is on the comparisons with SOTA, since in denoising, DnCNN is not SOTA, and many learning-based methods have been proposed in medical image denoising, recently. However, considering that this proposed method is unsupervised and has merits in the framework of traditional denoising framework, the paper can be accepted.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

9

back to top

Adaptive Squeeze-and-Shrink Image Denoising for Improving Deep Detection of Cerebral Microbleeds