Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Guibo Luo, Tianyu Liu, Bin Li, Michael Zalis, Wenli Cai

Abstract

Electronic cleansing (EC) is an image-processing technique for subtraction of tagged fecal regions in the colon for visualization of the entire colonic surface in CT colonography (CTC). This paper introduced a deep-learning based EC method in dual-energy CTC (DE-CTC), named “Deep-Cleansing”. First, we calculated the “effective” atomic number (EAN) by fractions of atomic mass number using the low- and high-energy images in DE-CT. Second, multiple types of combinations of input channels (images), and multiple backbone networks were investigated in U-Net architecture for the optimal performance of EC. Finally, we trained and evaluated our Deep-Cleansing by a total of 139 DE-CTC cases (approximately 80K DE-CTC images), which were randomly divided into training, validation and testing set at an approximate ratio of 55%:20%:25%. Our Deep-Cleansing method achieved DICE coefficients of 0.953 in validation and 0.956 in testing datasets, respectively. Overall, the average cleansing ratio was 96.14%, and the soft-tissue preservation ratio was 97.74%, both were significantly higher than EC without EAN maps. Our results indicated that the use of Deep-Cleansing substantially improved the accuracy of EC in the subtraction of tagged fecal regions in CTC.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87234-2_5

SharedIt: https://rdcu.be/cyl70

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The authors used an EAN map to differentiate colon lumen and tagged regions. They create a look up table for value mapping from DE-CT value to EAN map. DE CT image and EAN map are used as input to segmentation FCNs. Use of EAN contributed to improve segmentation accuracy.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

-This paper created a look up table for value mapping from DE-CT value to EAN map empirically. -Use of the EAN map as one of the inputs of segmentation FCN improved segmentation accuracy.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

-To define value mapping from DE-CT value to EAN, many parameters in equation (1) should be obtained. The authors estimated the parameters from X-ray attenuations of mixture materials they “empirically” created. I’m not sure whether they created the materials experimentally or they just selected parameter values experientially. The authors should describe how they created the materials and how they measured their X-ray attenuations in more detail. -EAN (or similar values) was used in a previous electronic cleansing method from DE-CT images [1]. The authors should explain difference from them. -In the Tables 2,3, improvements of accuracies by introducing EAN map were small. The improvements can be caused by random changes of parameters in the deep learning method. Statistical comparison of the segmentation results will be required.

[1] Tachibana R, Näppi JJ, Ota J, et al. Deep Learning Electronic Cleansing for Single- and Dual-Energy CT Colonography, Radiographics, 38(7), pp.2034-2050, doi:10.1148/rg.2018170173, 2018
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Reproducibility is good. Data and codes will be released.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

-I would recommend to describe how they created the materials for X-ray attenuation measurements in more detail. -Please include statistical comparison of the segmentation results to show performance of the proposed method. -Manual creation of ground truth data (or ROIs) of colon lumen and tagged region is difficult because their surfaces are unclear and complicated. I think the inter-annotator variation of the ground truth data is large. Explanations about how the authors compensated the inter-annotator variation will improve reliability to the ground truth data. -“effective anatomic number” means “effective atomic number”?
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

-This paper indicated that use of the EAN in colon lumen and tagged region segmentation improves their segmentation accuracies. However, a method that introduced similar idea [1] was already proposed. -Improvement of segmentation accuracies by introducing the EAN map was small.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

3
Reviewer confidence

Somewhat confident

Review #2

Please describe the contribution of the paper

Electronic cleansing is an image processing application in CT Colonography images to remove the contrast agent and the residual fecal matter and any other debris. Recently there are many papers on this technique where they consider the dual-energy CT images for processing instead of single energy CT images and based on the filtering techniques. This paper discusses a deep learning technique to cleanse such CT images before the colon polyp analysis. The contribution in the paper is good which adopts the mass attenuation coefficient concept for the calculation of voxel intensities of certain objects within the colon. The paper is fairly good from a technical point of view. Only the methodology, results, and evaluation metrics are discussed and there is no comparison with established results.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

This paper describes a deep learning method for the removal of contrast agents and the tagged fecal matter from the dual-energy CTC images. The organization of the paper content from the technical point of view and domain point of view is good. The methodology and results are discussed very well. The approach of solving the problem with the list of mass attenuation coefficients available in the NIST table for different materials is interesting as theoretically calculated Hounsfield are the same as the practically observed Hounsfield on CT images. This solves the majority of filtering and segmentation problems in CTC images when we do not get the images acquired with the required kVp values. However, few comments need to be addressed.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Despite the promising approach of virtual cleansing through this novel approach, even though a considerable amount of samples are considered for training, testing, and validation, the depth of testing looks too shallow. Mainly, the multiple set of multi-energy images were not considered, the entire image set is from only one pair of energy levels (80 and 140 kVp). It is not discussed anywhere that if other energy level images are given as input, how the model behaves in terms of results and accuracy.

The patient who is insufflated with CO2 which appears as noise in the colon lumen is not considered. It is very essential to consider such images also.

In the literature review, the state-of-the-art methods are only listed, their advantages or disadvantages are not discussed. The very first literature cited ([5]) is a too old reference which says that there is no accurate clinical solution, but there are many papers till recent past in this area using single energy CT or dual-energy CT. This old reference leads to confusion about the literature review. In results analysis, only quantitative evaluation is considered and there is no mention of qualitative analysis. It is directly not possible to accept the output of the black box without acceptance from a radiologist (at least one).
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The work can be reproduced on a high-performance computer with the dataset discussed in the paper. There is no difficulty in the coding as the predefined DL models are available which works on a set of images provided to the model. It is a bit difficult to check the performance of the DL model as it completely depends on the underlying hardware.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
This paper describes a deep learning method for the virtual cleansing of the tagged fecal material and endoluminal fluids from dual-energy CT images. The organization of the paper content from the technical point of view is good. There are few open questions from a domain point of view which need clarification to accept the outcome of this research as a potential candidate for a clinical solution.
1. Page 2: How is it in-homogeneously when the oral contrast is given? I believe that all the indigested food, residual fecal matter absorbs the contrast and starts exhibiting homogeneous intensities (only soft tissue structures remain the same in their behavior. Please check is this inhomogeneous?
2. Page 2: [5] It is the too old reference that says that there is no accurate clinical solution, but there are hundreds of papers till recent past in the area of colon virtual cleansing either using single energy CT or dual-energy CT. This old reference leads to confusion about your literature review
3. Page 2: [5] It is a too old reference that says that there is no accurate clinical solution, but there are hundreds of papers till recent past in the area of colon virtual cleansing either using single energy CT or dual-energy CT. This old reference leads to confusion about your literature review.
4. Were there any following problems noticed after cleansing? a. Soft tissue degradation; b. Soft tissue degradation of submerged structures; c. Incomplete cleansing; d. Incomplete cleansing due to partial volume effect.
5. In the literature review, the state-of-the-art methods are only listed, their advantages or disadvantages are not discussed.
6. In section 3.2, you have mentioned the air inside the lumen which is true. But as part of the CTC protocol, the patient is insufflated with Co2 also in some cases for the proper colon distension. In such cases, the Co2 appears as cloud formation inside the lumen with the intensity values in the range of -950 to -1020 HU (roughly) and this is usually the noise. Your algorithm did not discuss any technique for handling such noise within the lumen. Please clarify. Were there any such datasets in your pool?
7. The images of 80 and 120 kVp are discussed in the entire paper, but on Page 5, 3rd paragraph, you have mentioned 120 kVp, is this due to oversight?
8. Section 4.1: This is the window value for colon visualization, but CTC has two different window values (-200C, 1500W) and -350C, 1400W). Have you considered this?
9. Page 7, 1st paragraph: Did you try other quantitative evaluation techniques such as absolute volume difference, average symmetric surface distance, maximum symmetric surface distance, and root mean square symmetric surface distance. If so, which one did give a better result, if not which one in the used technique gave the good results?
10. Qualitative (subjective) assessment (observer’s opinion) is missing in the study.
11. Mentioning the HU values of colonic contents like air, contrast, soft tissue, water, fat, and the tagged fecal matter is missing. It is good to show the values as this paper talks about the DE-CT images.
12. Fig 5: The base (from where the haustral folds and polyps protrude towards the lumen center)of the colonic structures is not identified in this segmentation technique?
13. Fig 5: Partial volume effect cases are not shown?
14. Fig 5: Air-Contrast boundary usually has non-homogenous voxel intensities due to floating debris. This is a bit difficult to clean virtually as it could lead to erosion of soft tissue structures. How is this air-contrast boundary removed in the proposed method?
15. Fig 5: There is only one pair of dual-energy images shown in the paper. Was the study limited to only 80, 140 KvP or was there any other cases with other combination of energy levels? Have you tried your method on different voltage level combination images? If so did you modify any of the hyperparameters in your DL model?
16. Fig 5: Was there any Soft tissue structure erosion due to excessive post-processing? Were enhanced voxels corrected?
17. Was there any non-homogeneous attenuation coefficient near air-contrast boundary due to medium dose bowel preparations (any such cases)
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The objective of the study is achieved which is defended through the results and statistical analysis. Even though the abstract is convincing, the Conclusion is not strong enough which completes just in 4 lines. The paper is borderline as there are many questions raised seeking clarification. To evaluate the paper correctly, 60% of the explanation is acceptable but for the remaining 40% clarifications are required. Overall there is still more work remaining regarding the samples with different energy level pairs in dual CT.
What is the ranking of this paper in your review stack?

3
Number of papers in your stack

3
Reviewer confidence

Very confident

Review #3

Please describe the contribution of the paper

EAP maps plays a key role in this method, which is an important technology of image enhancement and can improve the performance of segmentation. EAP maps change the input of the deep learning.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

This paper proposed an effective and practical solution for the issue of cleansing the tagged fecal materials in the colon from the colonography via the dual energy CT. Three steps are involved in this method including EAN map generation, colon lumen segmentation, and tagged region subtraction. Compared with other methods such as SegNet and U-Net, the first step is an important stage of this method where five types of channel combinations with dual-energy images and EAN maps are proposed as the input of the segmentation. EAP maps plays a key role in this step, which is an important technology of image enhancement and can improve the performance of segmentation. Experiments demonstrates that this method not only effectively remove the fecal materials in colonography but also significantly improve the segmentation results for colon lumen.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

1, The five type of channel combinations should belong to the input of U-Net. So channel combination paragraph is suggest to move to Section 3.1. 2, Eq(1) and the coefficient calculation need some references. And the fusion for dual-energy images with EAN maps required more details. 3, The flowchart and the network structure will facilitate our understanding of this method if the paper has related figures. 4, All tagged regions should be labeled in original images. 5, Since the paper uses two type of Networks to run segmentation, Fig 5 should specify the segmentation method. 6, Comparisons with the state-of-the-art will make this paper much stronger. 7, The conclusion section is not clear and needs more works.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

this paper could be reproduced.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

DE-CT and DE-CTC are easy to confused. i personall suggest to change them. The results of 4th combination and 5th combination are very similar, some quantitative measures are required to show their differences.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This method use deep learning to carry out an interesting task which includes fecal materials removing and colon lumen segmenting. the image preprocessing is wonderful and its results is encourging.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

5
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

All three reviewers recommend borderline acceptance of this paper with construction suggestions. In particular, reviewer 2 provides detailed feedback on how to improve the paper, especially the testing and comparison with established methods. A rebuttal is invited to address the comments raised by reviewers.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

2

Author Feedback

1.Reviewer 1 (a) Create the materials for X-ray attenuation measurements and EAN calculation. We have developed accurate calculation of the mass attenuation coefficient (MAC) based on the photo-atomic interaction model in the context of Monte Carlo simulation for given X-ray energies and selected materials (like air, soft-tissue, fat, water, tagging mixtures etc.) over 80–140 kVp spectra. Another manuscript regarding the calculation and evaluation of MACs and EANs in numerical experiments and a phantom study has been submitted to “Medical Physics”, which is in the revision. We will add a table in Figure 2 to show the HUs and EANs of selected materials suggested by the reviewer. (b) The difference compared to literature [1] The EC method in Literature [1] is based on material decomposition in DE-CT. They did not discuss EAN and its application in EC. Literature [1] used CNN models to material-classify of each voxel: air, soft-tissue, tagged fecal material, air-tagging boundaries, etc., and then replaced the tagging materials and air-tagging mixtures with air, as that of conventional EC. In contrast, we used U-Net segmentation method to directly segment the entire colon lumen (including air, tagged materials, and air-tagging mixtures) and replace it with air. (c) The manual creation of ground truth data (or ROIs) of colon lumen and tagged region is difficult, and how to improve the reliability of the ground truth data. We will emphasis the difference of our method from conventional EC methods (such as Literature [1]). As the reviewer mentioned, it is difficult to create reliable ROIs of luminal air and tagged materials, because the air-tagging boundaries is highly variable. However, contours of colon lumen or surface layer is more reliable than ROIs in conventional methods. Moreover, we utilized intelligent scissors to optimize the contours. Therefore, our method significantly reduced the inter-annotator variations and improved the reliability of the ground truth.

2.Reviewer 2 (a) Whether the method still works on different energy level pairs in DE-CT or the colon lumen insufflated with CO2. EAN of a specific material varies weakly over 80–140 kVp spectra. We calculated multiple EAN look-up tables in terms of the different pairs of energies applied in DE-CT, such as 80-140 kVp or 100-140kVp.
CTC exams in our datasets were acquired using CO2 insufflation. Because our EC method segments the entire colon lumen regardless of the heterogeneity of luminal air (air or CO2) or different tagging contents, we observed that our method was not affected by CO2 noise. In addition, both luminal air or CO2 present high EAN values that differentiate from soft-tissue in colon surface. (b) The literatures reviewed are not the latest. We will update the literature review in the final version of the manuscript. Existing EC methods based on DL can be categorized as material classifier or GAN model. To the best of our knowledge, our paper is the first to build an EAN-based U-Net model to segment the entire colon lumen for EC purpose. (c) The concerns about experiments and EC artifacts such as colonic structures, partial volume effect, …, and erosion of soft tissue structures. We observed that three major EC artifacts mentioned by the reviewers were significantly reduced after we subtracted the entire colon lumen as they tend to be caused by inhomogeneity or mixtures within the colon lumen. A clinical study is in progress to evaluate various EC artifacts. (d) Some quantitative evaluation techniques To evaluate the technical performance of our U-Net models, we used relative volume difference (RVD), and two EC metrics: cleansing ratio (Rec) and soft-tissue preservation ratio (Rst), which measures the less-segmentation and over-segmentation (or erosion) of the colon lumen, respectively. These metrics are more specific than general metrics for the EC purpose.

3.Reviewer 3 We will add the comparison study with other methods as suggested by the reviewer.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The authors have addressed the comments from all three reviewers and the paper would be acceptable for MICCAI publications with the conditions that literature review and figures are updated in the final camera ready paper.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

2

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This is a solid paper attacking a problem of interest in CT colonography. All reviewers saw merit in this work although some concerns were raised. These concerns seem to be reasonably well addressed in the rebuttal.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

4

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The authors provided rebuttal accounted partially for the raised concerns. Lots of R2 raised points have not been addressed in the authors’ response (e.g., evidence for their reply to R2.(c) and others). Also, their response regarding “the difference compared to literature [1]” as raised by R2 I do not think ref [1] should be termed as “conventional method” as it is still a deep-learning-based method targeting the solution of the problem from a different view point (i.e., different methodology). Also, I did not see big improvements of using EAN as the results in Tables 2 and 3 indicates and no statistical analysis is provided to support the significance
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Reject
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

11

back to top

Deep-Cleansing: Deep-learning based Electronic Cleansing in Dual-energy CT Colonography