Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Shawn Mathew, Saad Nadeem, Arie Kaufman

Abstract

Haustral folds are colon wall protrusions implicated for high polyp miss rate during optical colonoscopy procedures. If segmented accurately, haustral folds can allow for better estimation of missed surface and can also serve as valuable landmarks for registering pre-treatment CT (virtual) and optical colonoscopies, to guide navigation towards the detected anomalies. We present a novel generative adversarial network for feature-consistent image translation of optical colonoscopy videos to virtual colonoscopy renderings with haustral fold overlays. A new loss is introduced that utilizes a three-step cycle in order to leverage ground truth information between haustral fold annotations and virtual colonoscopy renderings. We demonstrate the effectiveness of our model on real challenging optical colonoscopy videos as well as on textured virtual colonoscopy videos with clinician-verified haustral fold annotations. All code and scripts to reproduce the experiments of this paper will be made available via our Computational Endoscopy Platform at https://github.com/nadeemlab/CEP.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87199-4_21

SharedIt: https://rdcu.be/cyl3X

Link to the code repository

https://github.com/nadeemlab/CEP

Link to the dataset(s)

https://zenodo.org/record/4993651

Reviews

Review #1

Please describe the contribution of the paper

Fast and accurate method for detection of Haustral folds in colonoscopy videos. The detection of folds are useful for polyp detection and improvement in the colonoscopy screening and the paper presents a GAN based approach to do this.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. A important application with translation potential.
2. Strong results.
3. Data annotation is extremely time consuming on real data. The authors present an elegant approach to circumvent this problem.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Nothing.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The code is being shared on github which is useful to evaluate and reproduce results. Question remains if the data will be publicly available also.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Nothing specific here.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This is an important problem in the endoscopy domain. The paper presents a realistic approach to solving a known problem. An elegant workflow to annotate data is presented for virtual images, with ability to translate it to optical colonoscopy.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

4
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

The paper proposed a generative adversarial network, FoldIt, for feature-consistent image translation of optical colonoscopy videos to virtual colonoscopy renderings with haustral fold overlays. Its applications include: better estimation of missed surface; benefit to register pre-treatment CT (virtual) and optical colonoscopies.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

1) This task has good clinical application value. 2) It is novel to perform haustral folds segmentation based on the image translation methods. 3) The experimental designs well show the superiority of the proposed methods.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

1) Only XDCycleGAN attended the comparisons, more comparisons would be much better. 2) In the technical part, the key idea is cycleGAN which is existing. (Although the novelty seems limited, it will not affect the contribution. )
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

It is not difficult to reproduce.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

A table to include the quantitative comaprisons would be better.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Evenly the technical novelty is limited, the proposed problem and the resulted performance are good and will have clinical value.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Review #3

Please describe the contribution of the paper
- Presents a novel method to detect and segment haustral folds in colonoscopy videos in a semi-supervised manner using GANs.
- Uses a new three-step cycle consistency loss utilising the common VC domain to achieve domain translation from VC to OC domain.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The three-step cycle consistency loss and use of the common VC domain: Domain gap when training on synthetic data and testing on real data is one of the main challenges in medical imaging and deep learning research in general. This paper utilises the common VC domain which the OC and VC are translated into and compared in. This seems effective in training the network using the synthetic data with ground truth and real data without ground truth.
- Extensive experiments on real colonoscopy videos.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- I only have minor comments: please see below.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

It looks OK.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- In sec. 3, ‘The OC and VC data was’ -> ‘…data were’, ‘Data from 7 patients was’ -> ‘…patients were’, ‘Examples frames from’ -> ‘Example frames…’
- In sec. 3, ‘2 light sources on either side of the camera’: Where is the either side of the camera? Is the camera stereo?
- In eq. 2, D is not defined anywhere.
- In pp4, ‘The main insight here is that the translation to the common domain, C, should be consistent between our domains A and B.’: A, B are defined only in the figure caption, not in the main text.
- ‘triple step cycle loss’: Isn’t the relation represented in the loss (eq. 4) rather transitive than cyclic?
- In pp5, ‘In our case, we have ground truth correspondence between haustral fold annotations and VC’: Isn’t the haustral folds annotations domain B? Then, A should be B in eq. 6.
- In the line after eq. 6, ‘where E_{x,z~p(A,C)} represents the paired data distribution (A,C)’: Maybe better to remove E here.
- In eq. 7, ‘G(z)’: Shouldn’t this be ‘G_{AC}(z)’?
- In sec. 5, Dice and IoU: Maybe better to put the numbers in a table.
- In fig. 4 caption, ‘networks’ -> ‘network’s’.
- There are two ‘isn’t’ and one ‘doesn’t’ -> ‘is not’, ‘does not’.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This paper presents a new semi-supervised method for haustral folds detection and segmentation. The method is evaluated with synthetic and real data which show the potential of the method.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

The paper is about DNN based detection of haustral folds in colonoscopy. The proposed method is elegant, new and the results are convincing. All three reviewers are on the same page and the paper can thus be accepted.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

1

Author Feedback

We thank all the reviewers for their comments. We have incorporated the minor comments from Reviewer 3.

back to top

FoldIt: Haustral Folds Detection and Segmentation in Colonoscopy Videos