Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Hyeonwoo Cho, Kazuya Nishimura, Kazuhide Watanabe, Ryoma Bise

Abstract

The domain shift problem is an important issue in automatic cell detection. A detection network trained with training data under a specific condition (source domain) may not work well in data under other conditions (target domain). We propose an unsupervised domain adaptation method for cell detection using the pseudo-cell-position heatmap, where a cell centroid becomes a peak with a Gaussian distribution in the map. In the prediction result for the target domain, even if a peak location is correct, the signal distribution around the peak often has a non-Gaussian shape. The pseudo-cell-position heatmap is re-generated using the peak positions in the predicted heatmap to have a clear Gaussian shape. Our method selects confident pseudo-cell-position heatmaps using a Bayesian network and adds them to the training data in the next iteration. The method can incrementally extend the domain from the source domain to the target domain in a semi-supervised manner. In the experiments using 8 combinations of domains, the proposed method outperformed the existing domain adaptation methods.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87237-3_37

SharedIt: https://rdcu.be/cymat

Link to the code repository

https://github.com/chohenu/Cell_Detection-MICCAI

Link to the dataset(s)

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6233481/

Reviews

Review #1

Please describe the contribution of the paper

The main contribution of this paper is using unsupervised domain adaptation to the cell position heatmap to improve cell detection accuracy.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The main strength of this paper is a novel idea to use unsupervised domain adaptation applied to the cell detection problem.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The structure of the paper could be improved.
- Comparison with other conventional object detection method (e.g., Masked R-CNN) should be conducted.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The proposed method is based on a conventional generateive model, so I believe the reproduction of the method is relatively easy.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

This paper proposes an interesting domain adaptation method for cell detection. Unlike conventional domain adaptation methods usually convert the input data directly to the target data, the proposed method uses the result from the detection network as a pseudo label and matching them in an unsupervised manner. Even though the proposed method is built upon a conventional generative adversarial network, the idea of using the generator result as a pseudo label and using a discriminator to improve the detection network seems novel. I like the idea a lot.

One caveat is that the main idea was not clear until I reach the experiment section. For example, Fig 2 explains that pseudo labels are generated from the prediction of D (Step 3), but it is not explained anywhere (I was expecting to find these details in Section 2 but failed). I realized that D is trained using the discriminator when I read the experiment section. I believe the source of confusion might be from the fact that people usually consider pseudo labels are “generated” using some techniques in conventional weak supervised learning rather than the network itself is adapted to generate the correct labels after fully trained. I believe the authors can rewrite the text to clarify this in early Section 2.

Misc: The first paragraph in Section 2 seems redundant to the rest of Section 2. It could be shortened and put some resulting images in the main text.
Please state your overall opinion of the paper

strong accept (9)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

I like the idea and the results demonstrate the performance of the method well.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

4
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

In this work, the authors proposed to solve the domain shift problem in cell detection tasks by a pseudo-cell-position heatmaps and uncertainty based Bayesian network. Extensive experiments on various domain adaptation settings indicates the effectiveness of the proposed method by outperforming the comparison methods consistently.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

1 The proposed method utilize the pseudo labels in the target domain for detection, which alleviate the influence of the shape variance of the cell objects.

2 The proposed method achieves consistent improvement over the comparison methods under several domain adaptation settings.

3 The overall paper is clearly presented and easy to follow.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

1 There lacks direct comparison with typical proposal-based cross-domain object detection methods. Although the authors claim these methods might be not suitable for cell segmentation due to the scale variance, it would be more convincing to make a direct comparison. Some typical proposal-based cross-domain adaptation methods include:

[a] Domain Adaptive Faster R-CNN for Object Detection in the Wild, in CVPR 2018. [b] Strong-Weak Distribution Alignment for Adaptive Object Detection, in CVPR 2019.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Code and models are not seen during the review. The details on the data split and dataset uages are clear.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

1 Please explain how the features in Fig. 4 left are obtained. Are they obtained from t-sne?

2 It would be more convincing to include the comparison with pixel-level domain adaptation methods, i.e., typical image-to-image translation methods such as CycleGAN.

3 There are some previous work focusing on cross-domain cell detection in microscopy images, please include them in the introduction, such as:

[c] Unsupervised instance segmentation in microscopy images via panoptic domain adaptation and task re-weighting, in CVPR 2020.
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The idea of using pseudo labels and uncertainty models for cross-domain cell detection is interesting in this paper. The method is demonstrated to be effective under several domain adaptation settings. Overall, I think this paper has contributed to the cross-domain analysis for microscopy images.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

8
Reviewer confidence

Confident but not absolutely certain

Review #3

Please describe the contribution of the paper

This paper proposed a self-training based method for cell detection. A Bayesian discriminator is used to select pseudo label with low uncertainty for iterative training. It evaluates the method on two conditions: unsupervised domain adaptation and semi-supervised learning. Results show it outperforms other methods.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- It proposed a novel way to incrementally utilize the pseudo heatmap for cell detection.
- Performance is good in both unsupervised domain adaptation and semi-supervised learning settings.
- The proposed self-training framework has the potential ability to be adapted to other datasets for cell/ nuclei detection/ segmentation problems.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- It lacks the ablation study to prove the effectiveness of the Bayesian discriminator.
- It lacks some implementation details which is important for reproducing: a. How to choose the mean and variance of the Gaussian distribution? b. How to prepare for the training pairs of the discriminator? c. What’s the training strategy? d. How to choose the ‘S’ in the pseudo label? e. How to generate the pseudo labelling (step 3) according to the prediction?
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Currently, the reprobucibility is poor due to the lack of details. It is better for authors to add details mentioned before.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Basically, this paper proposed a self-training based framework for cell detection. The strengthness of the method is it performes well on two settings and it has the potential ability to be adapted to other tasks. One major problem for this paper is it lacks some technical details which is important for further reproducing (list in Q4). Due to the limitation of the paper length, it is better to shorten some parts and add these details to make the method more concrete. For example, in section 2 step 3, it is not difficult for readers to understand that the proposed method uses the predictions to generate the Gaussian shape pseudo label. But how to define the centroid? Is it directly used the peak point, or some filters are added to smoother the predictions before generating the pseudo label? Therefore, it is better to shorten narrative sentences and add some implementation details.
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The proposed method shows great results in both unsupervised domain adaptation and semi-supervised learning settings. It has the potential ability to be adapted to other cell/ nuclei detection/ segmentation problems.
What is the ranking of this paper in your review stack?

3
Number of papers in your stack

6
Reviewer confidence

Confident but not absolutely certain

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

This paper received one strong accept and two borderline accepts. While reviewers gave positive comments on the technical contributions and the good experimental performance, they pointed out that the paper presentation is not clear, e.g., a clear description of the proposed method is needed (Section 2) and important technical details as well as implementation details should be provided. Please consider improving the clarity and presentation of the paper when preparing the final version.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

3

Author Feedback

We would like to thank all the reviewers for their insightful comments and positive evaluation. For example, Reviewer 1, 2(R1, R2) commented that the idea of this work is interesting. R1, R2, and R3 commented that the results demonstrate the performance of this method well under several domain adaptation settings. We would like to address the concerns raised by the reviewers as follows.

How to choose the mean and variance of the Gaussian distribution? (R3) The variance of the Gaussian distribution was determined on the basis of the size of the center region of the cell. In an experiment, we choose 6 as the variance for all datasets. In our preliminary experiments, its value was not sensitive for the performance when 6 to 9.

How to prepare for the training pairs of the discriminator? (R3) As explained in the end of section 2, in the initial iteration, we deliberately make the incorrect ground truth as negative samples from the ground truth in the source by randomly adding or removing the Gaussian distribution. When training the Bayesian discriminator on the target, we make incorrect ground truth from the pseudo label in the target in the same way as above.

What’s the training strategy? (R3) We understand your question means how to train the Bayesian discriminator. We trained the Bayesian discriminator with cross-entropy loss and dropout. It is the same with a traditional CNN for a classification task. The difference is that we use dropout in both train and test. In the test, we K times apply the CNN for a single input with the different dropout, and then we estimate the uncertainty based on the K results. In all of our experiments, we set K as 10, where it is not sensitive.

How to choose the ‘S’ in the pseudo label? (R3) We understand your question indicated about ‘SPt’ in the paper. It indicates the pseudo labels that are used for training in the next iteration. We select ‘SPt’ based on the estimated uncertainty, where we select the 10% pseudo labels with the lowest uncertainty from all predicted target pseudo labels.

How to generate pseudo labels from the prediction map? (R1, R3) In our detection, if the peak of the predicted heat map is higher than a threshold, the position is detected. As discussed in the introduction, if cells in a target have a similar appearance to that in the source, the cell can be detected even the Gaussian distribution is distorted. Therefore, we rely on the detected positions and make clear Gaussian distributions as the pseudo heat map based on the position in the same manner to making the ground truth from the human-annotated position. Even the incorrect pseudo heat maps are generated, only the confident pseudo labels are selected by the Bayesian discriminator, which will be used in training in the next iteration.

How to obtain the features in Fig 4 left? (R2) We used PCA for visualizing feature distributions of cells in the target. In detail, we extracted features of cells on target using the Bayesian discriminator trained with source.

Comparison with other object detection method (R1,R2) We will conduct the comparison with other methods in future work. We here state that such bounding box based method could not work for cell detection when the cells are densely distributed and cell has various non-rigid shapes since a bounding box contains several cell regions in such cases. For cell detection tasks, the heat-map-based detection method is one of the state-of-the-art.

The effectiveness of the Bayesian Discriminator. (R3) In a preliminary study, the estimation scores of a standard CNN were not so reliable. The selected pseudo labels contain many incorrect cell positions, and thus we introduced the Bayesian Discriminator to select the better pseudo labels. It is obvious in the related works in Bayesian discriminator, and thus we omitted these experiments due to page limit.

back to top

Cell Detection in Domain Shift Problem Using Pseudo-Cell-Position Heatmap