Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Zhi Wang, Xiaoya Zhu, Lei Su, Gang Meng, Junsheng Zhang, Ao Li, Minghui Wang

Abstract

Robust nuclei detection is crucial prerequisite for histologic characteristics of nuclei that can assist various clinical tasks such as disease diagnosis and cancer grading. Despite of their success, most existing nuclei detection methods ignore the case where the testing (target) domain has different data distribution with the training (source) domain, which is known as the problem of domain shift. In fact, the domain shift problem is prevalent in histopathology images due to various reasons such as different staining procedures and organ specific nuclear morphology. Thus, the performance of a nuclei detection model in the source domain will be hurt if it is directly applied to the target domain. To address this problem, we propose a novel instance-aware domain adaption framework for nuclei detection in histopathology images, which includes both image-level alignment (IMA) and instance-level alignment (INA) components to minimize the domain shift. Especially, INA component extracts instance-level features by using nuclei locations as the guidance and effectively aligns the instance-level features via adversarial training. Furthermore, to facilitate instance-level feature alignment, a Temporal Ensembling based Nuclei Localization (TENL) module is introduced in INA component to automatically generate candidate nuclei locations in the target domain. We evaluate the proposed method on different benchmark settings and obtain remarkable improvements compared to existing methods on the challenging problem of cross-domain cell nuclei detection.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87237-3_48

SharedIt: https://rdcu.be/cyma0

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

In this work, the authors proposed a model for cross-domain nuclei detection in histopathology images, which align the features at the global image level and local instance levels. Extensive experiments on two domain adaptation settings indicate the effectiveness of the proposed method.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

1 The study for cross-domain nuclei detection is important since it can broaden the applicability in real clinics.

2 The overall paper is clearly presented and easy to follow.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

1 The overall paper lacks technique novelty, which is a simplified version of [6]. Similar to [6], this paper also proposes to align the features for global and local features. In addition, I think the contributions of this paper are less than [6]. In [6], the model can work on cross-domain detection for the location bounding box of each object, while this paper can only predict the centroid of them.

2 The experimental settings are questionable and the results are not convincing. This paper aims at solving the cross-domain object detection tasks in histopathology images, however, there lacks direct comparison with such methods, such as [6]. Although there are comparison results with [7, 16], the comparison setting is questionable, please refer to part 7 for details.

3 There lacks visualization comparison results.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The code and pre-trianed models are not seen for review. The data split details are clear.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

1 [7, 16] are two different methods, how they have the same results as Oracle in Table 1?

2 [7, 16] are also cross-domain object detection methods, why they are indicated as fully supervised manners in Section 3.3?

3 As illustrated in the method, your detection model is from [20]. Thus I think the fully supervised Oracle in Table 1 should be this model fully trained on the target images.

4 During local-level feature alignment, the width, and height for all the objects are set to 9. I think this setting will overlook the size variation issues for the nuclei objects in the histopathology images.
Please state your overall opinion of the paper

probably reject (4)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Given the lack of novelty and questionable experimental results, the reviewer proposes to reject this paper.
What is the ranking of this paper in your review stack?

8
Number of papers in your stack

8
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

The paper introduces instance-aware domain adaptation for cell nuclei detection in histopathology images. The proposed approach attempts to overcome the limitations of image-level alignment by aligning patches cropped from nuclei locations. The instance-level alignment, together with image-level alignment and temporal ensembling outperforms other domain adaptation baselines. The author also uses ablation studies to confirm the effectiveness of different components.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

(1) The proposed instance-level alignment for histopathology images is novel, intuitive, and addresses the limitation of image-level alignment. I believe this is an important contribution to cross-domain nuclei detection tasks. (2) The proposed approach is evaluated on two domain pairs and the ablation studies demonstrate its effectiveness.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

(1) The lack of strong feature-based domain adaptation baselines. The paper uses ADDA (Tzeng et al., 2017) to represent feature-wise domain adaptation methods, but I do not think it is representative enough due to some well-known limitations—the discriminative source training could make it difficult to adapt from depending on the dataset. The authors should consider more recent SOTA approaches such as [1]. (2) The impact of target domain accuracy on adaptation performance. In the early training epoches before the target domain reaches adequate accuracy, does less-accurate target instances hamper the performance of adaptation and accumulate these errors during training? It is also clear how are target instances created, perhaps at every training step/epoch, and does it incur additional computational cost?

[1] Bridging Theory and Algorithm for Domain Adaptation, ICML2019
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The paper provides training hyperparameters, but did not provide the code. Unsure if this paper will be easily reproducible.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Method

instance-level alignment vs. patch-level alignment

The authors should clarify why the proposed approach is named “instance-level alignment,” “instances” are not well-defined in this paper—any patch or patches only containing nuclei? The authors should also clarify what is the difference between instance-level alignment vs. patch-level alignment [ICCV2019].

visualizations of instances

I think Fig. 2 could be improved by adding visualizations of instances for the source and target domains.

limitations of the proposed approach

The authors should comment on the potential limitations of the proposed approach. For example, what is the impact of false positives on the target instances and how prevalent are the false positives. Its computational cost.

grammar

There are quite a few grammatical issues in this manuscript, e.g., missing “the.” The authors should carefully edit this paper for its final version.

[ICCV2019] Domain Adaptation for Structured Output via Discriminative Patch Representations
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Although the idea of patch-level domain adaptation has been done in image segmentation tasks, I think the proposed instance-aware domain alignment is an important contribution to the miccai community.
What is the ranking of this paper in your review stack?

3
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Review #3

Please describe the contribution of the paper

The paper describes a method for cell nuclei detection in histopathology images with an emphasis on solving the domain shift problem which is a common issue in histologic image analysis. The proposed detection approach mainly consists of two 4 blocks: 1) detection models for the source and target domain 2) instance level alignment module (INA) which adapts instances in the source and target domain 3) Image level alignment module (IMA) which adapt images in the source and a target domain and 4) Temporal ensembling based nuclei localization module (TENL) that performers nuclei detection for the target domain. The proposed detection method for domain adaption was applied on two datasets and promising results were achieved.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper aims to solve a problem that is a common issue in analysing histological images
- The proposed approach is novel and technically sound
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- There is no explanation in the paper why selected hyperparameters were chosen.
- There is no explanation for discriminator architecture and details which makes it rather difficult to follow the paper for that specific module.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

I think it would be difficult to re-design the model and achieve the same results without access to the development codes.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- While the integration of the proposed INA module in the model shows promising results for nuclei detection, it would be interesting to see if it would be also helpful for nuclei instance segmentation. As it is normal that nuclei shape and morphology may be different from organ to organ, aligning them for nuclei instance segmentation may not be an optimal solution. This can be discussed in the paper.
- in figure 2, the output images do not match with the input images which is a bit confusing.
- The detection model proposed in [20] was used in the paper which is based on regression. However, the proposed formula in (1) is a segmentation loss function. Please add an explanation for this in the manuscript.
- While the ablation experiments are rather complete, it would be interesting to see the results in the case of using only INA + TENL
- In one of the experiments, the magnification of source and target domain are different (20x and 40x). Is there any resizing applied as pre-processing?
- Please add the discrimination architecture and details as supplementary materials
- in the final set up,h and w were set to 9 but is it really big enough to cover the whole nuclei especially in case of the existence of large nuclei in the images?
- it is strongly recommended to release the implementation code on a public platform such as GitHub
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The approach is novel and the topic is very interesting for researchers in the field of histological image analysis.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

This manuscript received diverging scores. R2 and R4 appreciated the technical novelty of the proposed method, but R1 pointed out that the method is similar to [6]. Please clarify this in the rebuttal. In addition, reviewers raised concerns about experimental setup (e.g., selection of the Oracle model [7,16] is questionable in Table 1), comparison with other competitors (e.g., not compared with more recent state-of-the-art domain adaptation methods), and presentation of the method/paper (e.g., “instances” are not well-defined, the effects of less-accurate target instances are not clear, how the height and width of the local region affect the adaptation performance is not well explained, etc.).
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

7

Author Feedback

We thank all the reviewers for valuable comments. We’ll revise the paper accordingly in the final version.

Novelty (R1): Our method has significant differences with [6]: 1) Unlike our approach, [6] is a cross-domain object detection method that is based on Faster R-CNN and requires bounding box annotations for training. Therefore, [6] cannot be directly applied to cross-domain nuclei detection task for which bounding box annotations are usually unavailable. 2) Instance-level feature extraction in [6] depends on region proposals that are infeasible in nuclei detection [2]. In contrast, we specifically design an effective procedure for extracting instance-level features from nuclei by using their centroid coordinates as guidance. 3) More importantly, due to the small size of nuclei, location information of candidate nuclei in target domain is usually noisy, which largely hampers instance-level alignment in cross-domain nuclei detection. To address this issue, unlike [6] that only uses predicted location information from single training epoch, we tailor an efficient TENL module that ensembles multiple previous predictions of detection model for robust localization of candidate nuclei in target domain, thus leading to more powerful instance-level alignment for nuclei detection.

Experimental setup (R1,R2): 1) Apologies for being unclear in Oracle model of our paper. We cite references [7,16] to explain that Oracle stands for fully supervised results on the target domain. In fact, the fully supervised Oracle in Table 1 is obtained by our proposed detection model instead of the models in [7,16]. 2) We add the method in [6] as a strong feature-based domain adaptation baseline. We compare domain adaptation components in [6] that take instance-level features extracted by our method as input, since region proposals used in [6] are not applicable to our datasets. ———CoNSeP-to-CRCHisto, CoNSeP-to-BCNuP ——Precision,Recall,F1 score,Precision,Recall,F1 score —-[6] 0.712, 0.811, 0.758, 0.685, 0.796, 0.736 -Ours 0.721, 0.872, 0.789, 0.730, 0.812, 0.769 (— added for formatting)

How the height and width of the local region affect the adaptation performance(R1,R4): We add an experiment to assess the impact of the height and width of the local region on CRCHisto dataset. The best overall performance in target domain can be achieved when the height and width are both set to 9. Smaller height and width may lead to insufficient discriminative information for instance-level alignment, and therefore the performance is not satisfactory in target domain. Using height and width larger than 9 shows no substantial benefit and a too large region results in decreased performance probably due to the background noises. -HeightⅹWidth Precision,Recall,F1 score —————-5ⅹ5 0.708, 0.859, 0.777 —————-9ⅹ9 0.721, 0.872, 0.789 ————–13ⅹ13 0.721, 0.867, 0.787 ————–17ⅹ17 0.720, 0.863, 0.785 ————–21ⅹ21 0.704, 0.863, 0.775 (— added for formatting)

The effects of less-accurate target instances in the early training epoch (R2): To prevent the instance-level alignment suffering from less-accurate candidate nuclei locations in the early training epoch, we do not perform instance-level alignment until the candidate nuclei locations are accurate enough. Specifically, in the first 20 training epochs we only conduct image-level alignment, and the detection model learns to localize candidate nuclei in target domain. During the following training epochs, by leveraging our proposed TENL module to further ensure robust and accurate localization of candidate nuclei, instance-level alignment is performed along with image-level alignment for successful cross-domain nuclei detection.

“instances” are not well-defined(R2): Here instances stand for nuclei and instance-level features refer to the features extracted from local nuclei regions.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The proposed method conducts image-level and instance-level feature alignment for cross-domain nuclei detection, using adversarial learning and temporal ensembling. The rebuttal has addressed the major concerns, e.g., technical novelty, comparison with other methods, and presentation of the method/paper.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

10

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

Reviewers had concerns over novelty and performance comparison. These issues should be addressed. The rebuttal has provided reasonable responses. Novelty in terms of centroid-based detection is reasonable. Additional results have been provided and should be included in the final version.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

4

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

Although the proposed method shares similarity with [6], the application to MICCAI field can be interesting. However, the ‘instance-aware’ may be misleading since the authors extracted fixed-size patches since instance sizes are important for recognition. Overall, I am inclined to accept it favourably given the interesting application to the research community.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

5

back to top

Instance-aware Feature Alignment for Cross-domain Cell Nuclei Detection in Histopathology Images