Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Liang Han, Zhaozheng Yin

Abstract

Cell segmentation is a fundamental and critical step in numerous biomedical image studies. For the fully-supervised cell segmentation algorithms, although highly effective, a large quantity of high-quality training data is required, which is usually labor-intensive to produce. In this work, we formulate the unsupervised cell segmentation as a slightly under-constrained problem, and present the Unsupervised Segmentation network learning by Adversarial Reconstruction (USAR), a novel model able to train cell segmentation networks without any annotation. The key idea is to leverage adversarial learning paradigm to train the segmentation network by adversarially reconstructing the input images based on their segmentation results generated by the segmentation network. The USAR model demonstrates its promising application on training segmentation networks in an unsupervised manner, on two benchmark datasets.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87193-2_27

SharedIt: https://rdcu.be/cyhL5

Link to the code repository

https://github.com/LiangHann/USAR

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

This paper presents an unsupervised learning framework to segment cells by developing the adversarial training paradigm.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

A fully unsupervised training framework with a flexible solution by combining image decomposition and adversarial learning.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

How to ensure the sampled random vector z corresponds to the input image? If z does not correspond to the input image, then the synthesized B will not correspond to the background of the input image, and in that case all are wrong.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

likely
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

The authors should give explain why this method still works when the randomly sampled vector does correspond to the input image.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

1: the solid technical contribution 2: the good writing
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

In this paper, the authors proposed an improved redraw-net to achieve unsupervised network learning for 2-class image segmentation. The method is very similar to redraw-net where there is a decode generating images, an U-Net to perform segmentation, and a discriminator to distinguish real or fake examples.

Since this method is aiming to solve 2-class segmentation, the authors make the definition less under-constrained by assuming Z_1 = I. The author also introduced two additional components: content loss and pseudo label learning to further improve the performance.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Significant improvement over the previous unsupervised segmentation methods.
- Content loss and pseudo labeling learning are novel in this unsupervised segmentation framework.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Section 3.1 is somewhat confusing and didn’t contribute much to the method. More specifically, in the formulation, all B_i is assumed to be equal to B. But in the actual method, each B_i is generated randomly and not equal to each other. During the first pass of reading the paper, to me, it is very unnatural to not measure the loss between I_i and \hat{I}_i, but after reading the redraw-net paper, I kind of understand the motivation is not to require all the B_i to be the same, but actually make B_i unimportant so that they can be treated in the same way by the discriminator. I hope the authors could shorten this section and add more details for the content loss and pseudo label learning.
- Lack of detail for the content loss and pseudo label learning For content loss, please add the following detail: Is pre-train VGG pre-trained on ImageNet? Are image features the feature vector for all VGG layers or a specific subset? Are there any normalization after compute the l2 distance? For pseudo label learning: In the experiment, how many iterations of pseudo label learning is performed (e.g. 1)? Will more iterations of pseudo-learning help. Is the pseudo label hard PL or soft PL? What is the specific criteria for “high confident cell region”?
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Authors will share code if accepted.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
Minor comments:
- Please cite the redraw-net when refering to the “3N unknowns” in the introduction.
- Add more details on the method of selecting lambdas for each dataset. Simply mention they are selected on the validation set is not helpful for future usage of the method.
- Add more details on how U-Net results are converted to binary map M during network training (e.g. what is the threshold?)
Please state your overall opinion of the paper

borderline reject (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The proposed method is very similar to the redraw-net paper and there are not sufficient details about the key novel part of the paper (i.e. content loss and pseudo label learning).
What is the ranking of this paper in your review stack?

3
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Review #3

Please describe the contribution of the paper

This paper presents an unsupervised segmentation algorithm for foreground segmentation in simple microscope images. A segmentation network is responsible for generating the foreground mask. A generator network is responsible for generating the simple background. The original image, foreground mask and the reconstructed background are linearly combined to reconstruct the input image. The network is trained in an adversarial training setting. To prevent the trivial foreground mask (all foreground or all background), two novel regularization losses are proposed.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. An unsupervised foreground segmentation method based on adversarial training is presented;
2. Two novel regularization losses are proposed to prevent the trivial solution;
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. The formulation in 3.1 does not have connection to the proposed method. In Section 3.1, the foreground segmentation problem is formulated as a system of linear equations. It is not related to the core of the paper. It is recommended to remove such formulation.
2. The data used is too simple. It would make the method more convincing if more challenging data is used in the experiment.
3. Since the data used in the experiment is too simple, it is recommended to compare the proposed method against some traditional unsupervised methods or some thresholding based methods.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors claim to release code upon paper acceptance. In the paper, details of the implementation and training and hyperparameter settings are not discussed.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
1. The formulation in 3.1 does not have connection to the proposed method. In Section 3.1, the foreground segmentation problem is formulated as a system of linear equations. It is not related to the core of the paper. It is recommended to remove such formulation.
2. The data used is too simple. It would make the method more convincing and impactful if more challenging data is used in the experiment.
3. Since the data used in the experiment is too simple, it is recommended to compare the proposed method against some traditional unsupervised methods or some thresholding based methods.
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The proposed method is a novel unsupervised foreground segmentation method but the data used in the experiment is not challenging enough to make an impact.
What is the ranking of this paper in your review stack?

4
Number of papers in your stack

5
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

While reviewers gave positive comments on the technical novelty of the proposed method, they were concerned about the clarity and presentation of the paper. For example, Section 3.1 is confusing, and technical details for the content loss and pseudo label learning are missing. R3 also pointed out that the datasets used in the experiments are too simple and there is a lack of a comparison with traditional unsupervised approaches to verify the effectiveness of the proposed method.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

9

Author Feedback

The authors would like to express sincere thanks to the AC and reviewers for the review. We present explanations (A) to the major questions (Q) below.

Q1: How to ensure the sampled random vector z corresponds to background image (Reviewer 1). A1: Without using any supervision or prior knowledge of the dataset background information, we use a decoder to learn the dataset background image from a random vector z. The decoder alone will not generate the correct background image. Instead, the generated background, together with the generated N masks and original N images, are used to reconstruct the original images (Fig.1). Adversarial training and perceptual loss are used to guarantee the reconstructed images have the same distribution and content information with the original images, which forces the generated background image to correspond to the real background image of the dataset.

Q2: Why is the math formulation in Sec. 3.1 important? (Reviewer 2, Reviewer 3). A2: In Sec. 3.1, we formulate the unsupervised segmentation problem as a system of N linear equations (Eq.3) to solve N+1 unknowns, which is much less under-constrained. We use N images together as the model input. Each time the model segments N images simultaneously into N foreground masks and one background image.

Based on the formulation, we design our fully-unsupervised model to train cell segmentation networks, named USAR. As shown in Fig.1, we generate only one background image B for the input image set {I_1…I_N} (thus, all B_i equal to B). Besides, as described in Sec. 3.2, a segmentation network is used to generate the cell masks {M_1…M_N} for input image set {I_1…I_N}. The foreground (cell) regions are obtained by element-wisely multiplying the original images {I_1…I_N} and the generated cell masks {M_1…M_N}, and the background regions are obtained by element-wisely multiplying the generated background image B and the inverse masks {1-M_1…1-M_N}. Finally, we reconstruct the input images by combining the cell regions and the background regions with an element-wise summarization. The design of the USAR is exactly as we formulated in Sec. 3.1 (Eq.3). The much less under-constrained formulation brings us more accurate segmentation and stable model training.

The Redraw-Net [26] formulates the unsupervised segmentation as one linear equation, in which three unknown variables to be solved, thus it is severely under-constrained. Based on the formulation, Redraw-Net segments one image each time. Besides, the severe under-constraint causes inaccurate segmentation and unstable model training (i.e., model collapse mentioned in the original paper). Experiment results in Table 1 also show the superiority of our proposed model over the Redraw-Net.

Q3: Datasets and comparison with traditional unsupervised methods like thresholding (Reviewer 3). A3: For fair comparison, we use the “PhC-U373” and “DIC-HeLa” datasets in the U-Net paper. Compared to fully supervised U-Net, we attack a much more challenging problem of unsupervised segmentation network learning. In these two datasets, some cells have quite similar pixel intensities with the background, especially in the “DIC-HeLa” dataset. This makes it challenging to accurately segment the images by traditional unsupervised methods. Following the comment, we tried the Otsu thresholding on the datasets, but got unsatisfactory results (Dice of 46.95 on “PhC-U373” and Dice of 31.24 on “DIC-HeLa”), which are much worse than ours.

Q4: More details are expected (Reviewer 2). A4: The adopted VGG is pre-trained on ImageNet. All feature tensors of VGG are used to compute the content loss. We perform one iteration for pseudo label learning, and experiments show more iterations do not give us significantly better performance. We use hard pseudo labels which are in {0,1}, not [0,1]. Limited by page, we do not include all these implementation details in our paper, but all these details can be found in our codes, which will be released.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This manuscript introduces an adversarial learning-based segmentation method for cell images, which does not require any cell annotation for model training. The rebuttal has addressed the major concerns from reviewers, including clarity of method description. The reviewers also appreciate the technical contributions.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

13

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This work shows an entirely unsupervised method for cell segmentation, using adversarial learning. While of course restricted to a very specific domain of input data, the idea is interesting, well evaluated and worthwhile to present to the community. I do not agree with reviewers 2 and 3 who criticize lack of contribution and unclear formulation of the work, and this is also well addressed in the rebuttal of the authors. In case of acceptance, I encourage authors to refine their manuscript according to remaining reviewer comments and to make sure misunderstandings are ruled out.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

6

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

Reviewers have concerns over the method presentation (particularly Section 3.1), novelty and result evaluation. Comparison with other approaches has also been suggested, with comments on simplicity of the dataset. The rebuttal has provided satisfactory responses regarding the method formulation and result evaluation. The final version should address these comments with the additional results and also revise Section 3.1 to make the formulation more understandable.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

6

back to top

Unsupervised Network Learning for Cell Segmentation