Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Yuexiang Li, Nanjun He, Sixiang Peng, Kai Ma, Yefeng Zheng

Abstract

Due to the inter-observer variation, the ground truth of lesion areas in pathological images is generated by majority-voting of annotations provided by different pathologists. Such a process is extremely laborious, since each pathologist needs to spend hours or even days for pixel-wise annotations. In this paper, we propose a reinforcement learning framework to automatically refine the set of annotations provided by a single pathologist based on several exemplars of ground truth. Particularly, we treat each pixel as an agent with a shared pixel-level action space. The multi-agent model observes several paired single-pathologist annotations and ground truth, and tries to customize the strategy to narrow down the gap between them with episodes of exploring. Furthermore, we integrate a discriminator to the multi-agent framework to evaluate the quality of annotation refinement. A quality reward is yielded by the discriminator to update the policy of agents. Experimental results on the publicly available Gleason 2019 dataset demonstrate the effectiveness of our reinforcement learning framework—the segmentation network trained with our refined single-pathologist annotations achieves a comparable accuracy to the one using majority-voting-based ground truth.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87237-3_47

SharedIt: https://rdcu.be/cymaZ

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

They notice the inter-observer variation problem in pathological annotations, which is an important issue. Then, they propose a reinforcement-learning based technique to refine the annotation only given by a single pathologist. Furthermore, when using refined annotations to train the network can bring performance improvements.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The paper is reasonable in structure and clear in expression.
2. The motivation of this paper is important.
3. As far as I know, it is the first time that a discriminator is proposed to the annotation refining task.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. The review of relate works is weak because there is no review of existing annotation refining methods, making it difficult to know this paper’s contribution in this field (e.g., ref.[1] and ref.[2]). In addition, the review of exemplar learning methods seems redundant due to these works are not strongly relevant to this paper.
2. Lack of comparison results of other annotation refining methods. This strengthens my concerns about the performance of the method. Reference: [1] Liao X , Li W , Xu Q , et al. Iteratively-Refined Interactive 3D Medical Image Segmentation with Multi-Agent Reinforcement Learning[J]. 2019. [2] Lee H , Jeong W K . Scribble2Label: Scribble-Supervised Cell Segmentation via Self-Generating Pseudo-Labels with Consistency[J]. 2020..
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The network structure described in this paper is clear, and there is a detailed introduction to the parameter setting and optimization process, which guarantee the reproducibility of the method proposed.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
1. It is recommended that the author delete the content related to exemplar learning, which seems to have little relationship with this work, and should be replaced with research on annotation refining methods.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Due to the lack of comparative experiments, the novelty of the proposed method is questionable . However, considering the importance of motivation in this paper, I vote to accept.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

To tackle inter-observer variation of ground-truth annotation of pathology image, this paper proposes a reinforcement learning framework based on asynchronous advantage actor-critic algorithm to automatically learn from annotations of several pathologists to refine the annotation provided by a single pathologist. The proposed method can save tremendously the time needed for annotation and can maintain a similar performance compared to those using majority-voting-based annotations.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The paper solves a clinically important problem, and the results look encouraging.
2. The proposed method is elegant, especially when compared to learning a pseudo label. Using reinforcement learning can reduce the searching space and also explore the feature extraction ability of the convolutional neural network.
3. Compared to the Pixel-based A3C algorithm, the paper further proposes a discriminator network to regularize the network training.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. Though the proposed method fits the problem, the pixel-based A3C and the discriminator network are not sufficiently novel.
2. Ablation study on the effects of the number of exemplars used should be included, as the performance of the pseudo label learning may become better increasing the training samples. At some stage, the advantage of reinforcement learning may disappear. I would like to see more discussion on this. E.g. Only if the number needed occupies a large portion of the original training data, deep reinforcement learning makes more sense.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The method is easy to follow and implement.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

The idea is quite interesting, but to really demonstrate the advantages of the proposed method, the authors are encouraged to conduct more ablation studies, as stated in section 4, the weakness of the proposed method.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
1. More ablation study is needed to really demonstrate the merits of the proposed method.
2. The idea is interesting and results are encouraging.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Review #3

Please describe the contribution of the paper

In this paper, the author proposed a multi-agent reinforcement learning framework to automatically refine the set of annotations provided by a single pathologist based on several exemplars of ground truth. Experimental results on the publicly available Gleason 2019 dataset demonstrated the effectiveness of their multi-agent framework.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The paper is well written and propose a novel application of reinforcement learning. It can be used in annotation refinement tasks.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

State in section 7.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The reproducibility of the paper is very good. The main concern is the speed of the algorithm. The author treat each pixel as a RL agent and do the multi-agent learning for thousands of agents which will be very slow to converge. It is better to compare with state-of-the-art method in both accuracy and the speed. Also if the author can provide the training details like loss or reward figures, it will be better to understand the efficiency of the algorithm.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

This paper is well organized and their idea is clear.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

State in section 7
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

3
Reviewer confidence

Confident but not absolutely certain

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

This paper was reviewed by three experts. All three reviewers consistently agree the novelty and clinically importance of the work. The AC would recommend acceptance of the work.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

3

Author Feedback

N/A

back to top

Deep Reinforcement Exemplar Learning for Annotation Refinement