Back to top List of papers List of papers - by topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Zeyu Gao, Jiangbo Shi, Xianli Zhang, Yang Li, Haichuan Zhang, Jialun Wu, Chunbao Wang, Deyu Meng, Chen Li
Abstract
The grade of clear cell renal cell carcinoma (ccRCC) is a critical prognostic factor, making ccRCC nuclei grading a crucial task in RCC pathology analysis. Computer-aided nuclei grading aims to improve pathologists’ work efficiency while reducing their misdiagnosis rate by automatically identifying the grades of tumor nuclei within histopathological images. Such a task requires precisely segment and accurately classify the nuclei. However, most of the existing nuclei segmentation and classification methods can not handle the inter-class similarity property of nuclei grading, thus can not be directly applied to the ccRCC grading task. In this paper, we propose a Composite High-Resolution Network for ccRCC nuclei grading. Specifically, we propose a segmentation network called W-Net that can separate the clustered nuclei. Then, we recast the fine-grained classification of nuclei to two cross-category classification tasks, based on two high-resolution feature extractors (HRFEs) which are proposed for learning these two tasks. The two HRFEs share the same backbone encoder with W-Net by a composite connection so that meaningful features for the segmentation task can be inherited for the classification task. Last, a head-fusion block is applied to generate the predicted label of each nucleus. Furthermore, we introduce a dataset for ccRCC nuclei grading, containing 1000 image patches with 70945 annotated nuclei. We demonstrate that our proposed method achieves state-of-the-art performance compared to existing methods on this large ccRCC grading dataset.
Link to paper
DOI: https://doi.org/10.1007/978-3-030-87237-3_13
SharedIt: https://rdcu.be/cyl9S
Link to the code repository
https://github.com/ZeyuGaoAi/Composite_High_Resolution_Network
Link to the dataset(s)
https://dataset.chenli.group/home/ccrcc-grading
Reviews
Review #1
- Please describe the contribution of the paper
The paper presents a deep learning method for segmentation and classification of nuclei in ccRCC. An integrated segmentation and classification network is designed. The method is evaluated on a private dataset and shows improved performance over other well-known methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Method description is quite clear.
- Overall network architecture is new.
- Evaluation shows improved performance over U-Net, Mask R-CNN, etc.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Dataset is private, and hence there can only be very limited follow-up study on similar problem domains.
- The method involves some post-processing steps. It’s not clear how important these steps are. What if similar post-processing is applied to U-Net or Mask R-CNN? Will they obtain better results?
- There are other more advanced instance or panoptic segmentation methods designed specifically for nucleus segmentation. It would be good to see a direct comparison with those methods.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Reasonable level of implementation details has been provided in the paper.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
See above.
- Please state your overall opinion of the paper
borderline accept (6)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
There is some novelty in the network design, especially for the segmentation part, and the way of linking segmentation with classification. However, with a private dataset, there would be limited interest in this work.
- What is the ranking of this paper in your review stack?
2
- Number of papers in your stack
5
- Reviewer confidence
Very confident
Review #2
- Please describe the contribution of the paper
In this paper, the author proposed a deep-learning framework customized for Clear Cell Renal Cell Carcinoma Nuclei Grading. A two-stage network that can solve touch cell problem is proposed for nuclei instance segmentation. Nuclei classification is conducted in two different resolution scales and then fused for the final result.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Method has been compared with other approaches and the best result has been achieved by the proposed method. Code and data will be released upon publication.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Nuclei classification score is not impressive. Some details of method is missing.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Code and data will be released upon publication.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
HRFE net predicting nuclei class by taking the auxiliary feature maps as input. However, I cannot find details of how this auxiliary feature map is generated. It is desired to show some examples of nuclei of different grade to help understand the task. The author mentioned: “However, the grade 2 tumor nuclei is almost same as grade 1 at 100x magnification, as grade 3 at 400x magnification.” It is desired to show examples in 100x and 400x for comparison. The author visually showed prediction examples in the supplemental materials. It is desired to include some in the main text. If space not allowed, the author can at least show some success and failed examples of proposed method. In table 1 and 2, Dice/AJI/PQ can be separated with PQs by a vertical line as their focus are different: instance segmentation VS nuclei grading. Table 1 and 2 can be combined to make more room for example figures.
- Please state your overall opinion of the paper
accept (8)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This is a typical MICCAI paper: optimize methods for a specific clinical problem. Best performance achieved by the proposed method.
- What is the ranking of this paper in your review stack?
2
- Number of papers in your stack
8
- Reviewer confidence
Very confident
Review #3
- Please describe the contribution of the paper
This paper proposes a novel nuclei segmentation and classification method aimed for nuclei grading of clear cell renal carcinoma (ccRCC). Also, a corresponding dataset of ccRCCC with 70945 fine-graded labels (grades 1, 2 and 3 and endothelial nuclei) is contributed. Data and code are available which is important for reproducibility. Proposed method outperformed four competitors.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The main strength of this paper is a two-stage learning framework W-Net to generate nuclei distance map from the image patch. In the first stage U-Net generates binary map. In the second stage a lightweight U-Net is used to extract the shape and distance-related features from the generated binary maps, yielding the distance map. Another main strength is two cross-category classification system for generating aggregated probability map. The crucial part of this system is high resolution feature extractor (HRFE) comprised of parallel high-to-low resolution convolution network with three multi-resolution streams. It is shown in the ablation study that multi-resolution streams in combination with the two-stage segmentation network is crucial for obtaining good segmentation and classification results. Also, a corresponding dataset of ccRCCC with 70945 fine-graded labels (grades 1, 2 and 3 and endothelial nuclei) is contributed.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Instead of U-Net it would be more beneficial to use the DeepLabv3 network. Through the mechanism known as a-trous convolution this network preserves high-resolution feature map.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
. Data and code are available which is important for reproducibility.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
Instead of U-Net it would be more beneficial to use the DeepLabv3 network. Through the mechanism known as a-trous convolution this network preserves high-resolution feature map.
- Please state your overall opinion of the paper
strong accept (9)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Proposed nuclei segmentation and classification method aimed for nuclei grading of clear cell renal carcinoma (ccRCC) contains three important contributions: 1) a two-stage learning framework W-Net to generate nuclei distance map from the image patch. b) two cross-category classification system for generating aggregated probability map. The crucial part of this system is high resolution feature extractor (HRFE) comprised of parallel high-to-low resolution convolution network with three multi-resolution streams. c) a corresponding dataset of ccRCCC with 70945 fine-graded labels (grades 1, 2 and 3 and endothelial nuclei) is contributed.
- What is the ranking of this paper in your review stack?
2
- Number of papers in your stack
5
- Reviewer confidence
Confident but not absolutely certain
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This well-written paper proposed a novel composite high-resolution network based framework for nuclei segmentation and classification to support ccRCC nuclei grading, demonstrating promising experiment results. All reviewers are positive and in favor of accepting the paper.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).
1
Author Feedback
Thank you for your time and constructive comments, and we glad to have this opportunity to discuss our work.
Reviewer 1： Question 4:
Dataset is private, and hence there can only be very limited follow-up study on similar problem domains. Due to the blind review mechanism of the conference, our dataset and code are not open0source yet. However, it will be made public and a download link will be provided soon.
The method involves some post-processing steps. It’s not clear how important these steps are. What if similar post-processing is applied to U-Net or Mask R-CNN? Will they obtain better results? The ablation experiments in this paper verify the effect of post-processing on the segmentation performance of the model. As shown in Table 2, after the UDist and WDist operations are introduced on MHR, the performance of the model has been improved to varying degrees.
There are other more advanced instance or panoptic segmentation methods designed specifically for nucleus segmentation. It would be good to see a direct comparison with those methods. As far as we know, prior to this work, Hover-Net was a representative work to solve the problem of panoptic nuclei segmentation, and it obtained the experimental results of state of the art. At the same time, we also compared classic segmentation algorithms such as U-net and Mask-RCNN in the comparative experiment in Table 1 to prove the superiority of the proposed method.
Reviewer 2: Question 7: -HRFE net predicting nuclei class by taking the auxiliary feature maps as input. However, I cannot find details of how this auxiliary feature map is generated. From Fig.2., it can be seen intuitively that the input of the HRFE module is generated after the two original images of 100x and 400x are extracted by a res-block.
-It is desired to show some examples of nuclei of different grades to help understand the task. The author mentioned: “However, the grade 2 tumor nuclei are almost same as grade 1 at 100x magnification, as grade 3 at 400x magnification.” It is desired to show examples in 100x and 400x for comparison. Due to space limitations, we did not include example images of different resolutions (100x and 400x) and grading result images in the paper, which are covered in the supplementary materials.