Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Valentin Anklin, Pushpak Pati, Guillaume Jaume, Behzad Bozorgtabar, Antonio Foncubierta-Rodriguez, Jean-Philippe Thiran, Mathilde Sibony, Maria Gabrani, Orcun Goksel

Abstract

Segmenting histology images into diagnostically relevant regions is imperative to support timely and reliable decisions by pathologists. To this end, computer-aided techniques have been proposed to delineate relevant regions in scanned histology slides. However, the techniques necessitate task-specific large datasets of annotated pixels, which is tedious, time-consuming, expensive, and infeasible to acquire for many histology tasks. Thus, weakly-supervised semantic segmentation techniques are proposed to utilize weak supervision that is cheaper and quicker to acquire. In this paper, we propose SegGini, a weakly supervised segmentation method using graphs, that can utilize weak multiplex annotations, ie inexact and incomplete annotations, to segment arbitrary and large images, scaling from tissue microarray (TMA) to whole slide image (WSI). Formally, SegGini constructs a tissue-graph representation for an input histology image, where the graph nodes depict tissue regions. Then, it performs weakly-supervised segmentation via node classification by using inexact image-level labels, incomplete scribbles, or both. We evaluated SegGini on two public prostate cancer datasets containing TMAs and WSIs. Our method achieved state-of-the-art segmentation performance on both datasets for various annotation settings while being comparable to pathologist-baselines.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87196-3_59

SharedIt: https://rdcu.be/cyl3a

Link to the code repository

https://github.com/histocartography/seg-gini

Link to the dataset(s)

https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/RRKMHC

https://data.mendeley.com/datasets/9xxm58dvs3/2

Reviews

Review #1

Please describe the contribution of the paper

SEGGINI (SEGmentation method using Graphs from Inexact and Incomplete labels) is proposed for histology image segmentation via weakly supervised segmentation method utilizing inexact labels, incomplete labels or both. Graph Neural Network (GNN) learns contexturelized features for graph nodes, and then two branches, Graph-head and Node-head, are applied to weakly supervise the segmentation training.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. A new framework is proposed to utilize multiple different kinds of weak supersion for histology tissue segmentation.
2. The framework makes use of both local and global tissue information for the modeling process.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. The time consumption of the framework is a big concern, particularly for large whole slide images.
2. Since each WSI is taken as one input for the proposed framework. Because of limited cases (<=1,000) for both datasets and the weakly segmentation, the model training seems to be chanllenging.
3. Visulization results of the UZH is far from ground truth.
4. Whether this framework can apply to other tissues, organs, and datasets is a doubt.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The framework looks time-consuming, and the weakly superived training seems challenging. The reproducibility of the proposed method is doubtful.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
1. The experiemented TMAs and slides are far smaller than most whole slide images. Though the framework looks appealing, the applicability of the framework is a big concern.
2. The weakly supervised segmentation is a challenge task. Current study achieves superior performance than compared ones, but the performance is still far from applicable for downstream analysis.
Please state your overall opinion of the paper

borderline reject (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
1. The training/applicability of the proposed framework is a big concern.
2. The performance seems far from usability.
What is the ranking of this paper in your review stack?

3
Number of papers in your stack

5
Reviewer confidence

Somewhat confident

Review #2

Please describe the contribution of the paper

SegGini is applied to arbitrary image sizes and learned from weak multiplex supervision. It can also integrate local and global inter-tissue-region relations to build contextualized segmentation.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The model consists of tissue-graph construction and contextualization, graph head, and node head by weakly supervised semantic segmentation to boost performance. In the stage of graph-head, the author designs a graph classification and a feature attribution module, it is an interesting way to adopt GRAPHGRAD-CAM to determine the node labels.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The model consists of tissue-graph construction and contextualization, graph head, and node head by weakly supervised semantic segmentation to boost performance. In the stage of graph-head, the author designs a graph classification and a feature attribution module, it is an interesting way to adopt GRAPHGRAD-CAM to determine the node labels.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

It could be.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

First of all, the framework should be optimized, which is hard to follow. The method is not easy to understand and more quations or formulations rather than a bunch of statement should be helpful. The discussion of experiment section did not clarify the contribution in well structure, which should be further enhanced.
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

See the comment part.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

4
Reviewer confidence

Confident but not absolutely certain

Review #3

Please describe the contribution of the paper

The authors propose a weakly supervised approach for WSI segmentation which utilizes incomplete annotations and incomplete labels. Tissue graph representations from histology images are used as input to GNN to generate node embeddings. Two networks - 1 graph classification and 1 node classification networks are jointly trained on the node embeddings to improve the segmentation performance. The method has been validated on UZH and SICAPv2 datasets containing TMA and WSIs respectively.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The methodology is quite interesting as it utilizes inexact supervision to segment large images. Both local and global node-to-node i.e. tissue-to tissue relations have been contextualized using separate node classification and graph classification networks. The method has been robustly evaluated in different percentage settings of incomplete annotations and labels.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

1) Some predicted segmentations can be illustrated on the WSI images, instead of only showing maps, for visual explainability.

2) In ablation study, experiments can be done to show individual importance of Graph head and Node head networks - global and local context

3) One of the motivations for this approach is the often observed ambiguous boundaries among various tissue subcompartments. This, however, has not been adequately demonstrated.

4) What are the morphological and spatial features?
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The checklist mentions that analysis of failure examples have been provided. Did the authors miss this?

The authors mention that all code and models will be made available upon acceptance.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

1) If possible, the method can be compared against the framework proposed in Graham et. al 2) To demonstrate generalizability, the pipeline could have been robustly verified on at least one other dataset from another domain 3) How does the method perform for relatively easier tasks, such as cancer vs no-cancer?

Graham, Simon, Quoc Dang Vu, Shan E. Ahmed Raza, Ayesha Azam, Yee Wah Tsang, Jin Tae Kwak, and Nasir Rajpoot. “Hover-net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images.” Medical Image Analysis 58 (2019): 101563.
Please state your overall opinion of the paper

borderline reject (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The generalizability of the method is not clear from the presented evaluation.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

3
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

The paper propose a graph neural network method for histology image segmentation with various types of weak supervision (inexact and incomplete).

Reviewers generally considered the proposed method valid and interesting. However, they raised questions regarding the training efficiency, generalizability, result quality, and presentation clarity.

We invite the authors to provide rebuttal and to address these concerns.

Note the authors may not submit additional experimental results in the rebuttal. The rebuttal is only meant for clarification purpose.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

7

Author Feedback

We thank the reviewers, whose major concerns we group & address below. To reiterate, our contribution is a novel method that can learn segmentations from multiple types of weak supervision & can scale to WSIs.

R1:Used images smaller than typical: We disagree. SegGini is applied on public datasets including typical dimensions: TMA: 3Kx3K at 40x & WSI: 11Kx11K at 10x, as we reported. NOTE: SegGini using graphs is seamlessly scalable to arbitrary image sizes (a main contribution).

R1:“The performance seems far from usability” We disagree. On the contrary, SegGini’s cross-validated segmentation & downstream Gleason Grading performances are superior to its alternatives & comparable to inter-pathologist agreement on both datasets, indicating its feasibility for clinical practice. Indeed, achieving this high performance by using only weak supervision demonstrates the ability of SegGini to learn from readily available clinical reports without extra pathological annotations.

R1:“Visualization results of the UZH are far from ground truth” There may be a potential misunderstanding: avg. Dice for the sample is C: 64.6%, IE: 24%, IE+IC(10%): 64.5%. Clearly SegGini using multiple weak-supervisions is comparable to exhaustive complete supervision. These results are also far superior to an independent pathologist’s annotation of the same sample (avg. Dice of 37%, to be reported), demonstrating the large inter-pathologist variability.

R1:Processing time: A forward pass (inference) for processing a WSI (11Kx11K at 10x) on Nvidia P100 GPU is SegGini: 14ms, CLAM: 11ms, HistoSegNet: 8min (with post-processing). Patch feature extraction takes 10min: the most time-consuming step, also common across all the methods. Superpixel extraction & graph building take 28s on CPU. It can be optimized using GPU, e.g. Superpixel Sampling Network, Jampani et al, ECCV, 2018 (not a focus of our work). NOTE: this extra step (28s/10min~5%) is crucial for segmentation which is not possible with CLAM.

R1:May be challenging to train with limited cases: We mentioned node-level augmentation, i.e, randomly augmenting fraction of nodes in a graph, to effectively boost the number of training graphs.

Generalizability: -R3. to other datasets: We already show generalizability by evaluating 2 datasets with different tissue acquisitions (prostatectomy/needle), preparation, scanners & dimensions. Further results would go beyond the page limit. Nonetheless, there is no reason why our results would not generalize to other datasets. -R3. to binary classes: As SegGini provides better performance for complex & ambiguous Gleason patterns, applying to binary classes will be easier & provide similar performance gains. -R1. to other organs, tissue, or tasks: SegGini doesn’t use any organ, tissue, dataset or task-specific prior or post-processing. Utilizing generic components, i.e. image-to-graph translation & GNN, SegGini is agnostic to these data choices & thus widely applicable; e.g. similar components were used in HACT-Net, 2020 MICCAI W-GRAIL, for breast tumor classification.

Presentation clarity: -Agreeing with R2, we will simplify the Methods & Results. Given the page limit, we included only the most relevant equations. -Agreeing with R3 about Fig3: we will add WSIs as overlay, & zoom in ambiguous boundaries to show the benefit of graphs for tissue contextualization. -R3: Morphological & spatial features are already described in Page 4 Para 1. -R3: Individual importance of Graph-head & Node-head is presented as IE & IC settings respectively in Tables 1 & 2. -R3: Failed examples are shown in Supplementary Fig2; failed IE & corrected IE+IC -R3: SegGini cannot be compared with HoVer-Net, which is a nuclei segmentation method with nuclei specific horizontal & vertical prior that is not applicable in tumor segmentation.

R1, R3:Reproducibility: We already detailed tissue-graph parameters, model architecture & training parameters. We will open-source the code & checkpoints.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This paper seems to be addressing a very good problem: inexact (noisy) and incomplete (weakly supervised) labels in semantic segmentation. Unfortunately, none of the reviewers were excited about the paper. I would attribute this to a lack of appreciation of the novelty and intuition of the paper. The rebuttal did not alleviate the concern.

I think the presentation is partially the reason. The current presentation is too technical; detailed description of a giant system with equal focus on all components. The presentation can be significantly improved by focusing on how the method attempt to address noisy label and incomplete labels early on and explain how the network is designed to solve the challenges. Note that even though R2 is the only one complained about the presentation, he/she was the one giving the highest score.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Reject
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

15

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

Reviewers have concerns over method description, more ablation studies, generalisability of the method, training time and difficulty and visual results. These are important issues to be addressed. The rebuttal has provided some clear responses. Overall the method has some novelty and demonstrates good results. Main issue is clarity and visual result presentation. These should be addressed in the final version.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

9

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The paper propose a graph neural network method for histology image segmentation with inexact and incomplete annotations as weak supervision. Like reviewers and primary AC, I also consider the proposed method valid and interesting. It targets on a very common problem in computational pathology. In my opinion, authors also did a good job in addressing the major concerns of the reviewer in the rebuttal, especially the speed of the network and the reproducibility. I feel the paper can be accepted.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

8

back to top

Learning Whole-Slide Segmentation from Inexact and Incomplete Labels using Tissue Graphs