Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Zichen Wang, Jiayun Li, Zhufeng Pan, Wenyuan Li, Anthony Sisk, Huihui Ye, William Speier, Corey W. Arnold

Abstract

High resolution histology images contain information related to disease prognosis. However, survival prediction based on current clinical grading systems, which rely heavily on a pathologist’s histological assessment, has significant limitations due to the heterogeneity and complexity of tissue phenotypes. To address these challenges, we propose a deep learning framework that leverages hierarchical graph-based representations to enable more precise prediction of progression-free survival in prostate cancer patients. Unlike conventional approaches that analyze patch-based or cell-based pathomic features alone without considering their spatial connectivity, we explore multi-scale topological structures of whole slide images in an integrative context. Extensive experiments have demonstrated the effectiveness of our model for better progression prediction.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87237-3_22

SharedIt: https://rdcu.be/cymad

Link to the code repository

https://github.com/zcwang0702/HGPN

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

This paper develops a novel hierarchical graph pathomic network that integrates cell-level and patch-level graph structures for progression free survival prediction of prostate cancer. The authors also propose an efficient self-supervised learning method to pretrain the graph pathomic network, and this pretraining method demonstrates improved performance over training from scratch.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

This paper presents novel hierarchical graph-based representations of whole slide images for survival prediction, and extensive evaluations show superior performance of the proposed method over competing methods.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The impact of several key hyperparameters on model performance is not discussed.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The algorithm, datasets, and evaluation are clearly explained.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
1. The performance of tumor detection model should be present. More details should be given about selecting cells and patches to construct graphs.
2. The impact of some key parameters on the final survival prediction should be investigated, such as the number of nearest neighbors used to constrict graph.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Overall, this is an interesting and well-organized paper. The proposed hierarchical graph representation is novel and is strongly evaluated.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Review #2

Please describe the contribution of the paper

The main contribution of the paper is to integrate multi-scale graph representations and show that is a very good fit for predict progression-free survival in prostate cancer.

The paper describes a method for predicting progression-free survival in prostate cancer whole slide images using local and global graph network representations. The model learns hierarchical graph-based representations starting from nuclei features and aggregating them to represent a region in the WSI. Authors leverage the latest advances in graph-cnn models and contrastive learning to overcome the need for manually labeled regions. The model is trained with a large private dataset of ~22000 WSI and tested in the public TCGA-PRAD dataset. Results show that the model not only outperforms the standard clinical features including the Gleason grade group (c-index of 0.7254 vs 0.7934) but also state-of-the-art deep learning models using multiple instance learning and standard hand-crafted feature-based models.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The writing is clear and concise.
- The integration of the local patch graphs as nodes for global WSI representation is sound and well designed.
- The dataset used for training is large enough to have confidence in the results.
- The comparison with state-of-the-art models for survival prediction puts the results in context and highlights the importance of the method.
- The concordance index of the presented model is higher than what is found in similar works in the literature.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Size of patches used and resolution in microns per pixels is not stated.
- An ablation study on the dependency of slides used for training would have been great, even as supplementary material.
- There could have been more illustrative figures, for example showing the main modules in the end-to-end computing of image->c-index.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
- The paper results can not be reproduced since it uses a very large private dataset.
- The model can be easily re-implemented as it is explained clearly.
- The external test data is public (TCGA-PRAD), which guarantees comparison of future novel methods with this work.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- Please describe, at least briefly, what is and how is computed the c-index.
- The details of patch selection in section 2.1 are important and should be described at least in the supplementary material.
- Typo in page 5 after equation 3: “ by mapping node feature vectors to scalers”
Please state your overall opinion of the paper

strong accept (9)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The results are solid. The paper is well-written. The method is novel, despite it uses many ideas that have been around for a while, such as graph-cnn. The application is relevant and is an assesment that pathologist can not currently perform solely based on image data.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Very confident

Review #3

Please describe the contribution of the paper

The authors present a two-stage graph convolutional framework for survival prediction, which captures both cell-based and patch-based features in the hierarchical graph. A contrastive learning-based pre-trained method is used to further improve the representation ability of the network. Experiments on a public dataset show the effectiveness of the proposed method.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- A novel hierarchical graph network is proposed in this study. It contains the advantages of both the cell graph and the patch graph in previous works, which could have better capability to capture the hierarchical information in the pathological images.
- A novel extension of the contrastive learning pre-train method based on MoCo. It extends the MoCo framework on graph neural network as an unsupervised pre-train task, which shows benefit for the downstream task.
- The experiments are sufficient and the result is good. The authors conduct extensive experiments, including comparisons with conventional methods and widely used MIL based method. Feature selection strategies are also included.
- Easy to follow.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Some training detail is missing. For example, it does not include the details for the tumour detection model. Is it the same as [13] in the paper? How to finetune the model on a small dataset with coarse contour annotations is also not mentioned.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors claim to release the code after being publish. The writing is clear and well-organized. The dataset is publish available. Therefore the reproducibility should be good.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
The authors present a two-stage graph convolutional framework for survival prediction, which captures both cell-based and patch-based features in the hierarchical graph. A contrastive learning-based pre-trained method is used to further improve the representation ability of the network. Experiments on a public dataset show the effectiveness of the proposed method.

In sum, this paper is well-organized and easy to follow. The proposed hierarchical graph pathomic network is novel. It makes full use of the milti-scale topological structure information in histology image by encoding it through the multi-scale graph network. The experiments are sufficient and results outperform other general methods.

Apart from that, I have several questions:
1. It seems that the method use K-NN graph with some distance constrain. How much does the hyperparams in the graph construction affect the method?
2. Comparied with the Attention MIL-based aggregation method, how is the computational cost and the inference speed?
3. How to select the features to construct the graph?
Please state your overall opinion of the paper

strong accept (9)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This paper is well-written. The method is novel and has the potential capability to apply on other WSI analysis problems. Experiments are sufficient. Therefore I suggest to give a strong accept.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

This paper develops a novel method to integrates cell-level and patch-level graph structures for progression free survival prediction of prostate cancer. The strengths include: 1) this paper is well written and easy to follow; 2) A hierarchical graph network is proposed for survival prediction from pathology images, which has the advantages of the cell graph and the patch graph by capturing the hierarchical information in the pathological images; 3) the recent advanced techniques in GNN and contrastive learning has been integrated together to reduce the need for manually labeled regions. One key issue for this paper is the description of experimental results. The information about training details, hyperparameters, patch selection, fine tuning is missing. Consensus from Reviewers on Acceptance, but with raised questions, such as more detailed experiment settings, hyperparameters, more experiment results and so on.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

1

Author Feedback

We would like to thank all the reviewers for the positive assessment and insightful comments. In the following, we grouped multiple reviewer comments that pertain to the same issue into major categories, and provided the individual response to each one.

back to top

Hierarchical graph pathomic network for progression free survival prediction