Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Zhiyang Gao, Jun Shi, Jun Wang

Abstract

Convolutional neural network (CNN) has achieved superior performance on the computer-aided diagnosis for histopathological images. Although the spatial ar-rangement of cells of various types in histopathological images is an important characteristic for the diagnosis of cancers, CNN cannot explicitly capture this spatial structure information. This challenge can be overcome by constructing the graph data on histopathological images and learning the graph representation with valuable spatial correlations in the graph convolutional network (GCN). Howev-er, the current GCN models for histopathological images usually require a com-plicated preprocessing process or prior experience of node selection for graph construction. Moreover, there is a lack of learning architecture that can perform feature selection to refine features in the GCN. In this work, we propose a group quadratic graph convolutional network (GQ-GCN), which adopts CNN to extract features from histopathological images for further adaptively graph construction. In particular, the group graph convolutional network (G-GCN) is developed to implement both feature selection and compression of graph representation. In ad-dition, the quadratic operation is specifically embedded into the graph convolution to enhance the representation ability of a single neuron for complex data. The ex-perimental results on two public breast histopathological image datasets indicate the effectiveness of the proposed GQ-GCN.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87237-3_12

SharedIt: https://rdcu.be/cyl9R

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

This study presents a method that classifies histopathological images using the group quadratic graph convolution networks. Compared with many previous graph convolutional networks, this study considers feature map pixels obtained by CNN model as the graph nodes. They also design a new architecture of group graph convolution network.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The method has novelties in designing the architecture of group graph convolution network.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The major limitations for the study include: (1) many descriptions for the technique are not clear; (2) the evaluation dataset is quite small and only 5-fold cross-valuation is performed. The following are the main suggestions for authors to improve the paper.
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

for reproducibility, the authors need to make more clear explanations in the paper.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

(1) In the section 2.1 Adaptive Graph Construction shown in page 3, it is mentioned that ‘Then each pixel of feature maps is regarded as nodes embedding in graph domain and represents a corresponding patch of input image’. Can authors provide more explanations about how each pixel represents a corresponding patch of input image? What is the ‘k’ value used by k-NN in that paragraph? (2) In the Fig.2. Architecture of G-GCN shown in page 4, could authors provide more explanations why the architecture of G-GCN is performing better than other graph convolutional networks? (3) Authors should provide explanations how the output of the GCN module is flowed to pool and fully connected layer to produce the final output. How are the fully connection layers structures? (4) The evaluation images have the dimension of 2048x1536 pixels. Are these images directly input into the backbone CNN model? Have authors fine-tuned parameters in the CNN model or only used the pretrained model as the feature extractor? (5) In the Table 1, how the ResNet18 was trained for comparison? The dataset used in this study is very small, which is not enough to train the ResNet18 model.
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The method itself has novelties, but evaluations are not so convincing. more clarities should be provided in the paper.
What is the ranking of this paper in your review stack?

3
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Review #2

Please describe the contribution of the paper
1. This paper proposes a simple and effective method for graph construction.
2. This paper proposes to group the node feature of graph representation for feature fusion.
3. This paper proposes to add quadratic operation to graph convolution, which can enhance the feature extraction and representation ability.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. Splitting the node features into different groups for feature fusion is interesting and enlightening. The architecture of G-GCN is also simple and clear.
2. This paper introduces quadratic operation to replace traditional graph convolution of GCN, which is a meaningful attempt.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. The proposed method of graph construction is not explained why it can capture spatial structure feature. The edges are established by kNN using feature vector of pixel. However, the similarity of the representation does not mean spatial correlation.
2. The grouping and feature fusion operation play a role of dimensional reduction. It is good to compare with other dimensional reduction methods to reflect the effectiveness of the proposed group operations.
3. The method has not been compared with other state-of-the-art algorithms under the same benchmark or other GCN methods. This work has a certain degree of innovation, but the effectiveness of the method is not fully explained. Besides, this paper lacks an analysis of the intermediate results of the experiment.
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The architecture, parameters and dataset are very clear and detailed. This paper provides sufficient details about the models, datasets, and evaluation.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
This article uses GCN to model the high-order features extracted by CNN to improve the classification performance of the image. The author focuses on group GCN quadratic operation to achieve feature fusion and enhance feature aggregation and extraction capabilities.
- However, the method of constructing the graph is not convincing. The article needs to explain further why the similarity of the feature space can describe the spatial structure of the image.
- Besides, the experimental part of the article needs further improvement. First, the algorithm needs to be compared with the SOTA method of the two benchmarks and other similar GCN methods. Secondly, it is necessary to carry out relevant analysis experiments on the proposed module to prove the effectiveness.
Please state your overall opinion of the paper

borderline reject (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
1. The degree of innovation of the method or its inspiration to other research.
2. Whether the method perform optimally on a specific dataset.
3. Reasonable analysis of methods and experimental results.
What is the ranking of this paper in your review stack?

4
Number of papers in your stack

6
Reviewer confidence

Very confident

Review #3

Please describe the contribution of the paper

The authors present a histology image classification pipeline based on graph representations using CNN features for nodes. The graph information is then processed through a novel GCN architecture. Experimental evaluation was done on two breast cancer datasets and ablation studies show the benefits of the decisions made during the design of the pipeline.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The core of the contribution is the feature splitting and dual branch processing, together with the quadratic fusion. This novel architecture for GCN is very interesting for pathology images, where graph representations are becoming a trend.
- The authors report Accuracy, Precision, Recall and FScore for the ablation studies.
- The authors report the mean and standard deviation after 5-fold cross validation, which allows to see if the method is stable or not.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- It would have been beneficial to have a better explanation of some of the methods. For instance, the motivation for the method is to have an adaptive graph construction in contrast with cell-graph based approaches that require preprocessing to extract cell nuclei and features.
- However, since details about the adaptive graph construction are limited to one paragraph, one cannot fully understand if what the authors call an adaptive graph construction is anything else than a fixed grid graph with nodes corresponding to the field of view of the CNN block.
- If that is the case (which is certainly what it looks like, because there are no descriptions of criteria for node selection), the approach (even if it is effective for the datasets) would be not novel at all in terms of graph construction.
- The authors do not report any other methods from the literature for comparison.
- The authors do not discuss the limitations of their approach beyond the known limitation of extending to WSI images, which they propose to address through downsampling. But having shown the effectiveness of their approach at 200x and 40x it seems very optimistic to expect that level of performance if downsampling is performed.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The paper is somewhat reproducible from the text already. The datasets are public and sufficient details on the architecture are given. However, one has exactly the same cross validation the results would differ. But since the mean and standard deviation are provided, it would still be possible to compare if the reproduced results are in the correct range.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

The core contribution of the paper is more the dual branch processing and quadratic fusion than the adaptive graph generation, which judging from the text seems a fixed grid graph. If not, more details on the graph construction are absolutely needed.

For more detailed comments, see the weaknesses.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The dual branch and quadratic fusion architecture is novel enough and a good contribution. I would have preferred a more solid clinical problem than just evaluating on two generic breast cancer datasets, but l found the paper interesting to read for the rest of the community
What is the ranking of this paper in your review stack?

3
Number of papers in your stack

5
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

This study proposed a group quadratic graph convolution network for classification of histopathological images. Specifically, the authors explored to add quadratic operation to the graph convolution, which can improves the feature expression ability of the model. The authors do not report any other methods from the literature for comparison.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

8

Author Feedback

We would like to appreciate all reviewers for the positive comments. Our response to the main comments is as follows: The original image is directly input into the backbone model including the graph construction process. The reason why the constructed graph can capture the spatial structure feature of the original image is as follows: The pixels of the feature maps obtained by the CNN feature extractor are the different “receptive field” regions of the original image. It corresponds to different spatial regions of histopathological images. Edges are generated in terms of k-NN depend on the node features, which has more similar high-level semantic features corresponding to stronger correlation. In other words, the generated graph actually constructs the relationship among the regions that represent the spatial structure feature of the original image. The novelty of graph construction is that it can adaptively adjust the connection of the edges during the learning process. The nodes of graph are fixed, but the optimal feature expression of the node can be learned through the end-to-end network. Different node features lead to different connections of edges, so the final graph is different. Therefore, when this adaptive graph construction method is applied to different datasets, it adaptively constructs graph depend on the feature of different images. In addition, the whole process is automatically generated. The node selection criteria without constraint also simplify the complicated preprocessing steps compared with other graph construction methods for histopathological images, such as the extraction of cell nuclei in the cell-graph. G-GCN is proposed to obtain features with powerful category discrimination ability. It splits the node features into different groups for graph convolution. In this way, the inherent diversity of graph representations can be enhanced. The exchange and fusion of information between different groups are also achieved through the G-GCN structure. Meantime, there are gated control units in G-GCN that can selectively stimulate effective features and suppress redundant features (dimensional reduction). About the experimental part, it is extremely time-consuming and expensive to collect and annotate large amounts of medical imaging data. We use public dataset that has been used for model evaluation with many deep learning studies. The ResNet18 and the CNN feature extractor of all GCN algorithms are fine-tuned after ImageNet pre-training. Meantime, data enhancement is applied to avoid overfitting. It can train the ResNet18 and obtain a considerable result. The experiments of our proposed algorithm include the structural rationality experiment of G-GCN and the ablation experiment of each improved GCN module. Our focus is more on verifying the effectiveness of the proposed improved module. There are some misunderstandings about the analysis of the intermediate results of the network. This work is to apply GCN to graph-level classification, which pays more attention to final classification results. The analysis of the intermediate experimental results of GCN is often presented at the node classification. In addition, we think it is propriety in using 5-fold cross-validation, which is a common deep learning model evaluation method. In summary, the adaptive graph construction of our algorithm simplifies the complex graph construction process. The use of G-GCN and quadratic graph convolution enhances the graph feature expression. The experimental results show its effectiveness in this work. Some details about the experimental parameters are that the ‘k’ value in k-NN is selected as 8. The output of the last graph convolutional layer is followed by an average pooling layer and a two-layer fully connected layer. These suitable hyperparameter settings are obtained through relevant experiments. We do not give more details about it, and also cannot show more experiments and results due to the limitation of the page.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

In my opinion, this paper is very interesting, and the novelties of this work are enough for acceptance. It is a hot topic to apply GCN to learn the spatial correlation and topological features in pathological images to improve the diagnostic accuracy of a CAD model. Several pioneering works have indicated the effectiveness of the GCN-based diagnosis or prediction model on pathological images. However, these algorithms generally need sophisticated image preprocessing to construct a graph for GCN. While in this work, the authors propose a simple and effective method for graph construction by adopting the pixels in the feature maps as the nodes. A pixel in the feature maps is corresponding to a region in the original pathological image. Therefore, the constructed graph can represent the spatial correlation. Furthermore, this constructed graph is then embedded into the end-to-end network based on CNN and GCN. Therefore, the graph can be automatically updated and optimized during the process of network training, because the connection of the edges can be automatically adjusted according to the update of node features from CNN. The Review 3 think that it is a fixed grid graph. However, based on the explanation of the authors, it can be regarded as an adaptive graph construction strategy for different pathological images. I also agree with the comment of the Reviewer 2 that splitting the node features into different groups for feature fusion is interesting and enlightening. The architecture of G-GCN is also simple and clear. This strategy makes the GCN learn more effective and compact feature representation. The third novelty in GQ-GCN is that the quadratic operation is embedded into the GCN to further improve the performance of nonlinear feature learning. There are few works about the quadratic GCN, and this paper well studies the effectiveness. Although the proposed algorithm is currently developed for the ROI-level classification of pathological images, and the authors do not mention how to extend it to the WSI images, I think that the authors have answered the main questions by reviewers. I still think that this work is interesting and enlightening, which is novel enough for publication.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

2

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This paper proposes a novel method that use group quadratic graph convolution networks for histopathological image classification. They perform feature fusion on the node features, thus the node feature expression can be learned and boosted during the training. The authors have addressed most of the reviewers’ concerns. Although more theoretical analysis of the method such as graph construction should be provided for better understanding, overall, the proposed idea is interesting.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

4

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The reviewers have questions regarding method description and noted missing comparison with other methods. These would be important issues to address. The rebuttal provides some more clarify about the method design. However, the issue with lack of comparison with other SOTA or related methods still remains. This is a major issue limiting the contribution.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Reject
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

14

back to top

GQ-GCN: Group Quadratic Graph Convolutional Network for Classification of Histopathological Images