Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Chen Liu, Jinze Cui, Dailin Gan, Guosheng Yin

Abstract

Coronavirus disease 2019 (COVID-19), the pandemic that is spreading fast globally, has caused over 181 million confirmed cases. Apart from the reverse transcription polymerase chain reaction (RT-PCR), the chest computed tomography (CT) is viewed as a standard and effective tool for disease diagnosis and progression monitoring. We propose a diagnosis and prognosis model based on graph convolutional networks (GCNs). The chest CT scan of a patient, typically involving hundreds of sectional images in a sequential order, is formulated as a densely connected weighted graph. A novel distance aware pooling is proposed to abstract the node information hierarchically, which is robust and efficient for such densely connected graphs. Our method, combining GCNs and distance aware pooling, can integrate the information from all slices in the chest CT scans for optimal decision making, which leads to the state-of-the-art accuracy in the COVID-19 diagnosis and prognosis. With less than 1% of the total number of parameters in the baseline 3D ResNet model, our method achieves 94.8% accuracy for diagnosis, which represents a 2.4% improvement over the baseline on the same dataset. In addition, we can localize the most informative slices with disease lesions for COVID-19 within a large sequence of chest CT images. The proposed model can produce visual explanations for the diagnosis and prognosis, making the decision more transparent and explainable, while RT-PCR only leads to the test result with no prognosis information. The prognosis analysis can help hospitals or clinical centers designate medical resources more efficiently and better support clinicians to determine the proper clinical treatment.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87234-2_27

SharedIt: https://rdcu.be/cyl8n

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The author proposes a diagnoses and prognosis model based on the graph convolutional network. On the basis of the assumption that the relationship between slices of CTs is more important than the order of such CTs, the author treats each slice as a single node and model their relationship as a complete graph. A novel distance aware pooling is used to reduce the size of graph and extracts high-level features. An MLP layer is used to do multitask output and result visualization.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The investigated problem is important and practical.
2. The proposed achieves much higher performance than the best baseline.
3. Combined with the GCN, the model learns features of CTs from the total view instead of a local view.
4. With the help of using the novel pooling layer, less parameters are used and size of the graph is reduced prominently. The model infers faster than the baselines.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. The author claims that “a COVID-19 positive patient develops into several/critical”. However, the labels in the collected data do not support such analysis. As each graph gets only one class of label.
2. As the author claims, reception field is transferred from CNN into GCN. The more layers used in GCN, the larger the reception field the model get. As a result, the number of layers is a critical important hyperparameter which needs to do experiment to show the balance of performance and training time.
3. Only has one baseline, which makes experiment results less reliable.
4. The result of the baseline is not provided in a figure or a table. The results in table 1 can’t reflect the advantages of the proposed method over with the baseline, a 3D ResNet-18 classification network.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The paper is not open sourced. But the description can help to reproduce.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
1. Fig 1. needs to modified as not only two layers of GCN is used.
2. In section 2.1, “whether a COVID-19 positive patient develops into sever/critical illness status” should be ”several/critical illness status”, from my perspective.
3. A spare “k” appears at the end of section 2.
4. It should be “Related Work” instead of “Related Word” for the title of section A in Supplement Material.
Please state your overall opinion of the paper

probably reject (4)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

About the models: The author claims that “a COVID-19 positive patient develops into several/critical”. However, the labels in the collected data do not support such analysis. As each graph gets only one class of label.

About the experiments: As the author claims, reception field is transferred from CNN into GCN. The more layers used in GCN, the larger the reception field the model get. As a result, the number of layers is a critical important hyperparameter which needs to do experiment to show the balance of performance and training time. Only has one baseline, which makes experiment results less reliable.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

2
Reviewer confidence

Confident but not absolutely certain

Review #2

Please describe the contribution of the paper
1. This paper utilizes GCNs to extract sequence information of CT images, and it is of great significance for the diagnosis and prognosis of COVID-19.
2. In this paper, a novel pooling method called distance aware pooling combined with GCNs is proposed to better extract and integrate graph features.
3. The proposed method is said to be able to locate the slices with the most information, thereby reducing the workload of the physician during subsequent diagnostic procedures.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The proposed method mainly uses the GCNs to integrate the information between CT slices, which is a new attempt and is significant for feature extraction of CT images.
2. The problems solved in this paper are mainly focused on clinical practice.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. The main work was not well summarized, and too many additional works are added, such as comparative experiments of the feature extraction.
2. Some of the experiments were unconvincing.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The proposed method has a good clinical application prospect, and the code is expected to be released.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
1. The main work of this paper is the application of GCNs in CT images and a new pooling method. Why Inception V3 and wavelet are compared in the comparison experiment? In addition, why only used these two feature extractors?
2. This paper only shows the comparison of the results of various pooling methods under the premise of using GCNs, but does not show the comparison and analysis of the results with the existing 3D classification model in detail. The comparison with 3D ResNet is only mentioned in the abstract section.
3. The authors are encouraged to evaluate the proposed model on more datasets.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
1. This paper has very good innovation in the method, including the application of the existing methods and the improvements for the existing methods.
2. The proposed method has great significance for clinical application.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

2
Reviewer confidence

Somewhat confident

Review #3

Please describe the contribution of the paper

This study presents a novel distance aware pooling to abstract node information, combined GCN, the method has achieved state-of-the-art performance in public COVID-19 dataset both in diagnosis task and prognosis prediction. Furthermore, the one-drop localization introduced in this study, can mark most informative slices, the precision and recall is acceptable in clinical usage.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The academic writing is fluent enough for interpretation. And the proposed method is proven carefully to have both clinic impact and engineering feasibility. To prove proposed method, authors employed two feature extraction method, compared results with 3 existed technologies and reported both mean accuracy and standard deviation. The experiments are sufficient to support conclusion.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Author mentioned ratio of reserved clusters to next layer, aka k, is a hyper-parameter. Moreover, the number of layers in GCN should be a hyper-parameter too. Just wandering if authors can compare performance with different setup.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The method and experiment result introduced should be reproducible.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

It would be better for authors to provide implementation code in a public repository.
Please state your overall opinion of the paper

strong accept (9)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The academic writing is fluent, the experiments and discussions are sufficient. The topic and quality of this submission is appropriate for MICCAI.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

4
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

This paper uses GCN for COVID-19 prognosis and diagnosis. The relationship between CT slices are taken into account by using graph convolution. A novel distance aware pooling is applied for dimension reduction in the constructed graph. MLP is used for classification.

Strength: using GCN for classification and each slice is treated as a node. Different algorithms were tested but need justification about why, and the study addresses an important clinical topic and brings some new idea in methodology.

Weakness: the results of the baseline (e.g. 3D ResNet) need to be presented (this is the major comment to support the claim of the paper); justify number of layers and reception field if possible.

Writing: well written in terms of language and clearity
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

2

Author Feedback

We appreciate the reviewers’ and Area Chair’s valuable comments and insightful feedback. Below is the detailed response to the major comments:

Comparison with state-of-the-art (SOTA) baselines

We compared our model with SOTA baselines, including 3D CNN and GCN-based methods. As stated in the abstract, we compared our method with the clinically applicable AI system based on 3D ResNet-18 proposed by Zhang et al. (2020). It reached 92.49% diagnosis accuracy on the same 2019nCoVR dataset, while with less than 1% of total parameters, our model achieved 94.8% accuracy. Our GCN-based approach has an improvement of 2.4 % over the previous SOTA 3D ResNet-18 model. We will further clarify the result in Section 3.2 Quantitative Results in the final version. We have included the experiment results for the GCN-based models with three SOTA pooling methods in Table 1 in the manuscript.

Zhang, K., Liu, X., Shen, J., et al. Clinically applicable AI system for accurate diagnosis, quantitative measurements, and prognosis of covid-19 pneumonia using computed tomography. Cell (2020) 181(6):1423-1433.e11.

Justification for the choice of hyperparameters, including the number of layers, receptive field.

Our proposed pooling method inherently achieves locally clustering with the improved receptive field, which avoids clustering nodes far away. For the pooling ratio k, many clusters would remain in each layer if k is large. Then, we could have a smaller receptive field so that the cluster could represent low-level features. If k is small, to prevent information loss, we hope the cluster to include more node information, leading to a larger value of the receptive field. Based on the experiments, the performance would compromise when the pooling ratio k is smaller than 0.5. We also found our results are robust as long as k is greater than 0.5. For the chest CT scans, the value of the receptive field is suggested to be larger than 0.7 based on hyperparameter tuning. The performance would not further improve with more than 5 GCN and pooling layers.

The “severe/critical” status of COVID-19 patients in the dataset.

As stated in Section 3.1 Dataset, the 2019nCoVR dataset provides the clinical outcome information for the hospitalized COVID-19 patients. The severe or critical status is defined and recorded as admission to an intensive care unit (ICU), the use of mechanical ventilation, or death (Zhang et al., 2020). The clinical metadata are utilized for our task of COVID-19 prognosis.

Typographical errors.

We will correct all typos in the manuscript.

Section 2.1 Problem statement, line 8: changing “sever/critical illness status” to “severe/critical illness status”;

Section 2.3 Classification and Localization Based on Pooled Graphs, last line: deleting the spare “k”;

Supplementary Material, Section A, title: changing “Related Word” to “Related Work”.

We appreciate the kind and thoughtful comments. Following all constructive suggestions from reviewers and Area Chair, we will update the manuscript.

back to top

Beyond COVID-19 Diagnosis: Prognosis with Hierarchical Graph Representation Learning