Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Mianxin Liu, Han Zhang, Feng Shi, Dinggang Shen

Abstract

Latest diagnostic studies at the preclinical stage of Alzheimer’s disease focus on dynamic functional connectivity network (dFCN) from resting-state fMRI. However, the existing methods fall short in at least two aspects: 1) Single-scale atlas is generally used for building the dFCN, while functional interactions at and cross multiple spatial scales are largely neglected; 2) Features extracted from dFCN at each time segment are often simply pooled together, whereas the disease related meta-states, i.e., dFCN configurations, appear transiently and may not be sensitively captured. In the presented study, we designed multiscale atlas-based graph convolutional network, and utilized a multiple-instance-learning pooling to tackle these issues. First, we leveraged those previously established multiscale atlases to build hierarchical brain networks, represented by multiscale graphs, which were also applied to different time segments to form dFCNs. At each time segment, we processed these multiscale graphs by our specially designed multiscale graph convolutional networks that were connected based on the inter-scale hierarchy. A long short-term memory (LSTM) architecture was then implemented to pro-cess temporal information of the dFCN. The output from the LSTM was pooled with attention-based multiple instance learning to dynamically assign larger weights to disease related (more diagnostic) transient states. Experiments on 481 subjects show that our method achieved 77.78% accuracy (with 75.00% sensitivity and 78.57% specificity) in healthy control vs. early mild cognitive impairment (eMCI) classification, which outperformed the state-of-the-art methods. Our study not only fits the practical needs of eMCI diagnosis with resting-state fMRI but also highlights that the pathological of eMCI could manifest as abnormal transient meta-states of multiscale functional interactions.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87234-2_54

SharedIt: https://rdcu.be/cyl8Q

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

In this paper, authors proposed a multiscale atlas-based graph convolutional network, and utilized a multipleinstance-learning pooling to classify the eMCI patients.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

1) The topic is interesting to the field. 2) The structure is well organized. 3) The analysis method is novel.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

1) There are less details. 2) There need more experiments espeically comparision with other methods.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

It seems they are not easy to reproduce the work.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

In this paper, authors proposed a multiscale atlas-based graph convolutional network, and utilized a multipleinstance-learning pooling to classify the eMCI patients. In general, this paper is well orgnazied and easy to follow. However, I still have a few major concerns. 1) There are less details which make it hard to reproduce. 2) There need more experiments espeically comparision with other methods. 3) What is the influence of different atlas?
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The experiment details is not enough to support core idea.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

4
Reviewer confidence

Confident but not absolutely certain

Review #2

Please describe the contribution of the paper

This article introduces multi-scale atlas and attention-based multiple instances learning pooling based on graph convolutional LSTM, and then evaluates the model’s accuracy and efficiency, and then analyses the meta-state in one exemplified eMCI subject.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

the paper proposed a method for building dynamic hierarchical Brain Networks and Capturing Transient Meta-states.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. This model uses deep learning, so what is the configuration of the computer in this experiment and how long did the training take?
2. The process, atlas mapping in MAGCN, just uses the mean value of several small ROIs from the same coarse parcellation, is it means that the smaller ROI from the same coarse parcellation has a great effect in MAGCN than the bigger one?
3. The conclusion about meta-state just comes from one eMCI subject. Is it stable in all of the subjects?
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

the open ADNI dataset was used
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

See Q4
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The method is too complex and the acc is not too high
What is the ranking of this paper in your review stack?

4
Number of papers in your stack

5
Reviewer confidence

Very confident

Review #3

Please describe the contribution of the paper

The paper proposed a GCN model with LSTM and MIL pooling architecture using the information in dynamic FCN within and across multiple spatial scales, to characterize the MCI-related abnormality in transient patterns embedded in dFCN.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The paper proposed a novel multiscale atlas based GCN with LSTM and MIL pooling architecture model to investigate the spatiotemporal information embedded in dynamic functional connectivity.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The missing of some detailed explanations in method part is the biggest concern, eg. details of GCN model structure.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The range of hyper-parameters considered and specification of all hyper-parameters used to generate results were missing, which weakened the reproducibility of the paper.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
1. Provide more details of MAGCN model architecture.
2. Provide detailed specification and explanation of hyper-parameter of your model. It will help to understand your deep learning model.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The author proposed a novel method that took spatiotemporal information derived from dynamic FC within and across multiple spatial scales into account and revealed a good performance in classification task of clinical groups.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

3
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

All reviewers agree that this paper presents an interesting topic, but some details are missing. The authors need to clarify 1) settings and time, 2) the effect of smaller ROIs, and 3) reproducibility.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

6

Author Feedback

Answer to common questions (R1’s Q1, R2’s Q1 and R3’s Q1 and Q2): Below we clarify common concerns on reproducibility.

Implementation details The proposed model was implemented using Pytorch and trained with epoch=100, learning rate=0.001 and batch size=30. Adam was used as optimizer with weight decay=0.01 to avoid overfitting. The training of the network was accelerated by one Nvidia GTX 3080 GPU, and the time cost was around 30 minutes in the current dataset. To address the sample imbalance, we applied a weighted cross-entropy as the loss function. The weights were the inverse of the sample ratio in the training set (ranging from 1/3 to 1/4).

MAGCN model architecture We used the spectral graph convolution (Kipf and Welling 2017) to build GCN. The GCN is with ReLU activation function and dropout (rate=0.3). We then built the MAGCN with stacks of GCNs and atlas mappings in a hierarchical manner (Fig 3). The output from MAGCN was inputted to a classical LSTM (with all-zero matrix as initial hidden state), followed by the MIL-pooling and one fully-connected layer with Softmax to generate diagnosis.

Answer to R1: Q2: There need more experiments especially comparison with other methods. A2: First, please note that, although there are existing “multiscale” analyses working on single atlas based dFCN, few method used multiple dFCNs constructed using multiscale atlases on fMRI signal exists. Therefore, we compared our methods with voting and concatenation to demonstrate the effectiveness of our multiscale fusion scheme. In addition, we used a modified “Graph-U-Net” to replace our MAGCN (Gao and Ji, 2019), which extracts the hierarchical presentation of given single-scale graph in data-driven manner. With comparable architecture, on dFCN with 300 ROIs, it obtained 75.55% ACC, 60.00% SEN and 80.00% SPE at best, which did not outperform MAGCN. Q3: What is the influence of different atlas? A3: In the presented study, we aimed to utilize the spatial overlapping relationship in the multiscale atlases as a proxy of the true biological hierarchy. In principle, a more precise description of the biological hierarchy from the atlas should offer better diagnosis accuracy and vise versa. The Schafer’s atlas provides a set of brain functional parcellations at multiple scales, all of which preserve spatial structures of brain functional networks. Such an atlas is currently the most advanced and biologically-meaningful functional hierarchical parcellation, thus the best choice for our study.

Answer to R2: Q2: The process, atlas mapping in MAGCN, just uses the mean value of several small ROIs from the same coarse parcellation, is it means that the smaller ROI from the same coarse parcellation has a great effect in MAGCN than the bigger one? A2: If we understand correctly, the question is whether, in outputs of MAGCN, the node features for finer-scale FCN are more predictive for diagnosis. Please note that atlas mapping performs a summing on the node features at fine scale, as the mapping matrix is filled with 1s. Thus, it respects differences in ROI size and brain hierarchy information from the atlases during the node aggregation, which could reserve more information than simple averaging. In addition, in Table 1, performances of single-atlas-based methods did not monotonically increase along with scale changes, probably due to different associations from FCNs at scales to disease. So the features from fine scale may not necessarily have a greater effect in MAGCN. Q3: The conclusion about meta-state just comes from one eMCI subject. Is it stable in all of the subjects? A3: We extracted the normal and eMCI meta-states from all subjects and averaged them, respectively. The averaged eMCI meta-states show more broken module structures than the averaged normal meta-states, and thus the conclusion is stable. This description will be added to our final manuscript.

Answer to R3: Please refer to “Answer to common questions”.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

I think the authors did a good job in addressing the reviewers’ questions.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

7

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This paper introduces a novel idea of using multi-scale atlases and applying multiple instance learning strategy for eMCI identification via a combination of the GCN and LSTM framework. The experimental comparisons and results support the validity of the proposed method well. In regard to the meta-states analysis, the authors need to clearly describe their ways of getting the results.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

1

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This paper proposed multiscale atlas-based graph convolutional network together with multiple-instance-learning pooling framework for early MCI identification. They applied the framework to 481 fMRI data. The novelty and unique insights are certainly the paper’s strength. Besides, the paper was well written with clear structure.

The weakness is that some details are missing. In the rebuttal, the authors provided detailed and convincing response. The rebuttal is considered strong and complete. The work may inspire more work on fMRI-based brain network analysis. Therefore, an “Accept” recommendation is made.

The weakness is that some details are missing.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

1

back to top

Building Dynamic Hierarchical Brain Networks and Capturing Transient Meta-states for Early Mild Cognitive Impairment Diagnosis