Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Lin Zhao, Haixing Dai, Xi Jiang, Tuo Zhang, Dajiang Zhu, Tianming Liu

Abstract

Understanding the functional mechanism of human brain has been of intense interest in the brain mapping field. Recent studies suggested that cortical gyri and sulci, two basic cortical folding patterns, play different functional roles based on various data-driven methods from local time scale to global perspective. However, given the evidence that the brain’s neuronal organization follows a hierarchical principle both spatially and temporally, it is unclear whether there exists temporal and spatial hierarchical functional differences between gyri and sulci due to the lack of suitable analytical tools. To answer this question, in this paper, we proposed a novel Hierarchical Interpretable Autoencoder (HIAE) to explore the hierarchical functional difference between gyri and sulci. The core idea is that hierarchical features learned by autoencoder can be embedded into a one-dimensional vector which interprets the features as spatial-temporal patterns, with which the region-based analysis in gyri and sulci can be further performed. We evaluated our framework using the Human Connectome Project (HCP) fMRI dataset, and the experiments showed that our framework is effective in terms of revealing meaningful hierarchical spatial-temporal features. Analysis based on Activation Ratio (AR) metric suggested that gyri have more low-frequency/global features while sulci have more high-frequency/local features. Our study provided novel insights to understand the brain’s folding-function relationship.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87234-2_66

SharedIt: https://rdcu.be/cyl9c

Link to the code repository

https://github.com/liniez/HIAE/

Link to the dataset(s)

https://db.humanconnectome.org/

Reviews

Review #1

Please describe the contribution of the paper

The authors developed an autoencoder-based new deep learning model, a hierarchical interpretable autoencoder (HIAE). They tried to extract and interpret spatial-temporal different information between sulci and gyri using the proposed HIAE model. From the model, they successfully showed hierarchical spatial/temporal patterns extracted by their unsupervised autoencoder model and interpreted the functional difference between gyri and sulci.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

First, it is a novel application to utilize an autoencoder-based model with a kind of skip connection (i.e. FI, feature interpreter) model to extract and interpret the pattern of fMRI signal. And the results they reported seemed overall reasonable such as the functional difference between gyri and sulci in temporal (i.e. frequency domain) and spatial (i.e. local vs global features) with hierarchical manner.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

I think there could be potential issues when the authors showed their results using randomly selected features among 16 trained digits. Also, they randomly selected only 200 subjects from HCP tfMRI data. It can be one of the weaknesses of their work since HCP consists of more than 1,000 subjects, so their analytical power is not much higher compared to using the whole dataset (e.g. 1,200 subjects) from HCP. And the explanation of the data they used in the method section is quite hard to read and understand. In detail, I can’t fully understand how the splitted training and validation dataset became 8,000,000/2,000,000 dimensions, respectively.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

N/A
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

1) It would be good if you denoted the meaning of each number in Table 1 by using caption, even you mentioned it in text. 2) Also, if you made the figure such as bar plot or box plot to describe the increasing tendency of gyri activation in Table 2, it will help to understand your result more easily. 3) Not a big thing, but the image resolution of Fig. 3. seemed low. So I think you can make a more high quality figure. 4) There is a typo in 4. Discussion and Conclusion gyral/sucal (–> gyral/sulcal)
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The method they proposed seemed effective even though the model is based on simply deep convolutional autoencoders. The feature interpreter implemented in their model worked correctly and successfully extracted the hierarchical feature from fMRI signal.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

4
Reviewer confidence

Confident but not absolutely certain

Review #2

Please describe the contribution of the paper

This paper proposed a Hierarchical Interpretable Autoencoder to interpret fMRI data in both spatial and temporal domain hierarchically. By further mapping the interpreted data to gyri and sulci, the functional mechanism of these two cortical folding patterns is explored in terms of their hierarchical spatial and temporal response pattern.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The novelty of this paper mainly shows in the design of the neural network, in which Feature Interpreter (FI) layers are introduced into the middle of Encoder-Decoder layers. The 1d vector, in the middle of Feature Interpreter, is used as latent representation of a specific cortical area’s response to a stimulation. It is a clever design that the intermediate value of neural network is extracted for further analysis, especially for it endows explainable meaning to the intermediate value. The other strength of this paper is the original method it uses to interpret the hierarchical temporal information from the spatial information and the original input. This method is simple and explainable. The definition of activation ratio is also clever since it evaluates the response level of gyri and sulci as groups and analyzed the overall activation level to get a generalized result.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

This paper doesn’t not mention the network convergence result for gyri and sulci, if the final loss of one type of structure is significantly smaller than the other type of structure, the analysis result would be biased.
Please rate the clarity and organization of this paper

Poor
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

This paper uses open source fMRI data for network training and evaluating, the available dataset size is enough. The network structure is listed clearly in this paper as well as the training hyper-parameter. The data analysis method is also discussed in the paper. Therefore, the reproducibility of the paper is good. The only limitation is that due to the existence of fully connected layers in this network, the input fMRI data is limited to a particular length. Anyone with different length’s fMRI data need to retrain the network with their own data.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
1. The background introduction in the abstract section is a little bit confusing, it could directly state the key problem to be addressed in this paper.
2. The network training result should be added and discussed.
3. The organization of the paper could be improved. Section 2.3 introduces the HIAE structure with only two sub-sections, Autoencoder and Feature Interpreter. Interpreting hierarchical features (Fig.1 (d)) is the key innovative part in this paper, this could be explained in a separate sub-section to make it easier for understanding. How to do the mapping and regression should be more clearly stated
4. In section 3.1, the data used for Fig.2 demonstration is not explained clearly. The following sentence is confusing. “for each digit in the FI of the last layer (Layer4), we distinguished the most similar counterparts with the highest cosine similarity in the preceding FIs”
5. The network uses 4 layers, the author might could explore how varied network layers impact the result.
6. In table2, it seems MOT has an overall higher AR value than EMO. If this could be explained by any known research result.
7. The axes labels could be added (x-digits number; y-Activation Ratio) for better reading experience.
8. Consider using a different symbol for threshold, since fMRI data length has already use t for representation
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The network training result is not fully revealed and discussed. Without proving that network has achieved a good and comparable convergence between gyri and sulci, the data analysis result of mapping and regression is not persuasive. The structure of the paper should also be optimized to make it easier for understanding.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

2
Reviewer confidence

Confident but not absolutely certain

Review #3

Please describe the contribution of the paper

This work focuses on exploring the difference between gyri and sulci via proposing a novel autoencoder (AE).

Contributions:

1). It is very insightful strategy to investigate the functional difference of gyri/sulci using AE.

2). A novel AE is proposed in this work.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

1). Novelty. A new AE is proposed in this paper;

2). Some functional difference of gyri/sulci have been reported in this paper.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The reviewers’ concerns:

1). Biological meaning The vital contribution of this work is to explore some differences of gyri/sulci. Can authors explain the biological meaning of reported differences and the relations with functional connectivity? Moreover, this important conclusion needs to be supported by other references or different computational pipelines.

2). The hyper parameters Although it is difficult issue to estimate the hyper parameters of deep learning models, the fundamental explanation of parameters tuning is required. In my opinion, there have been reported a lot of neural architecture search (NAS) models. The authors can employ some NAS strategies to automatically determine the hyper parameters. Alternatively, the authors can investigate multiple hyper parameters and provide the convergence curve. In general, the hyper parameters can be reasonably decided by the reconstruction accuracy.

3). The mathematic issue

In Eq. (1) and (2), obviously, the variables in optimization function are matrices. The definition of L2 norm and Frobenius norm is different. I guess that the authors originally refer to the F norm.

It is hard to understand the variable ST in Eq. (1). If ST represent the production of S and T, authors need clearly describe the definition of ST.

4). Optimization

The reviewers are very curious about optimizer used in this work. Why do authors employ the Adam as the optimizer to optimize the AE model? For a non-convex optimization problem, e.g., DNN, the convergence of ADAM should be slow. If authors employ the ADAM to train the deep model, in my opinion, it will be very time consuming. Moreover, compared with stochastic gradient descend (SGD), more parameters tuning of Adam is required. Can authors provide more details of these parameters?

The original paper of Adam is below: https://arxiv.org/pdf/1412.6980.pdf
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors require the reproducibility examination.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

1). Explain the reported biological meanings of difference between gyri and sulci;

2). Hyper parameters tuning: The authors can do the NAS or set different sets of hyper parameters and provide the reconstruction error.

3). Reproducibility. The authors can simply include a longitudinal/test-retest data set and compare the variance between the results of test and retest.
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

1). Novelty of proposed AE;

2). Weak methodological validation and parameters tuning;

3). Require the examination of reproducibility.
What is the ranking of this paper in your review stack?

5
Number of papers in your stack

5
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

The reviewers appreciated the novel method and extensive validation. They noted a few ambiguities with the organization and clarity of the paper, which the authors should clarify in their final submission.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

1

Author Feedback

We appreciate all comments from the reviewers. As suggested by the meta-reviewer, we will improve the clarity and organization of our paper in the final submission.

back to top

Exploring the Functional Difference of Gyri/Sulci via Hierarchical Interpretable Autoencoder