Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Tananun Songdechakraiwut, Li Shen, Moo Chung

Abstract

A long-standing challenge in multimodal brain network analyses is to integrate topologically different brain networks obtained from diffusion and functional MRI in a coherent statistical framework. Existing multimodal frameworks will inevitably destroy the topological difference of the networks. In this paper, we propose a novel topological learning framework that integrates networks of different topology through persistent homology. Such challenging task is made possible through the introduction of a new topological loss that bypasses intrinsic computational bottlenecks and thus enables us to perform various topological computations and optimizations with ease. We validate the topological loss in extensive statistical simulations with ground truth to assess its effectiveness of discriminating networks. Among many possible applications, we demonstrate the versatility of topological loss in the twin imaging study where we determine the extend to which brain networks are genetically heritable.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87196-3_16

SharedIt: https://rdcu.be/cyl1G

Link to the code repository

https://topolearn.github.io/topo-loss

Link to the dataset(s)

http://stat.wisc.edu/~mchung/softwares/dti

Reviews

Review #1

Please describe the contribution of the paper

The paper proposes a novel topological learning framework that can integrate networks of different topologies. The method is applied to twin study in aligning functional network to structural brain network. In the experiments, they showed that their model outperformed state-of-the-art methods.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The major strengths of the paper are as follows:
- Scalable, discriminative method and sensitive in detecting subtle genetic patterns.
- Highly original contribution.
- On a methodological level, it is a reasonable appropriate method.
- The evaluation carefully tests the impact of different parts of their approach.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The abstract is not well structured. The first sentence should introduce the context or briefly introduce existing solution of the problem. Second, authors should tell the limitations of previous solutions that they aimed to address. Then, authors should mention the key steps of the proposed method. Finally, the last part should include the results (brief description).
- The format of the paper is not good; the conclusion part is formatted into two columns.
- Technical explanation could be more thorough. Although the paper provides some citations. However, some key steps deserved more explanation.
- In the conclusion, authors mentioned “The method is applied to twin study in aligning functional network to structural brain network.” However, distribution alignment is overlooked in this paper. I think that even the topological learning framework achieved promising results, it could be improved when aligning distributions.
Please rate the clarity and organization of this paper

Poor
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Authors could share their code to improve the reproducibility of the paper.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- This work clearly involves lots of work. Unfortunately, the way in which the manuscript is written makes it difficult to understand the main ideas behind it.
- It is better to give key words
- Conclusion part and fig. 3 should format into one column
- In table 1, the results of the proposed method could be written in bold.
- Authors numbered only one equation. It is better to provide numbers to all equations.
- Reduce the details in the manuscript to allow a bit more space for presenting the findings
Please state your overall opinion of the paper

borderline reject (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- The organization and structure of the manuscript make it very hard to read. I had to go through it several times before understanding the approach proposed by the authors
What is the ranking of this paper in your review stack?

3
Number of papers in your stack

4
Reviewer confidence

Confident but not absolutely certain

Review #2

Please describe the contribution of the paper

This paper design a novel topological loss function that can integrate brain networks of different topology through persistent homology.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The paper provide a novel network estimation method, which can effectively combine the topoligical information from the structure data and the correlation from the functional images. The results works better than the conventional graph matching methods.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The theory does not directly apply to the objective function as the tho is highly related to the Theta. It was not suitable to directly update the theta with a constraint tho, since it was a bi-level optimization problem of the the topological loss bi, and the tho may changed with different Theta.

The sensitive analysis of hyper-parameter is missing

The Brain Network Matching methods is not compared, such as [1 ,2]

[1] Surampudi, S., Naik, S., Surampudi, R., Jirsa, V., Sharma, A., Roy, D.: Multiple kernel learning model for relating structural and functional connectivity in the brain. Sci Rep 8, 3265 (2018) [2] Becker, C., Pequito, S., Pappas, G., Miller, M., Grafton, S., Bassett, D., Preciado,V.: Spectral mapping of brain functional connectivity from diffusion imaging. Sci Rep 8, 1411 (2018)
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

it was easy to reproduct.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

The updates rule of tho in topological loss remain unclear. In my opinion, it was a bi-level optimization problem of the the topological loss, since both tho and theta should be update. However, the author directly update the theta with a constraint tho. In my opinion, it was not reasonable. Maybe EM algorithm should be considered.

The sensitive analysis of the hyper-parameter lambda should be addded

The Brain Network matching methods such as [1 ,2] should be compared.

[1] Surampudi, S., Naik, S., Surampudi, R., Jirsa, V., Sharma, A., Roy, D.: Multiple kernel learning model for relating structural and functional connectivity in the brain. Sci Rep 8, 3265 (2018) [2] Becker, C., Pequito, S., Pappas, G., Miller, M., Grafton, S., Bassett, D., Preciado,V.: Spectral mapping of brain functional connectivity from diffusion imaging. Sci Rep 8, 1411 (2018)
Please state your overall opinion of the paper

borderline reject (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

some comparsion methods is not considered. The biological meaning of the topological loss is unclear.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

4
Reviewer confidence

Confident but not absolutely certain

Review #3

Please describe the contribution of the paper

This paper proposed a new algorithm that can integrate different topological networks using the concept of persistent homology. A new loss function L_top (Θ,P) was defined to measure the similarity (difference) of topological structures between two networks. In the task of detecting the difference of network topologies using computer-simulated data, it is proved that this model had the best performance among existing algorithms and was more sensitive to detect subtle topological differences. Then this new algorithm was applied to study the heritability of twins, showing that noisy cycles were removed in the functional connections generated by this new algorithm and some connections with high heritability in twins were detected.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

no
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The algorithm of this paper is innovative. But it is confusing in expression and difficult to understand. There are several problems as follows: This article covers too much content, including some basic graph theory, the establishment of the new algorithm, the use of simulated data to verify the effectiveness of the new algorithm and the application of the new algorithm in real data. The focus and main point are unclear. The background and motivation of the study are not described sufficiently. For example, the paper mentioned that it is difficult to integrate networks with different topologies together, but the meaning of it in brain network research was not described. The description of other similar studies was inadequate. On page 5, how did you get λ=1±0.0002 experimentally? In the part of the twin genetic experiment, why this algorithm was applied to study the genetic heritability of twins? The motivation of the application part was confused. Besides, the data processing description is too detailed and unnecessary. There’s no introduction of ACE model or the heritability index and the task is unclear and hard to understand. The discussion section is not sufficient.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

the used dataset is open
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

See comments to Q4
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The Brain Atlas is old and the mehtod is not novel.
What is the ranking of this paper in your review stack?

5
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Review #4

Please describe the contribution of the paper

In this submission, the authors propose a complete learning framework that integrates both the structural brain network and the functional brain network as a coherent statistical framework. Using multiple modalities to analyze brain networks is a currently popular topic. The proposed method realized a more efficient and effective framework for cerebral connectome analysis.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

One highlight is the theoretical achievements in designing a more applicable learning framework. The time complexity drops from cubic time complexity to O(nlogn). In validations, the authors also demonstrated this achievement; The second highlight is a new framework for brain networks of both structural and functional networks. Because of the resolution differences, fusing both networks has always been a tough question. This proposed work provided a novel way to tackle the problem. Finally, both the theoretical derivation and the validations are solid. Potential applications are interesting.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

(1) Comparing with the explicit explanation of theories, the validation and application parts are somehow less explained. I would like to see more explanations about the experimental results from the authors. (2) I am confused with some parts in the submission, more clarifications are expected. For example, are the birth value bi and the death value di continuous? Some other questions will be asked in section 6.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The framework of the proposed method is quite clear. But I may find difficulties when implementing some details of the proposed method.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

(1) Any reasons to choose squared sum in the loss function? (2) Please clarify why the proposed method is scalable. Why doesn’t the running time increase while the number of node increases? (3) Please further explain Table 1. Especially the comparison before and after the double line. (4) Besides the generic heritablility application, can you provide some other perspects of using the proposed method? Like, what’s the possibility of applying to the diagnosis of diseases.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The theoretical achievement is definitely a highlight. A solid theoretical analysis increases the confidence of the robustness and correctness of the methods. Although the experiment part could be further enriched, it is an excellent submission for a conference. Such creative work will greatly inspire the community.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

4
Reviewer confidence

Confident but not absolutely certain

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

This paper proposes a novel topological learning framework that can integrate brain networks of different topology. Specifically, the authors introduced a new persistent homology-based topological loss function and therefore bypassed the intrinsic computational bottleneck associated with matching networks across different imaging modalities and subjects. This work is original and highly innovative. The authors provide rigorous theoretical analysis and proofs. The experimental results are solid, comparing with several other methods.

In the rebuttal, among other questions, please address 1) Why \rho is used to update \theta? 2) On page 5, how did you get λ=1±0.0002 experimentally? 3) why is the current method scalable?, 5) explaining Table 1, etc.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

2

Author Feedback

We thank the meta reviewer (R1) and four reviewers (R2-R5) who complimented our work as “original”, “highly innovative” and “solid validation”. Unfortunately, reviewers 2 and 3 misunderstood our paper. Misunderstandings related to implementation will be easily addressed by additional editing and code distribution with publication.

R2: Distribution alignment is overlooked in this paper.

Actually, the topological loss (Wasserstein distance) we used is the distance between distributions on persistent diagrams through the Dirac-delta function (Cohen-Steiner et al., 2010). We will add additional explanation and reference.

R3: The theory does not directly apply to the objective function as the tho is highly related to the Theta. It was not suitable to directly update the theta with a constraint tho, since it was a bi-level optimization problem of the topological loss bi, and the tho may change with different Theta. The update rule of tho in topological loss remains unclear. … However, the author directly updates the theta with a constraint tho. In my opinion, it was not reasonable.

Reviewer R3 completely misunderstood our method. “tho” reviewer is referring to is actually “tau”. We indeed updated both tau and Theta during gradient descent. At each iteration, when Theta is updated, tau is also updated with respect to the updated Theta. The optimization of Theta is done through Theorems 1 & 2 analytically, drastically reducing runtime. This is our main contribution. This will become more evident when the code is distributed. We will add additional explanation.

R4: How λ=1 ±0.0002 was obtained experimentally?

The total loss is the sum of geometric and topological losses. We searched over different lambdas to find the value that minimized the total loss for each subject. For all the subjects, the average lambda is 1 with s.d. 0.0002. Small s.d. shows topological stability of the proposed method.

R5: Please clarify why the proposed method is scalable. Why doesn’t the running time increase when the number of nodes increases?

Our method is O(n^2 log n) with n number of nodes. In the validation section, we compare our runtime against four existing graph matching algorithms, which are of polynomial time of higher degrees. Our run time appears constant compared to the other slower algorithms since the y-axis is in logarithmic scale.

R5: Please further explain Table 1. Especially the comparison before and after the double line.

Table 1 displays the simulation-based validation results in terms of p-values. Results above the double lines show if the method can detect signal if there is signal (smaller p-value is better). Results below the double lines show if the method produces false positives if there is no signal (larger p-value is better).

Minor issues:

R3: …biological meaning of the topological loss?

The topological loss consists of two terms. The first term measures how close two networks are in terms of 0D topological bases (connected components). The second term measures how close two networks are in terms of 1D topological bases (cycles). 0D bases represent integration of a brain network while 1D bases represent how strong the integration is (Chung, 2019, ISBI).

R3: …methods such as [1,2] should be included as baselines.

We compared to four baseline methods. Since codes for [1,2] are available, we will add additional comparisons.

R4: The AAL brain atlas is old and not novel.

We are not proposing use of the AAL as a contribution. We used AAL since it is a well-established, widely used baseline template. Our method is applicable to any atlas.

R4: Why was this algorithm applied to study the genetic heritability in twins?

Twin imaging studies are the easiest ways to examine the extent to which brain networks are influenced by genetic factors. Our findings can be used as the baseline for studying more complex relations between brain networks and other phenotypes.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This paper proposes a novel topological learning framework that can integrate brain networks of different topology. Specifically, the authors introduced a new persistent homology-based topological loss function and therefore bypassed the intrinsic computational bottleneck associated with matching networks across different imaging modalities and subjects. This work is original and highly innovative. The authors provide rigorous theoretical analysis and proofs. The experimental results are solid, comparing with several other methods.

The weakness of this paper is with missing details in the algorithm design and description. The AC hopes the authors may improve these points in their future extended work.

The authors did a fairly good job in the rebuttal by addressing each of the major concern the AC has listed.

An “Accept” recommendation is made to recognize its strong originality, solid theoretical foundation, and valid experimental validation.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

2

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The paper proposes a new method to integrate topology of brain networks from different modalities. Although the concept of topological loss has been introduced in other context, its application to brain network analysis is new and well motivated. The concerns raised by reviewers seem to be addressed in the rebuttal. Thus I recommend the paper to be accepted.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

5

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The proposed idea of topological learning for brain networks incorporating a topological loss function has novelty. The authors addressed reviewers’ questions about the formulation and experimental details in the rebuttal.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

4

back to top

Topological Learning and Its Application to Multimodal Brain Network Integration