Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

S. Shailja, Angela Zhang, B.S. Manjunath

Abstract

We propose a novel and efficient algorithm to model high-level topological structures of neuronal fibers. Tractography constructs complex neuronal fibers in three dimensions that exhibit the geometry of white matter pathways in the brain. However, most tractography analysis methods are time consuming and intractable. We develop a computational geometry-based tractography representation that aims to simplify the connectivity of white matter fibers. Given the trajectories of neuronal fiber pathways, we model the evolution of trajectories that encodes geometrically significant events and calculate their point correspondence in the 3D brain space. Trajectory inter-distance is used as a parameter to control the granularity of the model that allows local or global representation of the tractogram. Using diffusion MRI data from Alzheimer’s patient study, we extract tractography features from our model for distinguishing the Alzheimer’s subject from the normal control. Software implementation of our algorithm is available on GitHub

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87237-3_17

SharedIt: https://rdcu.be/cyl9W

Link to the code repository

https://github.com/UCSB-VRL/ReebGraph

Link to the dataset(s)

http://adni.loni.usc.edu/

Reviews

Review #1

Please describe the contribution of the paper

This paper develops a computational geometry framework for representing collections of tractography curves, or tractograms. This can provide a mathematical representation that can more readily detect critical points and group together similar structures within tractograms.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The paper is clearly written and provides a comprehensive survey of related works in this area. Their approach is well-justified theoretically, and their techniques are sound. This is an interesting approach because it better connects tractography analysis to the wealth of mathematical tools for Reeb graph analysis. They alsoI think one issue with the approach is that it handles trajectories with a clearly defined start and end; however, diffusion MRI doesn’t provide this level of detail, so the start and ends of each curve are effectively arbitrary. I wonder what effect this might have on the results. Another issue is that tractography is well-understood to systematically produce some false positives, and this is not something that is discussed or addressed. I was left wondering if erroneous tracks could dramatically change the structure of the Reeb graph. An experiment looking at synthetic data, such as the ISBI fiber cup phantom or the ISMRM diffusion workshop challenge datasets, might help in this regard as false positives can be detected and included in a controlled way. apply their approach to a reasonable dataset of in vivo scans from ADNI.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

I think one issue with the approach is that it handles trajectories with a clearly defined start and end; however, diffusion MRI doesn’t provide this level of detail, so the start and ends of each curve are effectively arbitrary. I wonder what effect this might have on the results. Another issue is that tractography is well-understood to systematically produce some false positives, and this is not something that is discussed or addressed. I was left wondering if erroneous tracks could dramatically change the structure of the Reeb graph. An experiment looking at synthetic data, such as the ISBI fiber cup phantom or the ISMRM diffusion workshop challenge datasets, might help in this regard as false positives can be detected and included in a controlled way.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors provide a link to code for the paper (although blinded by review), and this can allow readers to reproduce the results. Their test data is public as well.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

I provided several suggestions and and constructive comments above, in particular related to handling false positives, clarifying the applications of the method, and considering the start/end orientation of the curves. One other consideration is the evaluation of the method. Because the paper is theoretical in nature, it is difficult to compare it to past work; however, this might temper the enthusiasm of the reader. So I wonder if the authors were to identify a particular analytic task that the method could improve, that could provide a concrete test to be compared with other tractography processing methods.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

I think the major factors influencing my recommendation were the minimal number examples of how the method could be used in practice, and the related issues of evaluating the method in comparison to previous work. If these were present, that would increase my overall score.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

4
Reviewer confidence

Confident but not absolutely certain

Review #2

Please describe the contribution of the paper

The paper proposes a novel graph representation of tractography bundles characterising the topological properties of the trajectories. The method enables the comparison of bundles using graph features. Results on shown on trajectories connecting two cortical ROIs on Alzheimer’s patients and controls.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The manuscript is clear and well written.

The paper proposes a novel framework to compare white matter bundle reconstructions. In particular, the method focuses on the topological organisation of trajectories within the bundle, which is something other method for comparing bundles typically cannot do. Thus, this may lead to interesting development in bundle analysis in the diseased brain.

Results in vivo data show the method can distinguished bundles from control and Alzheimer’s patient population.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

In vivo results are limited to studying two bundles.

The bundles are solely defined by an ROI. It would be interesting to extend the analysis to other bundle definitions, in particular bundles defined with more criteria or manually delineated.

It is unclear how spurious trajectories may affect the analysis. It would be of high interest to included an experiment with synthetic data where selected changes in the trajectories could be related to changes in the graph properties.

Results are reported using a selected tractography algorithm. It is unclear how sensitive the method is to the type of tractography reconstruction. e.g. Would the statistical differences remain using a probabilistic streamline tractography algorithm such as iFOD2?
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The code is provide on a github repository. The in-vivo data in from a publically available dataset.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

p2. A common way to compare bundle is by using “tract profiling” or “tractometry” techniques. The paper should acknowledge those technique in the “Related Work” section. e.g. Yeatman et al., 2011, 10.1371/journal.pone.0049790

p3. The method list “Appear” and “Disappear” events for endding location of trajectories. However, trajectories estimated from DW-MRI tractography don’t have a specific begening or ending, as DW-MRI is not sentitive to the direction of connections, i.e. reversing the order of the points of a streamlines should produce identical results. Thus, I beleive it would be more appropriate to have two “Ending” events per trajectorie. Please clarify or motivate the use of two events for this.

p3. Fig.1. The figure and caption lists arising, merging, splitting and ending behaviour, but the legend and text lists appear, diappear, connect, disconnect events. Isn’t the same thing? Please clarify.

p7. Fig.5B. What is the bundle shown? Please clarify how it was obtained. The paper would benefit from using a known bundle e.g. the corticospinal tract.

p7. Please add a figure showing the selected bundles on a control and patient dataset.

p8. Please detail the tractography algorithm used for the bundle reconstruction. I assume it is one of the algorithm offer in DSI studio (from Fig.5.), please clarify this.

p8. “We also calculate network properties such as clustering, centrality, modularity, and efficiency of R”. No results are shown or discussed regarding this. Please add the results to Fig.6 and discuss.

p.10. Reference 16 and 17 are duplicated.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper is clear and well written. It proposes a novel framework for white matter bundles comparison. The method and experimentations are sound. The method may bring interesting development in bundle analysis in the diseased brain.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

3
Reviewer confidence

Confident but not absolutely certain

Review #3

Please describe the contribution of the paper

Authors propose a novel approach to study tractograms using Reeb graphs.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The proposed approach is novel and it could be promising for studying brain’s structural connectivity.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The method is not clearly explained or it is incorrect.
- The experimental setup does not provide convincing results.
- The advantages of this proposed representation is not clear.
Please rate the clarity and organization of this paper

Poor
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Authors make a commitment to provide an open source implementation in GitHub. Also they use ADNI dataset, which is publicly available as well. Therefore, the claimed results could be reproduced.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Author propose a computational topology approach for representing tractograms. While this approach is highly intriguing, the underlying theory is not well explained or it is incorrect. Authors define their manifold as the union of n fibers in a tractogram. This means that each streamline constitutes a point on the manifold. Therefore each node of the Reeb graph must represent a streamline. And level sets of the manifold must be a set of streamlines that are connected by a certain rule that defines this topological space. On the other hand, authors define a level set as “If the set of points of trajectories at step k is H_k then, M ∩ H_k is the level set of k.”, which is unclear and also contradictory to the implied manifold definition. From the presented figures and results, I deduce that authors could have meant that the manifold is actually the union of all points in the tractogram. Because the nodes of the graph are placed in R^3, for example as shown in Figure 5. However, in this case, the definition for connectivity of this topological space is not consistent with what is done in the rest of the work. There are further confusing definitions and arguments, which cloud the clarity of the method. For example, authors write that, in the Reeb graph G=(V,E), vertices represent the set of trajectories. In that case, how can Figure 5(e), which shows the Reeb graph of 3 streamlines have 8 vertices? Overall, I am afraid, either authors are very unclear in their description or the proposed method is incorrect.

Secondly the experimental setup is not convincing. I appreciate that authors wanted to demonstrate the utility of their technique in the ADNI dataset, however, the explanation of the setup, implementation and presentation of the results are not satisfactory.
Please state your overall opinion of the paper

strong reject (2)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The proposed method is either incorrect or not explained accurately.
What is the ranking of this paper in your review stack?

4
Number of papers in your stack

4
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

This paper proposes a novel method for studying fiber tracts from diffusion MRI. While the Reeb graph modeling is novel for fiber tracts, serious questions were raised about the clarity and consistency of the presentation (see comments from Reviewer 3). The experiment section is also very weak and more clarification about how this proposed method can better solve a clinically relevant problem. In addition, questions were also raised about the lack of consideration of false positives in tractography.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

5

Author Feedback

We thank the reviewers for their constructive comments. They all agree that the proposed graph representation to characterize the topology of the brain tractogram is novel/intriguing. This would further enable detailed connectivity analysis and comparisons of individual connectomes for biomedical applications. While Rev1 and Rev2 have acknowledged the method’s contribution, Rev3 has raised some concerns which we would like to address below.

Specifically, Rev3’s concerns are related to the definition of the manifold and the Reeb graph construction. First, we would like to clarify that there is a misunderstanding of the manifold definition. As pointed out in the paper (Section 2 and Section 4), we adapt the formulation from Ref #[6] which addresses a different problem scenario–that of time-dependent trajectory analysis. Our definitions are consistent with those in [6], with the manifold defined as the union of n-fibers – which we refer to as trajectories in our paper. We understand the confusion this might have caused for Rev3 to misinterpret it as each trajectory constituting a single point on the manifold. It is important to note that a trajectory is an ordered sequence of points in R^3, and hence the manifold that is a union of all trajectories is also in the R^3 space, see Section 2 of [6]. Further, the definition of the level set is in accordance with that of this manifold. We will make these definitions and interpretation clear in the final version.

A second potential concern of Rev3 is due to the misinterpretation of the following sentence in the paper: [Section 4.1, Page 5, just below Fig 3]–“We maintain a graph G = (V, E) where the vertices represent the set of trajectories”. Rev 3 incorrectly states “authors write that, in the Reeb graph G=(V,E), vertices represent the set of trajectories.” We have noted multiple times throughout the paper that the Reeb graph (R) describes the spatial evolution of a group of trajectories [see Section 4]. Please note that G is a graph used in computing the Reeb Graph R, as clearly discussed in Section 4.1. Since G is not a Reeb graph as perceived by Rev3, this could have led to false deductions about the definitions and the approach, further exacerbating the confusion for Rev3. The vertices (nodes) in R represent the evolution of a group of trajectories. Figure 5(e) clearly shows the appearing, disappearing, split, and merge behaviors that have been represented by the nodes of R and not by the nodes of G (as claimed by Rev3).

Finally, we address Rev1 and Rev3’s comments on the experiments section being weak. The scope of the paper is to give a new computational geometry algorithm for tractogram modeling along with the time-complexity analysis. However, in our preliminary statistical analysis (Section 5), we show the sensitivity of critical points generated by our algorithm for distinguishing between Alzheimer’s and normal control. We have noted in this section that since significant points of neuronal fibers are captured by R, this proposed method can provide new insights to solve a clinically relevant problem. We understand that there are quite a few possible interesting future research directions with respect to the experimental analysis and we really appreciate the comments made by Rev1 and Rev2 towards this end. For example, the effect of false positives on our analysis can be studied by analysing synthetic data such as ISBI fiber cup phantom as pointed out by Rev1.

In summary, we pointed out the Sections in the paper that address Rev3’s concern about the manifold definition and Reeb graph construction. We will incorporate the other minor suggestions made by the reviewers (additional reference to be added, editing/adding figures, and clarifying that reversing the order of points in the trajectory will produce a similar Reeb graph as pointed by Rev1/Rev2).

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The rebuttal clarifies the use of the Reeb graph for fiber bundles. This representation itself seems very straight forward, so a convincing application is need to demonstrate its value in connectivity analysis. Unfortunately, the experiments are very weak and does not address challenges in Alzheimer’s disease research.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Reject
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

13

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The authors propose a new method to quantify and represent the trajectories of the neuronal fiber bundles using the Reeb graph, and evaluate the applicability of their method on Alzheimer’s disease diffusion MR images. The main criticism of the paper is by Reviewer 3, who says the method is either very unclear or incorrect. I think the authors have reasonably responded to the Reviewer’s comments, but I admit that the theoretical description of the method is still difficult to follow for a reader (like me) who is not familiar with the Reeb graph. The paper may be acceptable if the authors better clarify the method in their final version.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

9

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The paper proposes a novel algorithm to reconstruct neuronal fibers. The author answered the question of Reviewer 3 clearly in rebuttle.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

9

back to top

A computational geometry approach for modeling neuronal fiber pathways