Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Ye Wu, Sahar Ahmad, Pew-Thian Yap

Abstract

A central goal in systems neuroscience is to parcellate the brain into discrete units that are neurobiologically coherent. Here, we propose a strategy for consistent whole-brain parcellation of white matter (WM) and gray matter (GM) in individuals. We parcellate the brain into coherent parcels using non-negative matrix factorization based on voxel annotation using fiber clusters. Tractography is performed using an algorithm that mitigates gyral bias, allowing full gyral and sulcal coverage for reliable parcellation of the cortical ribbon. Experimental results indicate that parcellation using our approach is highly reproducible with 100% test-retest parcel identification rate and is highly consistent with significantly lower inter-subject variability than FreeSurfer parcellation. This implies that reproducible parcellation can be obtained for subject-specific investigation of brain structure and function.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87234-2_45

SharedIt: https://rdcu.be/cyl8H

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The main contribution is a novel parcellation of the brain white matter and grey matter derived from an atlas of white matter bundles.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The main strength is the novel idea of using bundles to produce exclusive parcels. This intuition, if properly pursued could be important for the community. Another good point of this work is the choice of the method, which results clear and also interpretable, to a certain extent.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

In my opinion, the two main weaknesses are: (i) the lack of anatomical evaluation, (ii) the results and the discussion. See below for further details.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The manuscript is based on the publicly available HCP dataset. Nevertheless, the experiments are based on the undisclosed AFODFs reconstruction related to reference “[22]”. So the proposed method is not reproducible.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
(i) WM/GM parcellations in the literature have many limitations but strongly rely on anatomy and have a clear interpretation. The parcellation proposed in this work starts from white matter bundles but after its computation, such relationship to anatomy is unclear. The manuscript lacks some fundamental analysis: parcels arise from bundles but parcels do not define bundles (see Fig 3c). Moreover, the lack of anatomical guarantees is visible if we look at two different parcellations using different M values e.g., 32 and 64 as in Fig. 4a. When M=32, the parcels are not related to the case of M=64: they are two different (independent) matrix factorizations.

Indeed, one could expect that under anatomical guarantees there would be a hierarchical relation between the two parcellations, meaning that if one parcel is present in the M=32 parcellation, such parcel should be composed of multiple parcels in M=64. This happens only to a certain extent by looking at Fig 4a: this is a direct effect of the independence of the way in which the two parcellations are computed, which is based on two independent matrix factorizations.

(ii) Results and Discussion (Section 3): unclear statements:
- comments in 3.1 about Fig.2 do not explain the criterion - if there is one - for which the image shown at alpha=beta=0.15 is necessarily better than the others.
- comments in 3.1 about Fig. 3c claim that the bundles associated with a certain parcel are consistent across-subject, but the figure is showing only a single-subject example.
- comments in 3.2 about the identification rate (IR) state that this score only counts the number of parcels (composed of at least one voxel) found by the methods, while it is unclear whether the parcellations found are consistent across subjects or not. The authors only show intra-subject consistency between test-retest, Fig.4a.
- comments in 3.2 about Fig.4b are difficult to understand. The authors claim that the Dice and RD of the proposed method are generally better with respect to those of Freesurfer, but the plots seem not to support this claim. The Dice and RD plots show that the amount of parcels of the same size with lower Dice and higher RD respectively is higher in NMF plots, and thus suggest better performances by Freesurfer. Also, in the plots, there is no information about the number of parcels present nor their density. All in all, my only takeaway from these plots is that there is an obvious relation between parcel volume and its variability across test-retest. Such an obvious relation is shown again in Fig 4c, but in this case, regards different subjects rather than test-retest.
Please state your overall opinion of the paper

borderline reject (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The idea behind the paper is interesting and novel. A fine-grained parcellation of the WM/GM based on bundles would impact many studies about the WM. Unfortunately, the paper has major weaknesses that prevent acceptance. It still needs a substantial amount of work about anatomical validation of the proposed parcellation, which is necessary in order to be adopted by the community. Moreover, the paper needs better quantification metrics, especially across subjects, where the identification rate alone is not enough.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

4
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

This paper proposes a novel strategy for consistently parcellating the whole brain, including both white matter (WM) and gray matter (GM). Non-negative matrix factorization is used for the parcellation based on tractography results. The results on brain imaging data suggests that the method is highly reproducible and outperforms competing methods. Overall, the methodology is interesting and the paper is well written. There are also some issues that can be addressed, and they are described below.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper addresses a relevant problem and the method is interesting. Previous works have not well addressed the WM parcellation problem.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Some details in the proposed method can be better clarified.
- A simulation experiment with known ground truth would be helpful.
The details of weaknesses are given below.
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Public data is used, but some implementation details are not clear.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- How are C (Eq. (2)) and A (Eq. (4)) partitioned in GM and WM? Do we need to first apply tissue segmentation to obtain GM and WM classification?
- Please better clarify the detailed design of A. In other words, how exactly is each entry in A computed? An equation or example would be helpful.
- It is observed that K = 800 gives the best consistency across scans. Is this observed based on a validation set or the test scans?
- In Figure 3C, the names of the fiber clusters are given. Is this anatomical definition derived from the unsupervised fiber clustering method, or did the authors give the names after having the clustering results? Also, how exactly were these fiber clusters obtained from the parcels using H? Please clarify.
- Since for real data the ground truth is not known, the authors could consider generating or using a simulation phantom to test the accuracy of the proposed method, for example, see the phantom in HARDI reconstruction challenge 2013 or the phantom in Maier-Hein et al. Nature Communications 2017.
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper addresses a relevant problem with a novel method, but the presentation is not very clear.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

6
Reviewer confidence

Confident but not absolutely certain

Review #3

Please describe the contribution of the paper

This paper developed the method to parcellate the brain into coherent parcels using non-negative matrix factorization based on voxel annotation using fiber clusters. Experiments performed on HCP dataset show promising results.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. This work provide a novel formulation about white matter and grey matter parcellation.
2. The way of using fiber tract clustering and NMF is novel.
3. The evaluation is rich and convicible.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. I cannot find where the reference [22] is.
2. It would be better to show which toolbox you used to generate fiber tracts since different toolboxes may yield different results even for the same method.
3. The details of clustering algorithm is a little bit vague to me.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The overall reproducibility should be OK.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

All good.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The idea presented in this paper is novel. The authors did abundant experiments to demonstrate their method as such that evaluation part is strong. This paper is easy to read.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

The work is novel however I agree with the reviewers on illustrating the technique on a simulated dataset with known ground-truth. Also, there seems to be an ambiguity between the plots shown and inference from the plots that needs to be resolved.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

5

Author Feedback

Evaluation on a simulated dataset with known ground-truth (R1, Meta): Unfortunately the dataset in Maier-Hein et al., 2017 was not specifically designed for regions-based parcellation evaluation and does not include a test-retest dataset.

More anatomical evaluation (R1, R2, Meta): Parcellation outcomes from our method can be subsequently annotated based on anatomical knowledge. However, there are many ways the parcels can be annotated. Parcellation nomenclature is beyond the scope of this conference paper. Our paper focuses on the reproducibility of the parcels generated in a data-driven manner. Test-retest reproducibility is an important criterion for reliable parcellation (Zhang et al., 2019; Glasser et al., 2016; Bazin et al., 2020). We believe that the anatomical annotation and validation are interesting topics to explore in the future.

The relationship between the plots shown and inference from the plots needs to be resolved (Meta): In the paper, we discussed the effects of false positives and regularization parameters on whole-brain parcellation and demonstrated the results in Figures 2 and 3. We further performed experiments to confirm the high reliability of our parcellation method on a test-retest dataset (Fig. 4). It can be inferred from the results that our method provides highly reproducible whole-brain parcellation.

More quantification metrics (R1): Based on 44 HCP scans, the results suggest that our method improves the lowest parcel identification rate by 37% and decreases the CV across subjects by 20%, when compared with Freesurfer. In our paper, validation has already been performed on different scales, covering population- and subject-specific parcellation.

Implementation details (R2, R3): We applied K-medoids clustering to the streamlines represented using cosine series [Chung et al., 2010; Wu et al., 2020] to enforce bilateral consistency across hemispheres without being affected by fiber lengths. The method has been shown to be effective, robust, and fast without requiring other constraints, such as [3].

Reference [22] and toolbox (R3): We employed tractography with asymmetric fiber ODFs owing to its unique advantages of capturing sub-voxel fiber configurations and mitigating gyral bias. The associated paper is currently anonymized but will be revealed in the final version. We agree with the reviewer and are happy to investigate in the future the effects of tractography algorithms on the parcellation outcome.

Pre-segmentation of GM and WM (R2): The brain is first pre-segmented into WM and GM voxels. Matrices C_{WM} & C_{GM} in Eq. (2) are partitioned based on the WM and GM tissue maps.

Design of A (R2): A is the adjacency matrix for encoding voxel spatial relationships (Arslan et al., 2018; Bastiani et al., 2012). The elements of matrices A_{WM} and A_ {GM} indicate whether each pair of voxels are adjacent.

Best consistency across scans (R2): The parameter K=800 was determined based on the test-retest reproducibility of streamline clustering with different numbers of clusters. The current setting results in low variation in streamline count per cluster and across subjects.

Anatomical definition (R2): We identified the fiber clusters corresponding to selected parcels. For each fiber cluster, we anatomically annotated a fiber cluster using the white matter query language (WMQL) (Wassermann et al., 2016). Note that there are many ways to annotate fiber clusters.

DT tractography (R1): We used a typical deterministic streamline tractography based on FODFs. The method, SD_Stream, is available in the Mrtrix3 and has been widely used in previous studies.

Minor issues (R1, R2, R3): Remaining minor issues can be easily fixed without increasing the length of the manuscript.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The rebuttal answers majority of the questions.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

5

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This paper proposes to parcellate the whole brain, beyond the cortical surface, but also within its white matter. To do so, it proposes to leverage information from fiber clustering. Their proposal is to evaluate the parcellation method using robustness of their parcels under test-retests. This work is relevant to the whole neuroscience field, with possible major impact.

One reviewer questions the fundamental disparity between parcels from bundles and lack of definition of parcels from bundles.

A second reviewer would like a ground truth validation, which is, in my opinion, difficult to obtain in this application.

A third reviewer appreciates the novelty of the method.

This submission has mitigated results. All appreciate the novelty in the computerized methodology, but valid fundamental concerns are raised on the disparity between parcel areas and fiber bundles areas. This paper, however, provides the methodological foundations to a new type of whole brain parcellation. Given that ground truth is indeed difficult to obtain, the method can hardly be further validated within a MICCAI submission, but constitutes a solid reference to such study in neuroscience. This may become a high impact paper. This is also all well addressed in the Author’s rebuttal, notably on the concerns on evaluation.

For these reasons, methodological novelty and potential impact in neuroscience, Recommendation is towards Acceptance.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

4

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

Although this paper indeed provides some interesting results and the rebuttal indeed answers part of the concerns from the reviewers, there are still some fundamental issues. 1) As reviewer1 pointed out, “The manuscript lacks some fundamental analysis: parcels arise from bundles but parcels do not define bundles (see Fig 3c). Moreover, the lack of anatomical guarantees is visible if we look at two different parcellations using different M values e.g., 32 and 64 as in Fig. 4a. When M=32, the parcels are not related to the case of M=64: they are two different (independent) matrix factorizations.”, this is not addressed in the rebuttal.

2) The in-stability of the results from tractography algorithms. For example, different parameter setting in fiber tracking or different fiber tracking algorithms will significantly affect the results, which will in turn affect the Parcellation results.

Based on the above two, I don’t think this work is sound enough for MICCAI.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Reject
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

8

back to top

Highly Reproducible Whole Brain Parcellation in Individuals via Voxel Annotation with Fiber Clusters