Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Yize Zhao, Xiwen Zhao, Mansu Kim, Jingxuan Bao, Li Shen

Abstract

Heritability analysis is an important research topic in brain imaging genetics. Its primary motivation is to identify highly heritable imaging quantitative traits (QTs) for subsequent in-depth imaging genetic analyses. Most existing studies perform heritability analyses on regional imaging QTs using predefined brain parcellation schemes such as the AAL atlas. However, the power to dissect genetic underpinnings under QTs defined in such an unsupervised fashion is largely deteriorate with inner partition noise and signal dilution. To bridge the gap, we propose a new semi-parametric Bayesian heritability estimation model to construct highly heritable imaging QTs. Our method leverages the aggregate of genetic signals to imaging QT construction by developing a new brain parcellation driven by voxel-level heritability. To ensure biological plausibility and clinical interpretability of the resulting brain heritability parcellations, hierarchical sparsity and smoothness, coupled with structural connectivity of the brain, are properly imposed on genetic effects to induce spatial contiguity of heritable imaging QTs. Using the ADNI imaging genetic data, we demonstrate the strength of our proposed method, in comparison with the standard GCTA method, in identifying highly heritable and biologically meaningful new imaging QTs.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87240-3_65

SharedIt: https://rdcu.be/cyl6I

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The authors propose a hierarchical Bayesian model to estimate the heritability of voxel-wise brain imaging quantitative traits. They use a Dirichlet Process prior to ensure only compact regions are assigned a non-zero heritability.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Mining voxel-wise imaging genetic associations is a highly relevant topic. Using a DP prior to cluster voxels into regions is appealing due to its nonparametric nature. Experimentals results indicate that the proposed method results in a sparse heritability map.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The related work is not discussed in sufficient depth. Previous GWAS performed voxel-wise analysis are not discussed in relation to this work (https://doi.org/10.1109/BHI.2019.8834450, https://dx.doi.org/10.1016%2Fj.neuroimage.2010.02.032, https://dx.doi.org/10.1016%2Fj.neuroimage.2011.03.077). Moreover, the experiments section is lacking a quantitative evaluation, it only comprises qualitative comparision to GCTA without knowledge what the true heritabilities are. Therefore, it is impossible to judge whether the estimated heritabilities are actually more accurate. A simulation study with known heritability is essential to judge the benefit of the proposed method.
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The reproducibility seems poor, because not all (Hyper-)priors are specified, e.g. for alpha, beta, mu, and phi.

Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

 Major Issues
 ------------
 - My biggest concern is that the experimental evaluation is inadequat. First of all, the authors only compared against GCTA, whereas other methods for heritability estimation do exist, most importantly LD score regression and MEGHA. Moreover, a quantitative evaluation of methods is missing. The experiments on real data cannot be used to judge whether the proposed method more accurately estimates heritability than competing methods. In section 3.3, the authors state "BHM heritability estimates in bilateral superior frontal gyri (0.532, 0.409) are higher than GCTA results (0.124, 0.119)" as this would indicate that BHM outperforms GCTA, whereas in fact the true heritability is unknown. A similar argument is made in relation to UKB: "BHM heritability estimates in left post central gyrus (0.485) is higher than this study’s GCTA result on ADNI (0.124) and the prior study’s GCTA result on UKB (0.300)." UKB is a different patient population that does not focus on Alzheimer's, therefore a quantitative comparison of heritability estimates is meaningless. The only valid conclusion one could make is that BHM results in a sparser heritability map.

 - Previous work on voxel-wise GWAS are not discussed in relation to this work (see above).

 - Considering that were are likely dealing with millions of SNPs and imaging traits, it is important to provide a complexity analysis of the proposed method to judge whether the proposed approach is feasible.

 - In the experiments, X from equation (1) remains undefined.

 Minor Issues
 ------------
 - Equation (1) refers to a mixed model, but it is not clear what the random and fixed affects are. Using indices to indicate the grouping would clarify it.

 - The text describing equation (3) contains M_1 twice.

 - It should be made clear that equation (5) is not the authors' contribution by citing the original work.

 - ADNI is a longitudinal study. How did the authors deal with multiple visits per patient?

 - The results should include credible intervals for each heritability estimate.

 - The (Hyper-)priors for all parameters should be specified.

Please state your overall opinion of the paper

reject (3)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Due to the lack of a true quantitative evaluation and missing state-of-the-art methods, I have to suggest to reject this paper.
What is the ranking of this paper in your review stack?

4
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Review #2

Please describe the contribution of the paper

The authors proposed a new semi-parametric Bayesian heritability estimation model to construct highly heritable imaging QTs. It solved the problem that the power to dissect genetic underpinnings under OTs defined in such an unsupervised fashion is largely deteriorate with inner partition noise and signal dilution. The authors validated the method on the ADNI dataset and the experiment results demonstrated the strength of the proposed method in comparison with the standard GCTA method.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The authors created a brain heritability map under a novel Bayesian integrative heritability analysis for high dimensional voxel-wise imaging phenotypes. The model jointly incorporated the brain connectivity information and spatial correlation among voxels to enhance analytical power and biological interpretation.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

(1) Lack of more comparison methods using ROI-based methods, although the authors claimed that the Bayesian Heritability Mapping (BHM) methods are highly unreliable compared to the GCTA. (2) The idea of the proposed method is reasonable, and it leads to biologically meaningful heritability between SNPs and QTs. However, it is unclear how to evaluate the performances of different methods. (3) I believe that the distance-dependent consensus thresholding strategy for generating group-level connectivity is very important to the heritability estimation results. The authors should give more details about it.
(4) There are some writing mistakes in the paper.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

No source code provided (including GCTA and BHM)
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

My concerns are included in the list in the main weakness of the paper. Here are some further comments for authors to consider as follows: (1) As the authors mentioned some experiment results on UKBiobank, I guess the readers want to know the results on different datasets. (2) The author should explain why the they adjusted the data for the first ten genetic principal components.
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Overall, this paper is well written and easy to follow, but the novelty of the proposed method sounds incremental, the contribution lies in considering the structure information dependency among adjacent voxels. However, The authors should make some improvements for some parts of my concerns.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

2
Reviewer confidence

Confident but not absolutely certain

Review #3

Please describe the contribution of the paper

The authors propose a novel method for learning more heritable traits defined by regions in the brain. These traits could prove useful to better quantify brain related biological measurements that have a stronger genetic underpinning.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The combination of methods presented is novel in this context, brain imaging, Bayesian semi parametric model and heritability/genetics. The method is described along with results on a state of the art dataset, furthermore the authors compare to a baseline (GCTA).
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Not enough effort is put into interpreting the results from table 2. E.g. there are some cases where the baseline method is doing better and there is no mention of this. Looking at he heritability maps in figure 2, we can see that the ROIs are very small, and a have a hard time believing that this can yield robust results. The results would need to be reproduced on another dataset to validate this.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors are quite thorough in providing parameters used for doing their analysis. The proposed solution is a combination of multiple things, regarding both imaging and genetics. Providing the code would significantly aid others in reproducing the results.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

The paper demonstrates a solid effort and some interesting new approaches. Finding heritable imaging QT traits is something that could help the genetics field move forward, and certainly help us understand some of the complex brain related phenotypes. Please consider the following minor comments. 1) Add more caption text to images, try to make the images be able to stand independently. 2) Incorporate eq 1 into fig 1 with the definition of g(s). Also consider making the fig appear later in the text, after the definition of the items in the figure. 3) There is a question mark in the first citation of section 3.1. 4) If table 1 contains all the covariates, then specify that in the table caption. Please try to be more explicit in general.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This paper introduces a novel method for finding heritable QT traits in volumetric images. The presentation is solid and is a good fit for MICCAI. The paper includes a comparison to a baseline and performance in an open dataset. For future work, the authors should consider some for of replication and also share their code.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

2
Reviewer confidence

Confident but not absolutely certain

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

Though both the topic and method are interesting, the reviewers have concerns about evaluation. Please clarify the validity of the result, validation/evaluation, and complexity analysis raised by the reviewers in the rebuttal.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

7

Author Feedback

Thanks for the insightful feedback.

Reproducibility (R1,R2,R3): We will share code and clarify hyperparameters.

Comparison with methods other than GCTA (R1,R2,MetaR2): Thanks for suggesting LD Score Regression (LDSC) and MEGHA as alternative methods for comparison. (1) We did not include LDSC for two reasons. First, it has been shown that LDSC yields estimates with higher standard errors than those from GCTA (https://doi.org/10.1038/ng.3865). Second, LDSC estimates heritability using GWAS summary data. We would need to run GWAS on all the traits and then apply LDSC to the GWAS results to get the estimates. This process is much slower than GCTA, and infeasible for our voxel-wise analysis. (2) MEGHA was designed to produce an approximate moment estimation for the GCTA heritability estimator with more efficient computation and reasonable estimation accuracy. Since we already had GCTA results, there was no need to re-do heritability estimation using MEGHA as an approximated version. (3) As to the lack of ROI-based methods, the GCTA results shown in Fig. 2 & Table 2 are from ROI analysis.

Evaluation & validation (R1,MetaR2): (1) We appreciate R1’s suggestion on performing quantitative evaluation via a simulation study. We have completed it and will summarize our simulation results in the new version. (2) Regarding the validity of our real data results, we want to clarify that we were not comparing the heritability accuracies on the same trait using different methods (i.e., BHM vs GCTA). Instead, our BHM method aims to identify new imaging traits that are more heritable than the existing ROI traits (e.g., using AAL atlas). GCTA (a widely-used method for SNP-based heritability analysis) is used to estimate the region-level heritability for the average measure of each ROI. Our BHM method is able to break the constraint from the brain atlas to define imaging traits by identifying sub-areas within ROIs that are truly heritable consisting of spatially contiguous voxels with strong genetic impact and thus to form more heritable imaging traits (see Fig. 2 & Table 2). We anticipate these new imaging traits can help better reveal the biological pathway from genetic determinants to imaging endophenotypes and to phenotypic outcomes. (3) We agree with R1’s comments on the UKB comparison and will remove the relevant phrase in discussion to avoid confusion.

Related work (R1): We focus on voxel-wise heritability analysis instead of GWAS, and so didn’t cite GWAS studies. We will add the GWAS references suggested by R1, since we still have space available.

Complexity analysis (R1,MetaR2): Thanks for the comments. Our analysis is a heritability-based analysis rather than GWAS, which means even though there are millions of SNPs, we only focus on their aggregated effect as a whole genome impact on the phenotype. Thus, our computational cost mainly depends on the number of voxel-level imaging traits we consider. To induce spatial contiguity and sparsity, we adopt an Ising-Dirichlet process on the variance components, which simultaneously reduces the computational complexity to O(nth) for n subjects, t traits, and h selected heritable traits. We have reported the detailed implementation and computation time in Section 3.2.

X from Eq (1) (R1): “clinical variable X” has been specified right before Eq (1).

Minor issues (R1,R3): These will be fixed/clarified.

How to evaluate performance (R2,MetaR2): Heritability is the criterion we use for performance comparison.

Distance-dependent consensus thresholding strategy (R2): We agree with R2 and will add more details.

Why adjust for 10 genetic PCs (R2): This is to control the effect from population structure.

Table 2 results (R3,MetaR2): We agree with reviewers, and will add the following discussion. (1) The baseline method also identified some heritable traits that are worth detailed imaging genetic analysis. (2) Some identified ROIs are relatively small and warrant replication in independent cohorts.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This paper proposed a novel semi-parametric Bayesian heritability estimation model to construct highly heritable imaging QTs. Thoug more evaluation and comparsion are needed in the future, the reviewers agree that the method is novel and the results are promissing. The major concers have been addressed well in the rebuttle.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

11

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The topic is important and the proposed method is interesting. The main issue is on the validation. (1) It seems that there is no ground truth and it is unclear if the findings are correct. The proposed method might lead to false positives or false negatives. (2) Instead of using AAL, the proposed method should be compared with other methods when using brain partitions with hundereds/thousands of smaller regions.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Reject
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

13

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The authors propose to combine a number of ideas (of which semi-parametric Bayesian modeling stands out) to estimate heritability from brain MR images. It seems that the major criticism, i.e. Reviewer 1’s on quantitative validation, has been promised by the authors to be addressed through a simulation study, and the computational complexity will also be reported. Provided these new additions, I think the paper is now acceptable for MICCAI.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

4

back to top

A Novel Bayesian Semi-parametric Model for Learning Heritable Imaging Traits