Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Sebastian Pölsterl, Christina Aigner, Christian Wachinger

Abstract

Deep Neural Networks (DNNs) have an enormous potential to learn from complex biomedical data. In particular, DNNs have been used to seamlessly fuse heterogeneous information from neuroanatomy, genetics, biomarkers, and neuropsychological tests for highly accurate Alzheimer’s disease diagnosis. On the other hand, their black-box nature is still a barrier for the adoption of such a system in the clinic, where interpretability is absolutely essential. We propose Shapley Value Explanation of Heterogeneous Neural Networks (SVEHNN) for explaining the Alzheimer’s diagnosis made by a DNN from the 3D point cloud of the neuroanatomy and tabular biomarkers. Our explanations are based on the Shapley value, which is the unique method that satisfies all fundamental axioms for local explanations previously established in the literature. Thus, SVEHNN has many desirable characteristics that previous work on interpretability for medical decision making is lacking. To avoid the exponential time complexity of the Shapley value, we propose to transform a given DNN into a Lightweight Probabilistic Deep Network without re-training, thus achieving a complexity only quadratic in the number of features. In our experiments on synthetic and real data, we show that we can closely approximate the exact Shapley value with a dramatically reduced runtime and can reveal the hidden knowledge the network has learned from the data.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87199-4_41

SharedIt: https://rdcu.be/cyl4p

Link to the code repository

https://github.com/ai-med/SVEHNN

Link to the dataset(s)

http://adni.loni.usc.edu/

Reviews

Review #1

Please describe the contribution of the paper

This paper proposes a heterogeneous neural network based on Shapley Value for explaining the Alzheimer’s diagnosis made by DNN. To leverage the exponential time complexity of the Shapley value, it transforms a given DNN into a Lightweight Probabilistic Deep Network, eliminating the complexity from exponential time to quadratic time.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

This paper employs a solid theory, i.e., Shapley Value, for network explanation. Further, it gives an approximate manner to solve the high complexity. The topic is interesting and is of importance. The paper is also well organized and described.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. The authors solely take the left hippocampus as the input of PointNet and do not describe the rationality.
2. Fig. 2 and 3 are not well explained. From Fig. 2, we can just see that each biomarker has more than one Shapley values, but their specific meanings are not well described. For Fig. 3, how are these values (like +3.95, +1.44) computed?
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
I guess it is a little difficult to re-implement the results because of the following reasons.
1. The creation of the synthetic dataset in Section 3.1 is not introduced.
2. The combination of the tabular data and the output of PointNet is ambiguous.
3. The training details of the network, such as the optimizer, learning rate, are not described.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
1. Please explain the reason that only the left hippocampus is utilized as the input of PointNet and the rationality.
2. It would be better if Fig. 2 and 3 are explained in detail.
3. From Fig. 1, the tabular data is combined with the output of PointNet. However, their specific combining manner is not depicted.
4. In Table 1, the representation of NE is not illustrated. Moreover, how do they get the synthetic dataset?
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The topic is interesting and very important for the clinical trial. The proposed method has fundamental support. However, some details are not well described.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

4
Reviewer confidence

Somewhat confident

Review #2

Please describe the contribution of the paper

The authors propose a method of approximating the Shapley value by converting a pre-existing neural network into a probabilistic version of itself. The authors apply the method to Alzheimer’s disease prediction using ADNI data.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The authors propose a novel method for approximating the Shapley value from a probabilistic neural network. This method allows the neural network to explain the contribution of its final output given its input. The authors demonstrate that this method can be performed on
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

While the mathematical details regarding the derevation of the SVEHNN may be complete, the paper is dense and difficult to understand.

The validation on the ADNI dataset is interesting but requires further detail to demonstrate that the Shapley value approach coincides with clinical findings.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The paper does not seem to include enough details to be reproducible and does not indicate whether code will be released. The ADNI dataset is public.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

As the method requires a good deal of computation, the authors should include timing information regarding the inference process.

While the assumption that inputs are independent normal distributions may be necessary for the math to work, it seems like locations in the hippocampus will not be independent.

Figure 2 shows hippocampus Shapley values that greatly exceed those of the other biomarkers. There is no discussion of why this is the case.

Figure 3 (left) is hard to interpret and has no description..

It is unclear what the outputs of the probabilistic neural network are and how they are used in computing the approximate Shapley value. For example, mu_k is not defined or well explained in the text and seems essential for calculating the expectation in (3).

Some description of the Shapley sampling and Occlusion baselines should appear in the text to allow readers to understand the gains offered by the SVEHNN.

The experiment section mentions 3 baseline approaches but only 2 are listed. Also it seems that the moving the point to the origin approach is not used.

The range of Shapley values is unclear. Hippocampus values in figure 2 go to 15 while they only go to 1.5 in figure 3 right. This makes the clinical results hard to interpret.

Validation of hippocampus shape classification as in figure 3 right should be shown for multiple patients to validate that the result is reproducible across patients.
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper proposes an interesting approach to Shapley value estimation, a technique for generating importance scores for a set of features. However, the paper is dense and some details appear to be missing from the method. The paper would also benefit from an expanded results section to confirm that the observed trends hold across patients.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Review #3

Please describe the contribution of the paper

The paper showcases a Shapley Value-based approach for (local) explainability and applies it to heterogeneous deep networks using the example of shape- (given as point cloud) and tabular biomarker input. The main contribution consists of the proposed approximation to determine the Shapley value in an efficient way by exploiting the equality of a known approximation method with the replacement of network layers by probabilistic ones as proposed by the Lightweight Probabilistic Deep Networks.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- innovative combination of existing work (approximation method for the Shapley Value, probabilistic layers leveraging aleatoric uncertainty) of different fields to achieve the desired (and necessary) efficient approximation
- good evaluation strategy with synthetic and real-world data
- well written and structured
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- An investigation of cases where the worst errors were observed (e.g. in the synthetic case) would be interesting
- An analysis of the disadvantages or weaknesses of the proposed method is missing
- No ideas for future work and research questions that remain open are provided
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The proposed algorithms and procedures are explained in sufficient detail and backed up by references that should allow the reproduction of the results. Furthermore, the code is supposed to be published at some point. Access to the ADNI dataset can be obtained, while details about the synthetic data set are unclear.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

A few sentences regarding the downsides or weaknesses of the approach would have been great.
Please state your overall opinion of the paper

strong accept (9)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The authors proposed an innovative solution to the problem of explainability of a specific network with heterogeneous input data and evaluated it in an appropriate way.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

3
Reviewer confidence

Confident but not absolutely certain

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

This paper proposes a heterogeneous neural network based on Shapley Value for explaining the Alzheimer’s diagnosis.

The key strengths include: 1) it employs a solid theory, i.e., Shapley Value, for network explanation. 2) it provides an approximate manner to solve the high complexity.

The key weaknesses include: 1) Need more details on how the Shapley value approach coincides with clinical findings on ADNI.

Based on all reviewers’ agreements, I would recommend accept this paper.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

3

Author Feedback

N/A

back to top

Scalable, Axiomatic Explanations of Deep Alzheimer's Diagnosis from Heterogeneous Data