Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Kristine Aavild Juhl, Xabier Morales, Ole de Backer, Oscar Camara, Rasmus Reinhold Paulsen

Abstract

The task of 3D shape classification is closely related to finding a good representation of the shapes. In this study, we focus on surface representations of complex anatomies and on how such representations can be utilized for super- and unsupervised classification. We present a novel Implicit Neural Distance Representation based on unsigned distance fields (UDFs). The UDFs can be embedded into a low-dimensional latent space, which is optimized using only the shape itself. We demonstrate that this self-optimized latent space holds important global shape information useful for reconstructing the anatomies, but also that unsupervised clustering of the latent vectors successfully separates the three different anatomies (left atrium, left/right ear-canals and human faces). Finally, we show how the representation can be used to do gender classification of human face geometries, which is a notoriously hard problem.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87196-3_38

SharedIt: https://rdcu.be/cyl2I

Link to the code repository

https://github.com/kristineaajuhl/Implicit-Neural-Distance-Representation-of-Complex-Anatomie

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

Following the recent trend in computer vision and machine learning, this paper proposes to model (anatomical) shapes in an implicit way by using distance functions represented via neural networks (neural distance functions). While most papers in the literature deal with signed distance functions, this paper comes up with a framework that uses unsigned distance functions. The general advantage of unsigned distance functions over their signed counterparts, is that they are potentially able to represent shapes with arbitrary topologies instead of only watertight ones. The proposed method is applied to three different data sets (faces, ear canals, and cardiac shapes) and results are presented for an unsupervised, general shape clustering task as well as face-based gender classification.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Neural distance functions applied to anatomical shapes
- Unsigned distance functions that allow its use in medical image analysis where shapes are frequently not watertight
- Method to reconstruct the iso-surface from its implicit representation
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Limited novelty as approaches for regressing unsigned distance functions instead of signed ones have been proposed before and it is hard to assess the novelty of the iso-surface reconstruction part
- Evaluation scenarios lack real world relevance
- Structure of the paper could be improved (Sec. 2 lacks details wrt. to the novelty of the presented method while Sec. 3 is too long)
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Evaluation details and data sets being used are sufficiently described and the authors promise to make their code publicly available through github upon acceptance.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

In general, I agree with the authors that implicit shape representations via neural network-based distance functions are a nice way to represent anatomical shapes. I also agree that for widespread use in our field, such a method needs to be able to represent non-watertight shapes/shapes with difficult topologies as many shapes of clinical relevance do not exhibit a simple structure. Hence, using unsigned distance functions totally makes sense here.

The main problem I see with this paper is related to its novelty as I do not really see any substantial novel contributions. The auto-encoding structure used to learn the implicit shape representations is similar to [15] with the only difference being the switch from signed to unsigned distances and the authors do not discuss how this approach differs from the NeurIPS 2020 work on usigned distance functions [9]. I do not see a real difference here. Furthermore, the zero-level surface extraction method presented in this paper and which is claimed to be a novel contribution also seems to be heavily influenced by what is being proposed in [9]. While this aspect is somewhat acknowledged by the authors (e.g., ‘our approach, inspired by [9]’), I miss a detailed discussion that explicitly states the differences. The part of Sec. 2 describing the zero-level surface extraction method is also much too short to really assess its novelty.

Even in the absence of any methodological novelty, I would still be happy to recommend its acceptance if the approach was outperforming state-of-the-art shape modeling approaches in real-world medical image analysis scenarios. However, in my mind, the data sets being used here (faces, ear canals, and cardiac shapes) in conjunction with the evaluation tasks chosen (clustering and face-based gender classification) are not really useful in assessing the real benefits of this methods for medical image analysis purposes and the results are also not really satisfactory. Why wasn’t the approach utilized to solve a real world face classification from our field like genetic syndrome classification? As another option, one could have used (publicly available) cardiac shapes to, for example, perform cardiac phase clustering.
Please state your overall opinion of the paper

borderline reject (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

In my mind, the novel technical contributions are minimal and the evaluation lacks relevance.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Review #2

Please describe the contribution of the paper

In this paper, the authors propose a method to represent complex anatomical shapes, of various topologies, for shape classification. The method optimizes a latent space, in which global shape changes can be clustered using k-means on principal components.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Main strength: Authors experimented on different datasets with different level of noise, offering an adapted validation of their method potential. They also compared their approach to several other deep learning approaches.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Main weakness: The introduction do not present well the motivation of using a geometric deep learning approach. I think this is an important point, as there exist methods able to cluster easily, without training, left and right anatomical structures of different topologies (even within the population).
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Page 2, the authors state that the code is available on github (link will be given at publication step).
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- Authors should say a word about what exactly the clamp function does. Would it be possible to use indicator function instead ?
- Surfaces of all datasets are rigidly scaled to fit into a unit sphere. Then different kernel sizes are used to add noise. What is the unit of the sphere regarding to the mm measure of kernel sizes ? (i.e. which percent of the unit sphere diameter represent the kernel sizes?)
- Careful with the numbers notations as it could be misleading : 250.000 samples, 16.384 points, 1.129 mm,… it should be 250 000 samples, 16 384 points, 1.129 mm (I guess)
- How did you chose the number of dimension for the latent space ?
- In results, you could add computation time and memory information, to compare to the 3D extension of CNN for 2D images (cf introduction).
- I think it would be interesting to compare the latent space optimized here, with the tangent space of a LDDMM (widely recognised as efficient for anatomical shape analysis) approach using varifold representation (no topology issues, no point to point correspondences), and see what does those two latent space captures (same information, different features, how many dimension,… ?)
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

I think this paper cannot be accepted without a detailed motivation/explanation for the use of deep learning approach, as classifying left and right structure is not powerful enough. Anatomical shape analysis methods can classify pathologies or predict its severity, so left vs right side is not very relevant to me, except is a good motivation is given.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

3
Reviewer confidence

Very confident

Review #3

Please describe the contribution of the paper

The paper proposes and investigates the use of unsigned distance fields and a neural network modeling approach for shape analysis of complex shapes. They evaluate the approach with several different biomedical datasets and measure performance compared to several related approaches.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

This approach is quite interesting, as it has the advantage of working flexibly with a wide range of shapes. The paper is well-written, and the methods are sensible. The authors appropriately consider past works and compare methods in the experiments.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

I was left wondering about the design of the neural network and how certain parameters were chosen, e.g. the number of units and hidden layers. Were there empirical tests done to pick these values, or were there technical limitations for memory? I imagine some of these could be important, particularly the 3D grid resolution, which may miss small features if it is not sufficiently high resolution.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors have released their code; however, the data is not publicly available (some was from a hospital though).
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

I think one more good example of how this might be useful is looking and unstructured triangle soup shape models, which would be better handled than by signed distance functions. I also think it might help the paper to comment more directly on the differences with the two closest related works:

Venkatesh, R., Sharma, S., Ghosh, A., Jeni, L., & Singh, M. (2020). DUDE: Deep Unsigned Distance Embeddings for Hi-Fidelity Representation of Complex 3D Surfaces. arXiv preprint arXiv:2011.02570.

Chibane, J., Mir, A., & Pons-Moll, G. (2020). Neural unsigned distance fields for implicit function learning. arXiv preprint arXiv:2010.13938.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper proposes a well-motivated approach and demonstrated good performance with realistic datasets. I think my score would be higher if there were some standardized benchmarks that could have been used to give a sense of how the method performs in comparison with other approach (besides the face classification).
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

4
Reviewer confidence

Confident but not absolutely certain

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

Reviewers raised concerns about the motivation for using deep learning for detecting complex anatomies. For e.g. the experimental results contain detection of the ears (left vs right) or Male vs Female, which are not focused on a specific application.

Concerns were also raised about various aspects of the novelty in the paper. For e.g. the signed distance representation (Chibane, J., Mir, A., & Pons-Moll, G. (2020). Neural unsigned distance fields for implicit function learning. arXiv preprint arXiv:2010.13938.) etc.

A strength of the paper was the code will be publicly released.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

5

Author Feedback

We would like to thank the reviewers for relevant and constructive comments on our manuscript. The main issues that were raised are addressed in the following bullets:

Limited novelty in comparison to related work The proposed method borrows ideas from multiple related works using neural implicit functions. These works, however, solely focus on reconstruction of 3D shapes from a low-resolution or noisy input. Our paper instead focus on shape classification and description using the latent space that are optimized using no guidance except the shape itself. Compared to [9] our method works directly on the mesh with no need to voxelize the input and loose important information. The method in [9] also does not define a single latent space holding local and global shape information. The method in [24] makes use of a similar network architecture as ours, but learns a combination of distance fields and normals. The main focus of their paper was to reconstruct known 3D shapes, and only a short comment is made on the ability to represent a class of shapes. We tried to make these points clear in the introduction, but will make an effort to emphasize them more clearly in the final paper. To do so we might slightly shorten Section 3 to also improve on the structure as suggested by Reviewer 1.

Evaluation scenarios lack real world relevance To the best of our knowledge, we are the first to make use of neural implicit functions to represent complex anatomical/biomedical shapes. These datasets often have few examples to learn from, while being complex geometries with large number of vertices, noise, etc.. We recognize that the reviewers long for relevant and complex clinical scenarios and we see a future of many interesting application in i.e. Left Atrial Appendage morphology as suggested in the discussion section. The use cases in this paper are however carefully chosen to demonstrate the core features of the method on tasks with a clear ground truth and objective. We believe the chosen scenarios demonstrate that highly variable shapes can be represented with a few features and that these features contain important global shape information useful for difficult shape classification tasks.

Limited novelty in the surface reconstruction method As stated above, the main contribution of this paper is not the ability to reconstruct surfaces, but the ability to describe complex anatomies with global shape parameters. We have decided to include some information about the reconstruction, as it is an important tool to understand and visualize the learned features. The main difference to [9] is the initialization using a thin marching cubes envelope, which allows us to estimate good normal without explicitly learning them as in [9]. A short sentence on this will be added to the methods section.

Reasons to use geometric deep learning Geometric deep learning is a rapidly increasing field showing potential for impact in biomedical image analysis. The proposed method shows that complex biomedical data can be represented in a deep learning framework and thereby make use of deep learnings ability to learn complex patterns. Many methods from traditional shape analysis could be compared. The focus of this paper was to demonstrate the principles in this method, while evaluating it on complicated biomedical tasks will hopefully be done by others and us in the future.

Details on design choices, network architecture etc. Parameters such as number of hidden units, hidden layers, etc. were chosen based on a trade-off between performance, memory limitations and training times, while also being inspired by design choices made in the related work. Details on these validations are omitted due to limitations on paper length, but all source code and parameter sets will be publicly available to ensure complete reproducibility.

We hope this rebuttal clarified the main issues and that you will consider accepting the paper for MICCAI2021.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper shows limited novelty in the methods.

In the rebuttal the authors say their method is 3D shapes from a low-resolution or noisy input. However, they show limited experimental results and don’t show how their method fares in comparison with other reconstruction techniques. Thus it is difficult to evaluate Table 1 as it only presents the authors’ method.

In Table 2, where the authors show experimental results on gender classification, the neural network solution vastly outperforms k-means, which is expected. However, the results from approaches in [1] and [2] seems better.
1. Abbas, H., Hicks, Y., Marshall, D., Zhurov, A.I., Richmond, S.: A 3D morpho- metric perspective for facial gender analysis and classification using geodesic path curvature features. Computational Visual Media pp. 17–32 (2018)
2. Abbas, H.H., Altameemi, A.A., Farhan, H.R.: Biological landmark vs quasi- landmarks for 3D face recognition and gender classification. International Journal of Electrical and Computer Engineering (IJECE) pp. 4069–4076 (2019)
There’s also an SVM based approarch. Han, Xia, Hassan Ugail, and Ian Palmer. “Gender classification based on 3D face geometry features using SVM.” 2009 international conference on cyberworlds. IEEE, 2009.

Although I understand that some of them involve more sophisticated mathematical machinery, comparisons become difficult.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

14

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

To some extent, I agree with R1’s concerns that the technical novelty in this work is incremental, and that the presented examples do not amount to a “killer application”. Despite this, I think the idea of implicit neural distance representations of shapes is fresh enough in the MICCAI context, and the case for its potential utility in our community is convincing enough, that I support acceptance, especially when accounting for the fact that we do not have a strict limit on the number of papers that can be presented at this year’s virtual event.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

7

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

This work proposed to model anatomical shapes in an implicit way by using distance functions represented via neural networks. The work proposed to use unsigned distance functions, to be tolerant to various topological structures. The proposed method was applied to three different datasets.

The work is overall interesting and well suited for shape analysis in medical image analysis. The weakness is that the current work may be further developed on more clinically important problem. The AC thinks the potential overweights the current weakness and therefore, makes an “Accept” recommendation.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

11

back to top

Implicit Neural Distance Representation for Unsupervised and Supervised Classification of Complex Anatomies