Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Kang Zheng, Yirui Wang, Xiao-Yun Zhou, Fakai Wang, Le Lu, Chihung Lin, Lingyun Huang, Guotong Xie, Jing Xiao, Chang-Fu Kuo, Shun Miao

Abstract

Bone mineral density (BMD) is a clinically critical indicator of osteoporosis, usually measured by dual-energy X-ray absorptiometry (DEXA). Due to the limited accessibility of DEXA machines and examinations, osteoporosis is often under-diagnosed and under-treated, leading to increased fragility fracture risks. Thus it is highly desirable to obtain BMDs with alternative cost-effective and more accessible medical imaging examinations such as X-ray plain films. In this work, we formulate the BMD estimation from plain hip X-ray images as a regression problem. Specifically, we propose a new semi-supervised self-training algorithm to train a BMD regression model using images coupled with DEXA measured BMDs and unlabeled images with pseudo BMDs. Pseudo BMDs are generated and refined iteratively for unlabeled images during self-training. We also present a novel adaptive triplet loss to improve the model’s regression accuracy. On an in-house dataset of 1,090 images (819 unique patients), our BMD estimation method achieves a high Pearson correlation coefficient of 0.8805 to ground-truth BMDs. It offers good feasibility to use the more accessible and cheaper X-ray imaging for opportunistic osteoporosis screening.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87240-3_4

SharedIt: https://rdcu.be/cyl5w

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

This paper aims at estimating bone mineral density (BMD), usually evaluated using Dual-Energy X-ray Absorptiometry (DEXA), from plain radiographs as a regression problem. The idea is original and interesting.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The novel and original aspect of this paper is the use of classical plain radiographs to estimate Bone Mineral Density (BMD). BMD is usually evaluated using Dual-Energy X-ray Absorptiometry (DEXA). DEXA is a low X-ray modality which is not very common. Especially in developing countries.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The paper is quite well written but lacks of details. Data are not well described. The literature is abundant on the subject. Bibliographic references should be improved.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Some mathematical formulas support the proposed strategy. Experiments are achieved on an in-house dataset which lacks description in the paper. The code is not available.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

This paper aims at estimating bone mineral density (BMD), usually evaluated using Dual-Energy X-ray Absorptiometry (DEXA), from plain radiographs as a regression problem. The idea is original and interesting.

The paper is quite well written but lacks details making the proposed strategy difficult to understand. Results should not be presented in the Introduction.

While few references are provided, the literature is abundant on the subject of osteoporosis in connection with machine learning too. Has to be expanded.

Data are not well described, paired hip X-ray image and DEXA measured BMD were taken within six months apart. Six months interval is enough to get changes of the BMD. It should be motivated and clarified. Is the DMO evaluated on the hip? Why 1,090 images from 819 patients?

The Adaptive Triplet Loss is not clear enough. How is achieved the embedding for each sample?

Equation (4) is not well defined as for the first term yn is used twice. Also, yp was not defined. It should be fixed.

In Algorithm 1, how is fixed the total training epoch, E.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- Originality of the idea
- Proposed methods to realize the proposed idea
- Achieved results
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Very confident

Review #2

Please describe the contribution of the paper

This paper proposes a framework for semi-supervised learning to predict bone mineral density (BMD) from hip X-rays. The paper has two contributions: the clinical aspect (BMD prediction has clinical value), and also a modification of the triplet loss to encourage embeddings of X-ray images to be close when the BMD values do not differ.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Good application
- Well written paper
- Triplet loss trick is very interesting
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Reporting of the results must be improved. The authors needed to have run the experiments multiple times, and report the standard error over different seeds (so as the means). Currently, the ablation study is not very convincing.
- I think that the paper is re-inventing interpolation consistency training in the context of triplet loss.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The paper is reproducible
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

I think that the paper is re-inventing interpolation consistency training, but in the context of triplet loss. The authors should have done more rigorous argumentation on why this idea of enforcing closeness of embeddings is plausible from a theoretical point of view. However, the paper is still good, and I will be willing to change my score to definite accept as soon as the results are properly reported (means and standard errors in the tables, all other numbers with confidence intervals, please).
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The developed methodology has a useful inductive bias, which comes from a practical problem. I believe that this is what MICCAI papers need to really show: how one uses domain knowledge to create better methods for medical imaging computing and computer assisted intervention.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

6
Reviewer confidence

Confident but not absolutely certain

Review #3

Please describe the contribution of the paper

Authors proposed a novel semi-supervised framework to predict BMD from hip X-ray images. They evaluated their pseudo BMD values versus ground truth esimtation for DXA measurements on a fairly large dataset.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The task of opportunistic screening for osteoporosis is of interest.
- The method backbone is based on VGG11 but integration with adaptive triplet loss is novel. The idea is to enforce samples with similar features yield similar BMD scores.
- Semi-supervised learning with unlabelled data was also an interesting idea in this study.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The main weakness is in the interpretation of results. Fig. 2 shows a scatter plot for ground truth BMD versus predicted pseudo BMD values. An overall correlation is evident with R>.85 but this is not enough. The measurements should be unbiased as well which has not been discussed at all. Utilising a Bland-Altman plot could help here. From Figure 2, low BMD values are overestimated. This means that osteoporotic patients with low BMD may not be diagnosed well.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Data is not public and codes are not available but enough details are provided to reproduce similar results.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- Introduction, Paragraph 2, what was the ‘undesired prediction performance’ of previous techniques based on CT?
- T-score is a linear transformation of BMD values: T = (BMD-mu)/sigma where mu and sigma are the average and standard deviation of BMD for healthy young cohort. Osteoporosis definition is currently based on T-score below -2.5. If possible please report predicted T-score as well and report the sensitivity and specificity of osteoporotic cases based on pseudo scores. This is essential to assess the clinical significance of the method.
- What is the expected BMD change in six months? You may discuss how the expected change compares with the RMSE of the framework.
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The manuscript proposed novel methodological contributions but results requires further clarification to assess the merits of the work.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

4
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

The idea of triplet-loss for semi-supervised training presented in this paper is found very relevant and quite interesting by all the reviewers. However, several shortcomings with the presentation were pointed out, with suggestions for improving these, which the authors should integrate to improve the quality.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

4

Author Feedback

N/A

back to top

Semi-Supervised Learning for Bone Mineral Density Estimation in Hip X-ray Images