Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Wenhui Lei, Wei Xu, Ran Gu, Hao Fu, Shaoting Zhang, Shichuan Zhang, Guotai Wang

Abstract

Deep learning networks have shown promising performance for object localization in medial images, but require large amount of annotated data for supervised training. To address this problem, we propose: 1) A novel contrastive learning method which embeds the anatomical structure by predicting the Relative Position Regression (RPR) between any two patches from the same volume; 2) An one-shot framework for organ and landmark localization in volumetric medical images. Our main idea comes from that tissues and organs from different human bodies own similar relative position and context. Therefore, we could predict the relative positions of their non-local patches, thus locate the target organ. Our one-shot localization framework is composed of three parts: 1) A deep network trained to project the input patch into a 3D latent vector, representing its anatomical position; 2) A coarse-to-fine framework contains two projection networks, providing more accurate localization of the target; 3) Based on the coarse-to-fine model, we transfer the organ bounding-box (B-box) detection to locating six extreme points along x, y and z directions in the query volume. Experiments on multi-organ localization from head-and-neck (HaN) and abdominal CT volumes showed that our method acquired competitive performance in real time, which is more accurate and 10^5 times faster than template matching methods with the same setting for one-shot localization in 3D medical images.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87196-3_15

SharedIt: https://rdcu.be/cyl1F

Link to the code repository

https://github.com/HiLab-git/RPR-Loc

Link to the dataset(s)

https://structseg2019.grand-challenge.org/Home/

http://www.imagenglab.com/wiki/mediawiki/index.php?title=2015_MICCAI_Challenge

https://wiki.cancerimagingarchive.net/display/Public/Pancreas-CT

Reviews

Review #1

Please describe the contribution of the paper
- To reduce the demand on human annotation, this paper propose a one-shot localization method for organ localzation
- The authors propose a coarse-to-fine framework based on Pnet for more accurate locatization.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- the motivation is straightforward and easy to understanding.
- The idea of localzation by the relative position is novel.
- the experiment is sufficient to prove the effectiveness of the propsoed method.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- the train strategy is complex, it contain two steps. They train two models respectively.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
- one of the dataset is avaiable.
- the two-stage training stategt is not easy to do reproducibility experiment exactly.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- more badly case analysis
- The experiment compared with fully-supervised method is necessary, althought the perfermence is worse, the reason is worth to discuss.
- The authors cloud try to train the model end-to-end by control the epoch_id(it maybe a engineering problem.)
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- the idea is simple and straightforward.
- the sufficient experiment to prove the effectiveness of proposed method
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Review #2

Please describe the contribution of the paper

A novel contrastive learning method which embeds the anatomical structure by predicting the Relative Position Regression (RPR) between any two patches from the same volume
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

1) A novel contrastive learning method which embeds the anatomical structure by predicting the Relative Position Regression (RPR) between any two patches from the same volume; 2) An one-shot framework for organ and landmark localization in volumetric medical images.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

some details need to be improved
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

reproducibility
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
1. The description of the performance of the experimental results in the abstract and the introduction is the same. “Experiments on multi-organ localization from head-and-neck (HaN) and abdominal CT volumes showed that our method acquired competitive performance in real time.”
2. The word “agent” is mentioned many times in the article. Although I can roughly understand its meaning, can you specify what it means?
3. The evaluation metrics in this article are only briefly mentioned, can you specify them through formulas? Also, using only two evaluation metrics may not be so convincing.
4. In the experiment of “Performance of organ localization and multi-run ensemble”, the number of MREs for the “Diagonal” strategy is only 1, and the number of MREs for the “Extreme” strategy is 1 to 20. The number of MREs for the “Diagonal” strategy should also be 1 to 20. This is not very rigorous.
5. In the supplementary figure 1, the height of the graphics with N=32, N=64, and N=128 is not vivid and beautiful, and the height can be adjusted appropriately according to the numerical value. Also, try not to show specific formulas in the figure. You can use images and symbols to indicate.
6. In Figure 2, the color of the box can be briefly explained in the corners of the figure. The content in the box can only be represented by symbols, and the image can be more varied and more complicated. 7.. In Figure 4, the effect of the prediction result compared with the ground truth can be seen, but it is not intuitive enough. The corresponding IoU should be noted on the graph. The combination of number and shape is more intuitive.
7. In this paper, there is no need to have an explanation for every symbol that does not appear once. You can make a table at the beginning of the second chapter to sort out the meaning of all symbols.
8. In this paper, the coarse and fine model Mc, Mf are trained separately. You set rc= 300, rf= 30 for HaN and rc= 700, rf= 50 for pancreas. This paper did not choose rc, rf through experiments. The best value can be found by setting different rc and rf comparative experiments. This can enrich the experiment and make the parameter settings more convincing.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

novelty is sufficient
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Somewhat confident

Review #3

Please describe the contribution of the paper

The paper aims to propose a new method for object localization in 3D medical images. The method is based on relative position regression in a contrastive learning strategy. It can learn a projection network that predicts the relative offset of two image patches. During inference, a support image can help to locate the desired position in the testing image. The method is validated on a head and neck CT dataset and a pancreas CT dataset.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The proposed method is novel and does not require ground truth during training and thus scalable easily. The experimental results show significantly better performance than other one-shot detection settings and comparable to a state-of-the-art supervised localization method. The paper is well presented. The results are promising with significant speedup.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. More discussions about the assumptions and limitations of the proposed method should be added. I think the method assumes that the training and the testing volumes in the same population with the same or similar properties, like body size and orientation. Perhaps, this is the reason that the results of the pancreas are much worse than brain structures, especially compared with the Retina U-Net, considering larger variability of abdominal organs?
2. Some experimental details are missing. Is the validation set only used during inference? Is it used in training to select a model? During training, are the query and support patches sampled under uniform distribution across the entire image volume? How many pairs are sampled in each epoch?
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The method is described clearly and not very complicated. It should be easy to reproduce.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

It would be very interesting to see more results for organs or structures with large variant in shapes and pathologies in the future work.

Some sentences need to be improved or corrected. For example, I don’t understand this sentence in the Experiments section (which maybe incomplete?): “Despite FM Cosine slightly outperformed our method in terms of IoU of the right parotids, its performance is much lower on the brain stem, and need to list the scores.”
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The method is novel, fast, and scalable. But there are some missing details and assumptions.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

All three reviewers recommend ‘accept’.

The authors are encouraged to absorb ideas from non-DL methods, which are very relevant: Zhou, Shape regression machine and and efficient segmentation of left ventricle endocardium from 2D B-mode echocardiogram, Medical Image Analysis, 2010 Criminisi et al. Regression forests for efficient anatomy detection and localization in CT studies, 2010.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

1

Author Feedback

We thank the reviewers for their insight and supportive comments. Based on their suggestions, we will make a more clear and comprehensive analysis of our framework in the future journal version.

back to top

Contrastive Learning of Relative Position Regression for One-Shot Object Localization in 3D Medical Images