Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Abdullah-Al-Zubaer Imran, Sen Wang, Debashish Pal, Sandeep Dutta, Bhavik Patel, Evan Zucker, Adam Wang

Abstract

With the rapid increase of CT usage, radiation dose across patient populations is also increasing. Therefore, it is desirable to reduce the CT radiation dose. However, the reduction in dose also incurs additional noise and with the degraded image quality, diagnostic performance can be compromised. Existing routine dosimetric quantities are usually based on absorbed dose within cylindrical phantoms and do not appropriately represent the actual patient dose. More comprehensive dose metrics such as effective dose require estimation of patient-specific dose at an organ level. Unfortunately, currently available systems are quite far from achieving this goal as well as limited by a number of manual adjustments, time-consuming and inefficient procedures. To overcome all these challenges in achieving the goal of patient safety through reduced dose without compromising image quality, we devise a fully-automated, end-to-end deep learning-based solution to perform real-time, patient-specific, organ-level dosimetric prediction of CT scans. Leveraging the 2D scout (frontal and lateral) images of the actual patients, which are routinely acquired prior to the CT scan, our proposed Scout-Net model estimates the patient-specific mean dose in real-time for six different organs. Our experimental evaluation on real patient data demonstrates the effectiveness of our Scout-Net model not only in real-time dose estimation (only 11 ms on average per scan), but also as a potential tool for optimizing CT radiation dose in specific patients.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87202-1_47

SharedIt: https://rdcu.be/cyhQ2

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The author presents an innovative CT dose prediction method based on deep learning. The dose prediction can be completed using only the Frontal slice and Lateral slice. Compared with the gold-standard MC-GPU algorithm, it is faster while no 3D CT scan data is required, which reduces the radiation damage to the patient. Experiments conducted on a dataset show that the algorithm can execute a fast measurement prediction while ensuring that the error of the result of the MC-GPU algorithm is within a certain range.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Novelty: In the field of dose prediction, the author proposed for the first time that only Frontal slice and Lateral slice combined with deep learning technology are used to complete the prediction, which has very good research significance; Applicability: Compared with the gold-standard MC-GPU algorithm, this algorithm is faster and has less radiation to patients; Article structure: The article is clearly expressed, and the principle is clearly expressed, the module description is detailed, as well as the experiment introduction is sufficient;
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Expression: Some expressions are doubtful: “body/organ-specific and organ/body-specific expressions are inconsistent”; Experimental dataset: The dataset is small, and cross-validation is not performed, the experimental results are not convincing enough; Experimental results: The segmentation results of some organs shown in the supplementary material need to be further improved; compared with the gold-standard results, the dose prediction results have certain errors and need to be further improved; in addition, the prediction performance of other dose prediction methods is not shown, and the exact performance of this method cannot be measured.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

According to the author’s description, the dataset and code are not provided, therefore the experimental results cannot be reproduced.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Experiment: The author proposes a new idea for CT dose prediction, but the experimental part of the article needs to be further strengthened. It is necessary to increase the size of the dataset, perform cross-validation, and verify the performance of the proposed algorithm on different datasets. At the same time, due to the unsatisfactory results of partial segmentation and the deviation of dose prediction results, the performance of the algorithm needs to be strengthened. And it is necessary to increase the performance comparison with other prediction methods, and to compare the performance of the proposed model more comprehensively.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Innovation， writing
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Review #2

Please describe the contribution of the paper

The papers introduces a new method for patient-specific organ level dose estimation in CT imaging, using scout images only. The methods is fully based on two scout projection (front/lat) and includes 2D detection and segmentation of the images/organ detection. Subsequently, organ specific dose is estimated taking configuration of the scanner into an account (e.g. filter, source distance, pitch, kVp)
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

It is a work of high clinical importance and technical novelty. The main advantage of the suggested method is that it allows for prospective organ-specific dose estimation prior to actual start of CT-acquisition. This could enable prospective CT protocol optimisation capabilities and, probably, would be way more effective dose reduction tool, when compared to CT image voxel-based metric (e.g. retrospectively register the dose, do not facilitate prospective dose optimization / reduction)
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The main challenge of the work is actual evaluation of estimated dose reports and future scalability of the method. Evaluation dataset was rather small (14 test cases), also there was no information on the range of patient anatomy /implant variation that were included into this database. Information and results supporting claims about accurate organ-level segmentation is missing. All results were pushed to supporting information, probably to allow for more space for textual description of the methods. Authors have used readily available simulator for GE scanner (rather simplified one). How realistic are these estimations and how they plan to keep up with new releases of CT scanners or transitions to scout-based dose estimation for other manufacturers?
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
- The work is reproducible, yet some aspects /details of the network are difficult to visualize from the text.
- Also, there is little information about organ segmentation part of the network
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
It is a nicely written work and I have enjoyed reading it. I have only a few minor remarks or suggestions
- The work is reproducible, yet some aspects /details of the network are difficult to visualize from the text. For example, authors spend a lot of time describing network architecture, while this information could have been way easier precepted in a graph (it could go to SI to save space). Fig 1 of the manuscript does give rather unclear idea about the structure of the network, but it does not contribute in any way to reproductivity of the network design.
- Information and results supporting claims about accurate organ-level segmentation is missing. All results were pushed to supporting information, probably to allow for more space for textual description of the methods. It is important to include and discuss those results in the manuscript itself (eg figure 3-4, Table 1 in SI).
- Evaluation dataset was rather small (14 test cases), also there was no information on the range of patient anatomy /implant variation that were included into this database. I think that it is important to evaluate the method on a larger cohort, just check when the method will fail (e.g. difficult organ edge detection for a specific patient category)
- Authors have used readily available simulator for GE scanner (rather simplified one). How realistic are these estimations and how they plan to keep up with new releases of CT scanners or transitions to scout-based dose estimation for other manufacturers? It is important to consider multi-vendor comparison of the method, considering that it was trained and tested using GE data only. This might help to pick-up plausible limitation /pitfalls. Also, I wonder how you would integrate more advanced Automatic Exposure Control methods into your calculations.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

High clinical importance, plus the authors have clearly showed that it is not required to reinvent the wheel to facilitate prospective organ-specific dose estimation. They’ve used readily available simulator in a smart way, and have developed only relevant novel blocks to fill in the gaps in the dose calculation pipeline. As a results, this work has way higher chances to be actually clinically integrated in the nearest future, when compared to a totally new methods for CT dose estimation.
Additionally, prospective organ dose calculation, if implemented into the clinic, will allow for significant image quality and dose optimisation
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

4
Reviewer confidence

Very confident

Review #3

Please describe the contribution of the paper

The paper develops a neural network (Scout-Net) to estimate organ doses from CT scans, using scout images and CT acquisition ranges as inputs. The method potentially allows real-time organ dose estimation before CT scan actually takes place, which enables potential on-the-fly CT parameter adjustment to balance radiation dose and image quality.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The idea of estimating organ doses from paired scout images is novel and interesting. The results also demonstrated the feasibility of this idea through a convolutional neural network.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Although the study results looked promising, there is no in-depth analysis of the results, especially on its robustness against patients of different sizes of anatomies. In addition, it remains unclear if the network only works for a certain anatomical site and how robust it is towards the scan range variations.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

NA
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

This is an interesting study. However, more analysis is required, especially on the network’s robustness to varying sizes of patient anatomy and organ locations, as well as the scanning ranges of the CTs.
Please state your overall opinion of the paper

Probably accept (7)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The study presents interesting ideas to estimate doses directly from simple scout images. It can potentially change the current dose reporting scheme if the network developed can be proved robust to different scenarios.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

All three reviewers concur in their assessment to accept the paper that I also follow. I encourage the authors to implement the reviewers’ suggestions for the final paper, if possible.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

4

Author Feedback

We would like to thank all the reviewers for their time in reviewing our paper. We are glad the reviewers appreciated the novelty and significance of our work aimed at predicting CT organ doses from scout images. We are also happy that the reviewers enjoyed reading our paper. As all three reviewers agreed, ours (Scout-Net) is the first ever prospective, patient-specific CT organ dose estimation method from simple lateral and frontal scout views leveraging deep learning. Moreover, the proposed method can make predictions of patient-specific mean body and organ doses in real-time, without requiring expensive computing resources. This will enable our model to be incorporated in the relevant scanner systems for realistic dosimetric quantifications even before performing the actual CT scans and as a result could potentially reduce high radiation exposure for children/vulnerable patients as well as for the most sensitive organs.

We leveraged the gold standard Monte Carlo dose calculation engine (MC-GPU) to configure the bowtie filtration and anode heel effect, for realistic simulation of GE’s Revolution CT scanner. MC-GPU’s x-ray physics generates accurate projections and dose maps of patient-specific voxelized volumes, as validated independently on an AAPM reference phantom. We have also confirmed the realistic simulations by comparing the dose maps to physical measurements in a separate study. We established the ground truth mean organ doses by applying the 3D AutoSeg tool to the MC-GPU generated dose maps. Since we are only interested in the average dose (mGy), segmentation performance can be tolerated a bit higher than other segmentation-based procedures. This was experimentally verified by Offe et al. SPIE MI 2020. Nevertheless, our ongoing effort focuses on improving the segmentation through improving the algorithm and training mechanisms.

In order to validate the proposed Scout-Net model, we collected paired scouts/CT data of adult body scans acquired from Revolution CT scanners. The dataset includes a broad cross-section of patients of different size (DW coverage for small to large adults [AAPM TG220]) and conditions at a major academic center. The scan ranges also vary in anatomical site: chest, abdomen, and/or pelvis. We continuously put our effort into collecting more data (from 64 scans up to 112 now). Even with the small dataset, our reported dose prediction results are still promising. We performed the evaluation after 10 runs for each of the variations of our proposed model and tabulated the mean (stdev) relative errors. The model was found to be consistent in predicting organ doses for different scan ranges. We are currently working on developing a baseline to compare against, albeit there was no prior prospective (scout-based) organ dose prediction method and we thought comparing against retrospective (CT-based) estimation would be unfair. In our latest results with increased data size, we have observed improved performance. For example, the prediction error for body dose has now come down to below 6% (vs 7.87%). For further generalizability and robustness analysis, we are collecting similar paired data from a separate institution. In terms of scalability and transitions to new scanner releases or other vendors, we could foresee minimal adjustment in our model and possibly separate versions.

Taking the reviewers’ suggestions into considerations, we are implementing the following changes for the final paper: 1) fixing the doubtful expressions to make sure all are consistent; 2) moving back the segmentation results (table and visualization) from the supplemental; 3) reporting the cross-validation results for the model; 4) reporting the dose prediction results on larger dataset (paired dataset of 112 scans); 5) comparison against our newly developed baseline method; 6) more in-depth performance analysis of the proposed Scout-Net model.

back to top

Personalized CT Organ Dose Estimation from Scout Images