Back to top List of papers List of papers - by topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Shuai Tan, Pin Tang, Xingchen Peng, Jianghong Xiao, Chen Zu, Xi Wu, Jiliu Zhou, Yan Wang
Abstract
Radiation therapy has been widely used in the treatment of cancer. However, a high-quality radiotherapy plan often requires dosimetrists to tweak repeatedly in a trial-and-error manner based on experience, causing it quite time-consuming and subjective. In this paper, we present a multi-task dose prediction (MTDP) network to automatically predict the dose distribution from computer tomography (CT) image. Specifically, the MTDP network consists of three highly-related tasks: a main dose prediction task for generating fine-grained dose value for each pixel, an auxiliary isodose lines prediction task for providing coarse-grained dose range for each pixel, and an auxiliary gradient prediction task for capturing subtle gradient information such as radiation patterns and edges of the dose distribution map, to obtain a more accurate and robust dose distribution map. The three related tasks are integrated via a shared encoder, following the multi-task learning strategy. To strengthen the correlations of different tasks, we also introduce two additional constraints, i.e., isodose consistency loss and gradient consistency loss, to enforce the match between the dose distribution features produced by the two auxiliary tasks and the main task. The experiments conducted on an in-house dataset with 110 rectum cancer patients have demonstrated the effectiveness and superiority of our method compared with the state-of-the-art methods. Code is available at https://github.com/DeepMedLab/MTDP-network.
Link to paper
DOI: https://doi.org/10.1007/978-3-030-87234-2_71
SharedIt: https://rdcu.be/cyl9i
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This paper addresses the creation of an automated dose distribution prediction for radiotherapy from CT pelvic images for rectum cancer. The convolutional neural network method proposed is multi-task, aiming for pixel-wise dose, isodose lines and “texture”. Isodose consistency and “texture” consistency are incorporated as loss terms. The method was evaluated on 110 patients and compared with alternative methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The multitask formulation is novel in this context.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
I don’t think the term “texture” is appropriate for a Sobel map; it is misleading. As a way of emphasizing the edges in the dose distribution map, it appears to have utility.
The differences found between alternatives and the proposed method in the conformity index (CI) and homogeneity index (HI) appear quite small. While statistically significant, these differences may not be practically different. This is also true of the ablation study.
The performance or utility of this approach is not clear as the generated dose distribution is not put to use, for example, to guide automated treatment planning.
In addition to CI and HI, a simple measure of difference (e.g. absolute or squared) should be shown.
The English is poor.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Many of the needed details for reproducing this result are included in the paper. The data is only available in-house.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
A clearer formulation and evaluation in terms of the ultimate clinical goal would make the work more useful.
- Please state your overall opinion of the paper
probably reject (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
While the method is interesting, the results are unimpressive.
- What is the ranking of this paper in your review stack?
4
- Number of papers in your stack
5
- Reviewer confidence
Confident but not absolutely certain
Review #2
- Please describe the contribution of the paper
This paper presents a multi-task dose prediction neural network to predict radiotherapy dose prediction from CT images and delineations of the tumor volume for rectum cancer. The three tasks are voxel-level dose prediction, isodose lines prediction and texture or “gradient” prediction. To improve the learning process, these tasks are linked at the loss level by two additional constraints (on dose map/isodose lines and dose map texture / predicted texture). More precisely derived 2D U-Nets are used with a shared encoder for the three tasks, a self attention module is included for the dose map prediction task. The experiments are conducted on an in-house dataset of 110 rectum cancer patients and the approach is compared to recent related works. The results are promising and show a real improvement over sota.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Smart formulation of the problem as three related tasks (through the constraints) that outperforms sota. Each of the proposed modules contributes to the results as shown by the ablation study. It seems also simpler to implement than other works in the field.
- Paper is well-written and easy to follow.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Ground truth is not described. It seems that several dose maps (as shown in Table 1) are considered as ground truth for each slice (perhaps made by different operators). If it is the case how are the metrics computed. It would have been interesting to study the inter-operator variability.
- It seems everything is done in 2D for computational reasons. Performance metrics should also be given at patient level. Given the physical process producing the real dose map, one could expect a 3D approach to be more adequate.
- Precise validation process is not described which prevents having a sufficient insight on the generalization of the algorithm. It looks like training/test split is performed randomly but no information is given on the number of times it has been done.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
I have some concerns about the reproducibility of the paper although technically-speaking one could reimplement the method without too much difficulty given the paper :
- dataset is not public and GT or image input are not described (sizes, preprocessing, normalisation, …).
- initial weights are not mentioned (but should correspond to Pytorch defaults). Yet it should be clearly stated.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
The approach is elegant and interesting but the paper could be improved significantly by:
- giving a little more context on radiotherapy and the importance of having pre-computed dose maps.
- describing seriously the dataset used. How GT (delineations and dose maps) is obtained? Is there any kind of preprocessing on images?
- extending the discussion or methodology description: was a fully 3d approach considered, analyze variability and uncertainty if possible (influence of the tumor delination,…). If results are obtained slice by slice, please compute performance metrics at the patient level (especially since there has been a selection of the slices).
- strengthening the validation: describe precisely the way it was done. Use multiple run of cross-validation to offer a better insight on the generatlization of the algorithm.
- tuning the hyperparameters independently if possible. What is the reason why a L1 loss function is used for the dose map prediction?
- Please state your overall opinion of the paper
Probably accept (7)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I think this paper and the presented approach are interesting but could be improved for reproducibility and to strengthen its experimental and statistical validation.
- What is the ranking of this paper in your review stack?
3
- Number of papers in your stack
5
- Reviewer confidence
Very confident
Review #3
- Please describe the contribution of the paper
The authors propose a multi-task dose prediction (MTDP) network to predict the dose distribution from CT images. The MTDP network consists of three encoder-decoder networks for isodose lines map prediction, dose distribution map prediction and texture map prediction, respectively. The paper describes both the encoder and decoder of each network that together leads to satisfactory performance. Performance on the prediction of the dose distribution from CT images seems to be the main goal of the authors. The methodology presents some kind of innovation in how the dose distribution can be well predicted .
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The research area is so interesting.
- The paper is generally well written, well structured, and easy to follow.
- The proposed method contains some kind of innovation.
- The proposed method trains the dataset by using a combination of five loss functions.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The loss functions described in the paper are unclear.
- The proposed method is trained by using only CT images and PTV contours. Generally, the dose distribution is predicted from tumors and OARs contours such as GAN method. It is unclear if the proposed method can predict the dose distribution from only PTV contours.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
- Additional description about the loss function and the obtained results are needed.
- Additional experimental results are needed to show the performance of the proposed method.
- Many typos in the paper.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- The loss function is unclear. It is better to describe the five loss functions deeply. In addition, a description about training the proposed method by using Loss total is needed.
- The proposed methods use only PTV contours as input data. However, it is unclear if the proposed method can predict the dose distribution with high performance without OARs contours such bladder contours, etc. Since related works used tumors and OARs contours as input dataset, It is better to add some experimental results that use PTV and OARs contours to show the performance of the proposed method compared with conventional methods.
- There are many typos in the paper that should be corrected.
- In the discussion, it is better to add the limitation of the proposed method.
- Please state your overall opinion of the paper
borderline accept (6)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper almost describes all the steps that the authors took to get to satisfactory performance. The research area is so interesting. However, training the network with only PTV contours is unclear. Additional experimental results are needed to show the performance of their method by using PTV and OARs contours.
- What is the ranking of this paper in your review stack?
1
- Number of papers in your stack
7
- Reviewer confidence
Somewhat confident
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Reviews are mixed for this paper. On the strengths, reviewers all agree about the novelty of the approach for an automated dose distribution prediction, by integrating isodose lines and texture features in the network architecture. The paper is generally well written and workflow properly described, and thorough experiments showcase the capability of the method to generate physically plausible plans.
On the other hand, R#1 is not convinced with the underperforming results, and highlights the lack of explanations on how the method can actually help to guide treatment planning. Other comments mention poor description of the ground-truth, lack of reproducibility with the dataset being internal, and there are several typos and grammatical errors in the paper that need to be corrected.
The authors are invited to address the reviewer’s comments in the rebuttal phase.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).
7
Author Feedback
We thank all the reviewers for their acknowledgement about the novelty of our method for an automated dose prediction, and their constructive comments for further clarification. 1)Reproducibility(R2&R3) Our code has been released at https://github.com/s0219/MTDP. The code link will be provided in the final paper. 2) Experimental results & absolute difference measurement(R1) Compared with the SOTA dose prediction methods, our method shows a relative increase of 2.28%(U-net), 1.55%(GAN), 1.79%(DoseNet), 0.83%(DeepLab v3+) on CI metric, and 19.1%(U-net), 14.1%(GAN), 11.0%(DoseNet), 15.7%(DeepLab v3+) on HI metric. As suggested, the absolute differences between the ground-truth and predicted plans are calculated, and our method relatively boosts the quality of dose map by 23.8%(U-net), 12.7%(GAN), 17.5%(DoseNet), 11.1%(DeepLab v3+) respectively. Throughout the t-tests, the p-values of all competing methods are less than 0.05, indicating that the improvements by our method are statistically significant. In addition, experimental results have already been approved by dosimetrists. To sum up, our experimental results are reliable. 3)How to guide treatment planning(R1&R2) Treatment planning is a complex design process involving multiple medical professionals. The dose received by a new patient is not clear for dosimetrists, and it requires dosimetrists to tweak repeatedly in a trial-and-error manner to obtain an acceptable plan. So, the total process is labor intensive, time-consuming, and costly. Our model in this study can be employed as a clinical guidance tool to alleviate the above issues. Concretely, the dose map generated by our model can be converted into the appropriate objective functions and provides dosimetrists with an initial point close to the ideal plan. In this manner, the trial-and-error steps and planning time can be reduced. In addition, oncologists can also have an accurate expectation of the treatment plan through the predicted dose maps. We will clarify the guidance of our method in the final paper. 4)Description of the ground-truth(R2) Ground-truth is the dose map planned by experienced dosimetrists, with the size of 512×512×172. And there is only one ground-truth(3D dose map) for one patient. To accelerate the convergence of the algorithm, the Min-Max Normalization is employed on the dose map. In the test phase, we evaluate the model performance at patient level instead of slice. All means and standard deviations in Table 1, including those of ground-truths, are calculated on all patients in the test set. We will clarify it in the final paper. 5)Comprehension of “texture”(R1) As suggested, we will choose another more appropriate term “gradient” in our final paper to replace “texture” for a clearer representation of the radiation patterns and edges in the dose map. 6)3D model & Performance metrics should be given at patient level(R2) Our current work is based on 2D due to limited computational resource, which will be emphasized in the limitation. Our future work will focus on extending our method to a 3D model. As for the performance metrics, we evaluate the model performance at patient level actually. Specifically, we predict the 2D slices of the patient and then stack the outputs into a complete 3D dose map according to the order of slices. We will clarify it in our final paper. 7)Additional OARs inputs(R3) We agree with the reviewer that additional OARs could provide more anatomical information for dose prediction task. However, some patients of the dataset used in this work lack the OARs maps owing to the time-consuming contouring process. In the future, we will try our best to acquire a more complete CT dataset with both PTV and OARs. We believe our model will perform better with the introduction of the additional OARs. For the comments on deeper description of loss functions and typos, we will address them in the final paper. And cross-validations will be conducted if the complete dataset is available.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
After reading the rebuttal, the authors seem to have properly addressed several of the issues raised, particularly by better defining the clinical rationale and integration, as well as enhanced the evaluation section with clearer explanation of the metrics, and how the proposed method has improved results over competing techniques.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).
6
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper predicts dose distribution for radiotherapy given CT pelvic images and labeling. The solution tends better by incorporating multiple losses through multi-task learning. The tool of this work can be practically useful in saving time. Reviewers raised several questions, point to phrases in the paper, comparisons to baselines, and utility of the method. The authors covered most issues and responded properly in rebuttal.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).
8
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper proposes a multi-task based dose prediction for radiotherapy from CT pelvic images. Three U-net networks correspond respectively to three tasks and are linked by lost functions. Overall, the proposed framework is not very recent. Moreover, since the dose distribution normally depends on the location of the tumor and OAR contours, this point is not taken into account to make the prediction more convincing. The future investigation is necessary. My proposition is “reject”.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Reject
- What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).
15