Back to top List of papers List of papers - by topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Yongsheng Pan, Yuanyuan Chen, Dinggang Shen, Yong Xia
Abstract
The missing data issue is a common problem in multi-modal neuroimage (e.g., MRI and PET) based diagnosis of neurodegenerative disorders. Although various generative adversarial networks (GANs) have been developed to impute the missing data, most current solutions treat the image imputation and disease diagnosis as two standalone tasks without considering the impact of diagnosis on image synthesis, leading to less competent synthetic images to the diagnosis task.
In this paper, we propose the collaborative diagnosis-synthesis framework (CDSF) for joint missing neuroimage imputation and multi-modal diagnosis of neurodegenerative disorders. Under the CDSF framework, there are an image synthesis module (ISM) and a multi-modal diagnosis module (MDM), which were trained in a collaborative manner. Specifically, ISM is trained under the supervision of MDM, which poses the feature-consistent constraint to the cross-modality image synthesis, while MDM learns the disease-related multi-modal information from both real and synthetic multi-modal neuroimages. We evaluated our CDSF model against five image synthesis methods and three multi-modal diagnosis models on an ADNI datasets with 1464 subjects. Our results suggest that the proposed CDSF model not only generates neuroimages with higher quality, but also achieves the state-of-the-art performance in AD identification and MCI-to-AD conversion prediction.
Link to paper
DOI: https://doi.org/10.1007/978-3-030-87240-3_46
SharedIt: https://rdcu.be/cyl6m
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The authors propose a multimodal approach for medical image analysis. Considering the typical problem of missing data, the authors propose a new methodology to use the existing data for synthesizing the missing image data. The model will bring the pathological information for the synthesized data.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The classification model. The synthesised results performance, mainly for PET.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Not to be the best in MRI synthesis, which means that the model needs to have further work.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
seems possible
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
No specific comments
- Please state your overall opinion of the paper
strong accept (9)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is good, well organised and well validated.
- What is the ranking of this paper in your review stack?
5
- Number of papers in your stack
5
- Reviewer confidence
Confident but not absolutely certain
Review #2
- Please describe the contribution of the paper
The paper proposes a collaborative diagnosis-synthesis framework (CDSF) for joint missing neuroimage imputation and multi-modal diagnosis of neuro degenerative disorders such as Alzheimer disease. The proposed method basically get benefit from MRI and PET features to generate artificial image. Based on the state of the art, the proposed work doesn’t show a significant improvement comparing to the current techniques. Also the proposed work use the current techniques to contribute without a real technical contribution.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-Use the public data with a large datasets e.g., ADNI1 ADNI2 -motivation of the imputation technique for many applications
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- no real novelty in technical aspects.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Except the codes, the data images are available from public ADNI website
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- Comparative between the techniques based on performance metrics needs also a significance tests to confirm the differences. -Propose techniques don’t not outperform the state of the art as shown in Table 1. For example if we look to MSE.
- How the author measured the overfitting or the data leakage, did they apply any strategy to avoid these noises.
- Please state your overall opinion of the paper
reject (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The proposed model is basically close to the previous models and I didn’t see a clear technical novelty. + the results didn’t show an improvement comparing to the state of the art.
- What is the ranking of this paper in your review stack?
3
- Number of papers in your stack
3
- Reviewer confidence
Very confident
Review #3
- Please describe the contribution of the paper
The paper describes a framework to train a model that works with missing data in a multi-modal classification problem. Previous approaches typically impute or generate missing data independently of the final classification task but the proposed method combines data imputation and the final classification task. The resulting system produces better imputation and achieves state-of-the-art performance in the final classification task of Alzheimer’s disease identification.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The idea of using the final classification task alongside the generation of missing data is novel and can be easily applied to other problems.
- Validated on a large well-known dataset
- State-of-the-art performance
- The paper is very well-presented
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Needs minor clarifications on some parts of the paper.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
I think the paper is highly reproducible and that the technique can easily be applied to other medical imaging problems.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- Fig. 2: What is RNB here?
- Fig. 2: What is K in the bottom right diagram?
-
Equations 3 & 4: i in Equation 2 referred to the subjects so I assume it is the same case in Equations 3 & 4 as well but why is the upper limit of i = 5 in this case? Shouldn’t it be j as referred to in Section 2.4 i.e. the feature maps of the j^th layer?
- Minor issues: – Page 4: Double quotation marks in latex can be achieved by typing `` and ‘’ – Page 4: “All layers are with the instance normalization”. Incomplete sentence. – Page 7: “…benifit the corss-modal”. Typo.
- Please state your overall opinion of the paper
strong accept (9)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Overall great paper but there are some bits of the paper which can be a bit clearer.
- What is the ranking of this paper in your review stack?
1
- Number of papers in your stack
5
- Reviewer confidence
Very confident
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper describes a framework to train a model that works with missing data in a multi-modal classification problem, which can be easily applied to other problems. This paper is written and organized well. Pleased the authors modify some issues according to the suggestions of the reviewers. For example, the author should point out what the RNB in Fig. 2 stands for. Also, how does the authors prevent the problem over-fitting due to small data sets?
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).
2
Author Feedback
We sincerely thank all reviewers and ACs for their insightful and invaluable comments and suggestions. Here are responses to some specific comments and the corresponding changes will be applied to the camera-ready version.
Q1: Not the best in MRI synthesis. (R2) A1: Since PET is the major missing modality, synthesizing PET modality is relatively more important. Meanwhile, while testing our algorithm, only real MRI scans are used since all subjects have real MRI scans. In our further work, we will also take some measures to enhance the quality of synthetic MRI.
Q2: Technical novelty. (R3) A2: The major novelty of paper is the collaborative diagnosis-synthesis framework (CDSF) for joint missing neuroimage imputation and multi-modal diagnosis. It is the first to create a two-way information flow, which forwards both the synthesis model and diagnosis model to the same diagnosis goal. For a fair comparison, we use a similar network structure to previous work. The overall performance benefits mainly from the collaborative strategy.
Q3: Performance of image synthesis. (R3) A3: It should be noticed that our goal is to do diagnosis on incomplete multimodal images, and hence image synthesis is not our final task. Since we do not take specific techniques to enhance the visual quality, it is foreseeable that our synthesis does not lead to obviously improved visual quality (e.g., MSE).
Q4: Overfitting and the data leakage. (R3) A4: It should be also noticed that synthetic images are also used to train the diagnosis model, leading to hugely increased data variety. Meanwhile, the subjects appeared in both ADNI-1 and ADNI-2 are removed from ADNI-2. Thus, these is no overlap between the training and test sets.
Q5: Minor clarifications (R4) A5: We have thoroughly checked our manuscript, and the corrections to these mistakes and typos will be applied to the camera-ready version.