Back to top List of papers List of papers - by topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Li Lin, Zhonghua Wang, Jiewei Wu, Yijin Huang, Junyan Lyu, Pujin Cheng, Jiong Wu, Xiaoying Tang
Abstract
Optical coherence tomography angiography (OCTA) is a novel non-invasive imaging technique that allows visualizations of vasculature and foveal avascular zone (FAZ) across retinal layers. Clinical researches suggest that the morphology and contour irregularity of FAZ are important biomarkers of various ocular pathologies. Therefore, precise segmentation of FAZ has great clinical interest.
Also, there is no existing research reporting that FAZ features can improve the performance of deep diagnostic classification networks. In this paper, we propose a novel multi-level boundary shape and distance aware joint learning framework, named BSDA-Net, for FAZ segmentation and diagnostic classification from OCTA images. Two auxiliary branches, namely boundary heatmap regression and signed distance map reconstruction branches, are constructed in addition to the segmentation branch to improve the segmentation performance, resulting in more accurate FAZ contours and fewer outliers. Moreover, both low-level and high-level features from the aforementioned three branches, including shape, size, boundary, and signed directional distance map of FAZ, are fused hierarchically with features from the diagnostic classifier. Through extensive experiments, the proposed BSDA-Net is found to yield state-of-the-art segmentation and classification results on the OCTA-500, OCTAGON, and FAZID datasets.
Link to paper
DOI: https://doi.org/10.1007/978-3-030-87237-3_7
SharedIt: https://rdcu.be/cyl9M
Link to the code repository
https://github.com/llmir/MultitaskOCTA
Link to the dataset(s)
https://ieee-dataport.org/open-access/octa-500
http://www.varpa.es/research/ophtalmology.html#octagon
Reviews
Review #1
- Please describe the contribution of the paper
The authors propose a new method for the analysis of the OCTA images and the segmentation of the FAZ regions as well as diagnostic classification of the images.
To do so, the authors designed an architecture combining different branches to specifically analyze the segmentation as well as other supplementary properties. The different analyzed characteristics of the branches are finally integrated hierarchically with the complementary diagnostic classifier.
The method seems to be interesting and well designed.
Also, the validation was performed using 3 different public datasets of reference, with different metrics.
Ablation studies were performed to motivate decision designs in the architecture. Additionally, comparison with other architectures were also provided.
I found this work interesting, in a novel image modality of increasing interest, but with some concerns specially related with the multitask issue.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The OCTA imaging is an image modality of increasing popularity and interest, therefore this work is of interest for the audience.
The multitask approach is interesting, providing the most important region in OCTA, the FAZ as well as the pathological classification.
The architecture design seems to be adequate and provides a positive impact in the performance.
The method was validated in 3 public datasets of reference with accurate results.
The method includes ablation studies to motivate different decisions, enriching the analysis and motivating the proposal.
Also, comparisons were performed with other accurate architectures in the issue.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The multitask aspect of the contribution is the most weak part of the work as it was barely discussed, analyzed and motivated.
An adequate pathological identification and further information as the FAZ region is interesting and with potential, but there are many aspect that I found missing in the analysis:
The different pathologies of the different datasets, an analysis about that, potential etc. How both issues are combined? Have the authors studied the performance separated? FAZ segmentation alone and diagnostic alone? Etc.
There are many aspect of the multitasking that was barely covered in the work. I think the main strength of the work is the combination of both objectives in a single architecture that offers accurate results in both parts. Despite that, the authors put a lot of attention in the refinement of the FAZ segmentation and further analysis of the multitask issue would provide more interesting information for the reader.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The reproducibility of the work is a positive aspect of the contribution. The architecture is reasonably described and detailed, the experimentation was performed with different public datasets.
Also, the authors indicate the offering of the code.
For all of that, as I said, a consider this aspect of reproducibility a positive point of the work.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
For all that was commented before, my main important suggestion for the authors is to reduce and gain some space with all the discussion and refinement of the FAZ que dedicate some analysis and discussion to the multitask aspect of the work, analyzed diseases, performance, potentials, how is the performance of two networks specifically for FAZ segmentation and other for diagnosis? etc.
This part is very interesting of the work and is slightly covered in the analysis.
- Please state your overall opinion of the paper
accept (8)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
My recommendation was motivated by the multitask proposal and the architecture design that was performed that offered satisfactory results, in a recent image modality of increasing popularity and interest and in issues of increasing interest at the moment. Also, the aspects of improvement related to the multitask part that I indicated was considered in this recommendation.
- What is the ranking of this paper in your review stack?
1
- Number of papers in your stack
4
- Reviewer confidence
Very confident
Review #2
- Please describe the contribution of the paper
In this paper, the authors propose a novel framework called BSDA-Net to segment foveal avascular zone and to classify diagnostic from OCTA images. In this framework, the authors construct boundary heatmap regression branch and signed distance map reconstruction branch to improve the segmentation performance. Besides, these two added branches are also benefit to the diagnostic classification.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The whole structure of this paper is clear.
- The novelty and contributions are outstanding.
- The descriptions of experiments are clear and reproducibility.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The title of this paper is not accurate based on the real mission. It may mislead the readers that this paper aims to segment retinal vessels and classify the artery and vein. “BSDA-Net: A Boundary Shape and Distance Aware Joint Learning Framework for Foveal Avascular Zone Segmentation and Diagnostic Classification from OCTA Images” may better.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The experiments can be reproduced based on the description of this paper.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
- The title of this paper is not accurate based on the real mission.
- In the second paragraph of the Introduction, the authors states that the FAZ segmentation can be divided into two categories: unsupervised methods and deep learning based methods. Is there any other supervised methods not belongs to deep learning methods in the previous works?
- x represents a point in Equation (1) and (2), but means an input image in Equation (5). Change another symbol instead of x in Equation (5) will be better.
- Please give the references of the metrics in the section “Experiments and Results”. The computation of a metric may have several ways in different references.
- In table 3, the authors need to supplement the experiments of using boundary heatmap regression branch and signed distance map reconstruction branch individually.
- Please state your overall opinion of the paper
strong accept (9)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The contributions of this paper are outstanding. The whole paper is readable and organized clearly.
- What is the ranking of this paper in your review stack?
1
- Number of papers in your stack
4
- Reviewer confidence
Very confident
Review #3
- Please describe the contribution of the paper
This paper proposed a multi-task learning framework for segmentation and classification on the OCTA dataset.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
(1) Classification and Segmentation boost the performance of each other (2) Achieving SOTA performance on three public OCTA datasets
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
(1) Limited Novelty. Previous biomedical image segmentation works [1,2,3,4,5] have done on region segmentation with the help of boundary regression or signed distance map reconstruction. The proposed region segmentation, boundary regression, signed distanced map reconstruction branches are simply a combination of previous works. The difference is that the author did on the OCTA dataset. I see it as an incremental component.
[1] Zhang, Zhijie, Huazhu Fu, Hang Dai, Jianbing Shen, Yanwei Pang, and Ling Shao. “Et-net: A generic edge-attention guidance network for medical image segmentation.” In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 442-450. Springer, Cham, 2019.
[2] Wang, Shujun, Lequan Yu, Kang Li, Xin Yang, Chi-Wing Fu, and Pheng-Ann Heng. “Boundary and entropy-driven adversarial learning for fundus image segmentation.” In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 102-110. Springer, Cham, 2019.
[3]Murugesan, Balamurali, Kaushik Sarveswaran, Sharath M. Shankaranarayana, Keerthi Ram, Jayaraj Joseph, and Mohanasankar Sivaprakasam. “Psi-Net: Shape and boundary aware joint multi-task deep network for medical image segmentation.” In 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 7223-7226. IEEE, 2019.
[4] Xue, Yuan, Hui Tang, Zhi Qiao, Guanzhong Gong, Yong Yin, Zhen Qian, Chao Huang, Wei Fan, and Xiaolei Huang. “Shape-aware organ segmentation by predicting signed distance maps.” In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 07, pp. 12565-12572. 2020.
[5] Li, Shuailin, Chuyu Zhang, and Xuming He. “Shape-aware semi-supervised 3d semantic segmentation for medical images.” In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 552-561. Springer, Cham, 2020.
(2) Three decoders have the same structure, making no difference for region segmentation, boundary regression, SDM reconstruction tasks. As those three tasks focus on different features, three same decoders may not be appropriate.
(3) Incomplete experiments; please see detailed comments.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Satisfactory
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
-
Because the GT of three tasks in the segmentation branch are all from (or generated from) the same GT binary mask, thus the information gain from ground truth may not increase. Thus, the performance boost may come from a bigger model, instead of the auxiliary task, the author should maintain the similar model size to do the ablation of Tab.2 and Tab.3.
-
The ablation of the hyperparameter of the loss function is missing. There are four loss terms; the ablation study of trade-off parameters is essential to see the contributions between different tasks.
-
The idea behind boundary branch using Gaussian then regression instead of pixel-wise segmentation is unclear. Why is regression better than pixel-wise segmentation? Besides, an ablation study replacing regression with segmentation should be done to prove.
-
- Please state your overall opinion of the paper
borderline reject (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
– The experiments part should be improved (e.g. maintaining a similar model size); some necessary ablation studies are missing. – The novelty is limited as previous methods did similar works (see weakness part).
I will raise the score if the author can address my concerns.
- What is the ranking of this paper in your review stack?
3
- Number of papers in your stack
5
- Reviewer confidence
Confident but not absolutely certain
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This work proposes a novel method for fovea segmentation from OCTA images. State of the art performances were achieved in experimental results on three public datasets. Some questions were raised about the novelty of the proposed method as compared to previous segmentation methods that uses boundary regression or signed distance maps.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).
2
Author Feedback
We thank the reviewers and AC for their valuable comments. We will first address the criticisms from reviewer#3 who gave the only negative score and then others.
#3： Q4.1&7.3: Limited novelty and the boundary branch. A: As reviewer#1 has pointed outed, OCTA is a novel modality of increasing interest, and our work is the first one in this area to have simultaneously performed accurate FAZ segmentation and pathological classification, showing that FAZ-related reconstruction features can be effectively utilized in joint leaning and boosting diagnostic performance. Our framework is novel. Firstly, none of the five works provided by reviewer#3 was designed for joint classification and segmentation but purely segmentation. Secondly, the way how we deal with the boundary branch differs from those published. Specifically, [1-3] deal with the boundary constraint in a pixel-wise segmentation manner. However, manual labeling is inevitably subjective to some extent and its boundary might not be completely accurate, especially for medical images. Inspired by some works related to soft labeling in classification and landmark detection, we propose the novel Gaussian heatmap regression branch, which performs better than pixel-wise segmentation (+0.5% in Dice on OCTA500). Furthermore, [3] utilizes normal DM by calculating only background distance, and [4, 5] employ either SDM as the only objective or a shared decoder structure. SDM alone has negative effects on contour (smooth the edge, which is unfavorable when the boundary irregularity contains pathological information). In such context, we design a boundary regression task to restore the boundary information. Meanwhile, these tasks reconstruct FAZ-related features from different aspects and are jointly utilized for classification. As such, the classification itself is also a very important component of this work.
Q4.2&7.2: Same decoder structure and loss weights. A: By employing decoders with the same structure, our purpose is to fairly reconstruct features from different tasks and input them to the classifier in a consistent but hierarchical manner. Fig. 2 and Fig. A1 show representative outputs from BSDA-Net. Clearly, SDMs and boundary heatmaps can be well reconstructed, and thus the proposed decoder’s structure is appropriate. With that being said, we have to emphasize that we treat the tasks differently even with the same decoder structure, as described in our text “Features from …contour perception and preservation” (see page#4). And we assign different weights for different task-related loss terms based on extensive experiments, suggesting that 3:1:1 (S:D:B in Seg) and 1:1 (Seg:Cl) work relatively the best.
Q7.1: Model size. A: Please note the sizes of the encoder E and the decoder S remain unchanged in all ablation analyses. The other two branches can be treated as additional sources of loss for segmentation and can be removed in the test phase (won’t affect the segmentation performance). Even so, we have tried replacing the encoder with ResNeSt101 (46M) in our baseline model (E+S, 25M+9M), making it have a bigger size (54M) than BSDA-Net (w/o C, 52M). However, the performance only increases slightly (+0.1% in Dice, which is far less compared with the improvement induced by our auxiliary tasks). As for classification, the bigger ResNeSt101 does not perform better than ResNeSt50 on all three small datasets. These all suggest that our improvements were not due to model size increase but from additional segmentation losses and a utilization of FAZ-related features from different aspects for classification.
#1: Q1: More analysis. A1: We have already provided results on analyzing the individual performance of each sub-network in Tables 2 and 3 and detailed reports in Tables A1 to A3 for the classification task. Given that MICCAI has space limitations, we will add more details in future journal version.
#2： Q2: Title, formulas and metric references. A2: We will make these revisions.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This work proposes a novel method for fovea segmentation from OCTA images. State of the art performances were achieved in experimental results on three public datasets. The rebuttal addressed reviewer concerns very well.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).
4
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors have provided an articulated response to the critically raised points. The rebuttal is organized and to the point with compelling arguments.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).
1
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The method has been evaluated on a large amount of scans and has obtained largely positive reviews. The remark on the novelty from R3 has been in my view well addressed by the rebuttal, where they were able to show how their method differs from the related boundary segmentation methods.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).
2