Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Or Bar-Shira, Ahuva Grubstein, Yael Rapson, Dror Suhami, Eli Atar, Keren Peri-Hanania, Ronnie Rosen, Yonina C. Eldar

Abstract

Breast cancer is the most common malignancy in women. Mammographic findings such as microcalcifications and masses, as well as morphologic features of masses in sonographic scans, are the main diagnostic targets for tumor detection. However, improved specificity of these imaging modalities is required. A leading alternative target is neoangiogenesis. When pathological, it contributes to the development of numerous types of tumors, and the formation of metastases. Hence, demonstrating neoangiogenesis by visualization of the microvasculature may be of great importance. Super resolution ultrasound localization microscopy enables imaging of the microvasculature at the capillary level. Yet, challenges such as long reconstruction time, dependency on prior knowledge of the system Point Spread Function (PSF), and separability of the Ultrasound Contrast Agents (UCAs), need to be addressed for translation of super-resolution US into the clinic. In this work we use a deep neural network architecture that makes effective use of signal structure to address these challenges. We present in vivo human results of three different breast lesions acquired with a clinical US scanner. By leveraging our trained network, the microvasculature structure is recovered in a short time, without prior PSF knowledge, and without requiring separability of the UCAs. Each of the recoveries exhibits a different structure that corresponds with the known histological structure. This study demonstrates the feasibility of in vivo human super resolution, based on a clinical scanner, to increase US specificity for different breast lesions and promotes the use of US in the diagnosis of breast pathologies.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87234-2_11

SharedIt: https://rdcu.be/cyl76

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

This paper describes a learning based approach to achieve “super resolution” ultrasound imaging of breast cancer. Super resolution refers to distinguishing features smaller than the wavelength of ultrasound and would be helpful in breast cancer characterization because it would give insight into the microvasculature of tumours and therefore cancer progression. The results are shown in vivo (which is challenging for superresolution techniques compared to phantoms) on a prospective study (n=21).
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The main strength of this paper is the promise of super resolution imaging in vivo. Moreover, breast cancer is the leading type of cancer in women and current screening and diagnostic tools are inadequate. Ultrasound is already the most frequently used imaging technique after screening mammography so improving the capability of ultrasound could potentially have a huge impact. Moreover, the technique could be applied to a range of ultrasound scanners which further increases impact.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Although it is promising to show results in vivo, the data set (n=21) is small so these should be considered preliminary results to show the viability of a NN-based approach. Of course, the need for contrast enhancements also makes this is more specialized imaging technique.

Another weakness is the lack of quantitative analysis of the results. Figure 2 is promising with three example images, but what about the rest of the data?

There is no comparison to other networks or classical methods, so it is hard to judge whether the proposed network structure offers any significant advantage.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The method is fairly easy to replicate but the unique CEUS dataset is not readily available to other researchers.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

The Discussion can be greatly reduced in length to make room for an in-depth analysis of the results. Since only three sample images are shown in Figure 2, the paper would be improved with a quantitative analysis of all of the results. The goal of the analysis is to explore the strengths and weaknesses of the proposed superresolution method. In particular, comparison to traditional techniques is expected.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The possibility of achieving super resolution ultrasound imaging in vivo would have a large impact on breast cancer imaging and other imaging applications. Using the super resolution results to help classify lesions would be very valuable. The novel use of a network-based approach is certainly worth exploring. Although there is a lack of comparison to standard methods, nor quantitative analysis of the results, the paper is still interesting due to the novelty of the approach and the clear clinical application.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Confident but not absolutely certain

Review #2

Please describe the contribution of the paper

Neoangiogenesis is an important parameter for pathological determination of breast tumors to increase specificity. With their trained network, the authors recover the microvasculature structure in a clinically appropriate time without prior point spread function knowledge. This may be valuable in reducing call-back rates by increasing specificity in malignant lesion detection/diagnosis. They present in vivo human results and a cyst, fibroadenoma and cancer are imaged and analysed. They documented different vasculature structure in each of these cases which corresponds with the histopathology. The results apply to any vasculature related pathologies.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

their exposition is clear: They use Ultrasound Localization microscopy (ULM) to obtain super-diffraction limited spatial resolution and facilitated fine visualization and give a nice history of this process and how contrast enhanced ultrasound CEUS can lead to super resolution when the PSF is known. Usually US implementations use high frame rate scanners (HFR). Barriers to clinical use include limited time window, some organ motion and low frame-rate imaging.

This work addresses these issues. By relying on model-based convolutional neural networks which are unrolled versions of the standard iterative algorithm for sparse constrained reconstruction, iterative soft threshold algorithm (ISTA).

Here is is used for the first time in in vivo scans with low frame rates. First time results for in vivo lesion images of three different patients and lesions, which are cyst, fibroadenoma, and cancer. this demonstrates the feasibility of in vivo super resolution, in a clinically realistic setting to increase specificity of US for lesion characterization, which is an important step to reduce false call backs and promotes clinical US in breast pathology diagnosis. The B-mode and CEUS images were obtained and saved for offline processing. The patients were ‘guided’ to be exceedingly stable to reduce out-of-plane motion. Apparently, this was verbal instruction but may have involved constraints, this is not made clear.

There is a clear description of the probe used, frame rate, MI, etc. They used highly correlated frames, assigned to the same subsequence, to correct for translation using image registration. The transformation matrix was computed using the first frame of a sequence of B-mode images. An SVD based filter was applied to the CEUS frames to extract moving UCA signals, which then led to the super-resolution recovery.
The architecture of the network is an unfolded ISTA algorithm, a standard for sparse imaging. Fixed parameters are replaced with learnable parameters and convolutions. Training was supervised, and the loss minimization had a quadratic and an L1 norm term. The Tikhonov-like regularizing parameter was set 0.01, and the network output was compared to a Gaussian filtered version of the true image of UCA locations, for stability. ADAM optimization was used. Lack of training data is overcome by using synthesized data of the CEUS image. The pixel size was chosen of 31.25 microns, to achieve a blood vessel resolution under 100 microns.

The use of unrolling gives more clarity and interpretability to the network, giving the clinician the ability to identify potential failure cases. The results indicate feasibility of this imaging modality.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The patients were ‘guided’ to be exceedingly stable to reduce out-of-plane motion. Apparently, this was verbal instruction but may have involved constraints, this is not made clear. this is a very minor point however, The exposition was clear but there were some misprints, e.g. ‘reviled’ for ‘revealed’ on page 7 line 1.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Would like to see a bit more code, but good description, meet the requirements I believe
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Paper is well written and reports noteworthy results. There were some minor typographical errors but this result shows clearly the feasibility of the vasculature imaging as a clinically viable modality.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The clarity of exposition and the clinically relevant results.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

5
Reviewer confidence

Very confident

Review #3

Please describe the contribution of the paper

The authors used a deep neural network architecture, which which the microvasculature structure is recovered in a short time.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

recovering microvasculature structure is of high importance to the clinicians. So if the method works efficiently, the results will be of a great significance.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Some form of validation is required to validate the results.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The results are reproducible.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Please add some form of validation for the data you presented in Fig2, how can one should know the results are correct.
Please state your overall opinion of the paper

strong accept (9)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The methodology and results are interesting but there is no validation for the produced results that can be easily addressed.
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

3
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

The application target of imaging breast cancer is a very relevant and important topic. Ultrasound localization microscopy (ULM) is also a promising emerging techniques.

The novelty is posed as (1) the low-frame-rate clinical ULM imaging of breast cancer (2) using a deep unrolled structure. For the first part, this is already presented successfully in [18]. It is not clear to me what the added value is from the learned PSF in (2). Moreover, this deep ULM was also presented in [19], Fig. 7b of which further overlaps with the Fig1 of the current submission. So, I am not entirely clear what the methodological novelty is. Whether it is the alignment of frames and/or the learned vs. tuned parameters, one would expect this aspect to be studied with an ablation / comparison study.

A study of 21 patients is mentioned, but only 3 selected cases are shown, and those only with qualitative descriptions. How were these 3 selected? How would the results compare to [18], [19], or any alternative method? With a comparative or ablative study, it is not clear which parts of the proposed method are contributing to the presented results.

Together with the above, the reviewers’ comments also need to be addressed for the acceptance of this work.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

6

Author Feedback

The authors would like to thank the reviewers for their comments. We believe we have addressed all the issues raised in the following rebuttal. For each comment, we have first highlighted the issue, then we provided an answer.

Comparison to alternative method (all reviewers): The authors performed a comparison between the method described in the paper to a classical ULM technique in which local maxima are computed for each frame. Since the computation is applied on the preprocessed data (as described in paper), the detected maxima are assumed to correspond to UCAs. Then, the intensity-weighted center of mass of each UCA signal is calculated to obtain coordinates of its localized position. Finally, all localizations are summed to obtain the final image. Both methods showed similar profiles further supporting the findings. Nonetheless, denser UCA localizations are achieved via the learning-based method (ours) which promotes recoveries in highly populated regions. The authors verified that a figure that presents the comparison and an analysis can be added within the page limitation.

A study of 21 patients is mentioned, but only 3 selected cases are shown. How were these 3 selected? What about the rest of the data? (meta-reviewer, reviewer #1): Here we present preliminary results of three patients. The remaining data is currently processed and hopefully will be reported in future submissions. The 3 cases shown were selected as preliminary examples in order to gain a wider perspective regarding the feasibility of the method when applied to this type of data (i.e., images from clinical US scanners operating in low frame rates) by exploring different types of lesions (i.e., different benign and malignant lesions).

Clinical ULM imaging of breast cancer was presented successfully in [18] (meta-reviewer): We believe that there might be a mistake with the above reference ([18]) and that the intention was to reference [17]. In [17], the authors reported in vivo human results on different types of malignant tumors (One patient with HER2-positive breast cancer, and two patients with triple-negative breast cancer). Here we address different lesions, both malignant and benign. Furthermore, the method used here harness the power of deep neural networks thus promoting the use of the technology in the medical domain.

Deep unrolled ULM was also presented in [19]. What is the methodological novelty (meta-reviewer): Our novelty is not in the method but in the application. Although the method was used before for an in vivo animal model with a high frame rate scanner, here it is used for the first time for in vivo human scans with a clinical US scanner operating at low frame rates. The authors still saw the importance of detailing the method and adding visualizations in order to better introduce it to readers that encounter this method for the first time.

What the added value is from the learned PSF (meta-reviewer): Learned PSF makes the need for prior knowledge about the system PSF redundant. This is important since the system PSF is often not known and needs to be estimated from the data in order to perform super-resolution. Estimating it leads to a greater dependency on the user and to an increased sensitivity of the results on the proper estimation. Learned PSF alleviates this dependency and assists in making the process of super-resolution accessible.

How were the patients ‘guided’ to be exceedingly stable to reduce out-of-plane motion? (reviewer #2): Verbal and written instructions.

dataset is not readily available to other researchers (reviewer #1): The clinical data is strictly protected. It cannot be shared or made public.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

All major concerns stand after the rebuttal.

Limited-to-no methodological novelty: As the rebuttal also confirms, the methods were already published in [19], including a method figure almost the same as here. The only technical contribution might have been related to the conventional US scanner, but nothing is declared nor explained on that in the submission; nor in the rebuttal, despite this been asked directly in the reviews. So, I have to assume no technical contribution on conventional scanning either, or none that is willing to be shared in this MICCAI submission.

Limited comparison with other methods: There was no comparison in the original submission. The authors proposed in the rebuttal to compare with “conventional ULM” with peak-picking by local maxima, which is certainly not state-of-the-art (sota) in any terms. Even the submission reference list has several methods to compare with, including but not only of that in [19]. There would also be many others, even any CNN-based baseline would do, such as the one in Youn et al. “Detection and Localization of Ultrasound Scatterers”, IEEE TMI 2020. But, no sota comparison is given in the submission.

No quantitative evaluation: Only 3 selected cases are presented, only with qualitative descriptions.

The method has value, there is no doubt, and [19] already presents this. Additional improvements, results, outcomes would be much welcome, but these were not provided in the submission. MICCAI norms require each submission to present either sufficient methodological novelty or extensive evaluation, and this submission does not meet either criterion. So, I cannot recommend its acceptance.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Reject
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

17

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The authors propose a superresolution ultrasound method and present results in in vivo data for breast ultrasound images. This is quite a milestone and as such all reviewers have prised the interest of the paper. In my opinion this wil be of interest to the community and with high potential clinical impact.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

1

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The authors rebuttal has effectively cleared most of the raised issues/concerns. The only point that is not convincing is point #2 in the rebuttal “The remaining data is currently processed and hopefully will be reported in future submissions” which make me wonder about the applicability of the method and its computational cost.
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

Accept
What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

6

back to top

Learned super resolution ultrasound for improved breast lesion characterization