Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Alan Q. Wang, Aaron K. LaViolette, Leo Moon, Chris Xu, Mert R. Sabuncu

Abstract

Compressed sensing fluorescence microscopy (CS-FM) proposes a scheme whereby less measurements are collected during sensing and reconstruction is performed to recover the image. Much work has gone into optimizing the sensing and reconstruction portions separately. We propose a method of jointly optimizing both sensing and reconstruction end-to-end under a total measurement constraint, enabling learning of the optimal sensing scheme concurrently with the parameters of a neural network-based reconstruction network. We train our model on a rich dataset of confocal, two-photon, and wide-field microscopy images comprising of a variety of biological samples. We show that our method outperforms several baseline sensing schemes and a regularized regression reconstruction algorithm. Our code is publicly-available at https://github.com/alanqrwang/csfm.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87231-1_13

SharedIt: https://rdcu.be/cyhUE

Link to the code repository

https://github.com/alanqrwang/csfm

Link to the dataset(s)

https://github.com/yinhaoz/denoising-fluorescence

Reviews

Review #1

Please describe the contribution of the paper

Design a stochastic sensing model with the Hadamard sensing basis; Simultaneously train the Hadamard sensing pattern and the U-Net in fluorescence Microscopy to learn the importance of different Hadamard coefficients
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

A novel application: apply deep learning in the application of Hadamard sensing fluorescence microscopy Joint end-to-end optimization allows to learn the optimal Hadamard sensing pattern
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

n.a.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Yes
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
1. What is the compression value alpha for the raw data displayed in Fig. 4?
2. It is better to emphasize that you train the sensing model and the reconstruction model simultaneously in the caption of Fig. 1. Otherwise at the first look the readers like me may think that you train the U-Net only to post-process the preliminary image, which is obtained from a fixed Hadamard sensing pattern.
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The first article in literature to apply a learning-based method to optimize the Hadamard sensing pattern
What is the ranking of this paper in your review stack?

2
Number of papers in your stack

4
Reviewer confidence

Confident but not absolutely certain

Review #2

Please describe the contribution of the paper

This paper presents an approach for joint sensing and reconstruction in fluorescence microscopy, and it is achieved by jointly optimizing over the Bernoulli distribution and a U-Net.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Some tricks are also proposed to improve the performances such as differentiable relaxation of the sampling operator, and the differentiable closed form computation for normalization.

Though the idea of joint optimizing over sensing and reconstruction is not new, the application in the specific fluorescence microscopy is quite interesting.

This paper is well written and easy to follow, and the presentation is quite concise and clear.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Experiments are only conducted using one dataset.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Presentation of the approach is sufficient, and implementation should not be difficult though the codes not provided.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
Some minor comments can be found below.
1. The authors should give more information about the dierentiable relaxation of the sampling procedure in Section 3.3.
2. I don’t see essential dierence between eq(8) and a direct normalization of ~ P. Why cannot we also use the same scaling operation when bar{p}<alpha?
Please state your overall opinion of the paper

accept (8)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Some tricks are also proposed to improve the performances such as differentiable relaxation of the sampling operator, and the differentiable closed form computation for normalization.

Though the idea of joint optimizing over sensing and reconstruction is not new, the application in the specific fluorescence microscopy is quite interesting.

This paper is well written and easy to follow, and the presentation is quite concise and clear.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

4
Reviewer confidence

Very confident

Review #3

Please describe the contribution of the paper

In the paper the authors propose to jointly learn both the optical sensing scheme and the image reconstruction in compressed sensing microscopy, which employs multiple measurements with (Hadamard) patterned illumination masks to reduce additive noise. Specifically, the papers proposes to jointly a) optimize the probability with which a certain pattern (Hadamard row) is included in the final measurements (i.e. the sensing scheme), and b) to learn the inverse transformation that yields the reconstructed image.
The paper finally applies this method on a real 2D microscopy dataset and demonstrates that the jointly learned sensing scheme/reconstruction performs better than other uninformed sensing strategies/baseline reconstruction methods.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Interesting idea and clear presentation of the method
- Clearly outperforms given baselines schemes/reconstruction methods
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Not entirely clear how strong the baseline really is
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Hard to reproduce:
- In-house training dataset not available
- code not available
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
1) Baseline:
- “Widly used” -> the (paper) reference for the “TV-W” baseline is missing. “TV-W” refers to “HQS” from Wang et al 2020?
- The authors state that TV-W cannot be used for uniform masks. Why would it not be possible to use for every Hadamard coefficient simply the first alpha*S measurements?
- To fully appreciate the quality improvement in Fig 4, one would need to see the output of Hz with uniformly sampling (i.e. the naive reconstruction without regularization).
2) Other:
- “residual U-Net architecture” -> which one exactly? [18] is not residual…
- “temperature hyperparameter” -> what is meant by that? (the loss in Eq. 7 is MSE)
- “FMD dataset” -> what is FMD?
- “alpha-percent” -> “a fraction of alpha”
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Generally well written and solid experiments, yet not entirely clear how strong the baseline really was
What is the ranking of this paper in your review stack?

3
Number of papers in your stack

5
Reviewer confidence

Somewhat confident

Review #4

Please describe the contribution of the paper

The authors propose a joint optimization scheme of measurement + DNN-based reconstruction optimization that remains unexplored for compressed sensing fluoroscence microscopy.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The paper is well-written, with strong motivation and well designed experiments. I am new to FM modalit, but seems like the novel contributions include 1) joint optimization of the sensing pattern and reconstruction 2) thorough evaluation on simulated and real (raw) data
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The discussion on the learned masks seems inadequate. I would represent Table 1 as box plots. The trend would be much clearer. Also there is a lot of information inscribed.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors have made the code available with trained models.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Usage of box plots for Table 1 as suggested earlier. Robustness analysis on other datasets and varying levels of SNR would make the paper stronger.
Please state your overall opinion of the paper

strong accept (9)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Novel contribution, strong evaluation and comparison.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Somewhat confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

Reviewers are in agreement about the value of this work on the joint optimization of sensing and reconstruction using neural networks, especially in the context of the given application field of single-pixel fluorescence microscopy imaging. I suggest the authors to address the concerns pointed out by the reviewers to improve clarity.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

1

Author Feedback

N/A

back to top

Joint Optimization of Hadamard Sensing and Reconstruction in Compressed Sensing Fluorescence Microscopy