Back to top List of papers List of papers  by topics Author List
Paper Info  Reviews  Metareview  Author Feedback  PostRebuttal Metareviews 
Authors
Seungjae Han, EunSeo Cho, Inkyu Park, Kijung Shin, YoungGyu Yoon
Abstract
Calcium imaging is an essential tool to study the activity of neuronal populations. However, the high level of background fluorescence in images hinders the accurate identification of neurons and the extraction of neuronal activities. While robust principal component analysis (RPCA) is a promising method that can decompose the foreground and background in such images, its computational complexity and memory requirement are prohibitively high to process largescale calcium imaging data. Here, we propose BEAR, a simple bilinear neural network for the efficient approximation of RPCA which achieves an order of magnitude speed improvement with GPU acceleration compared to the conventional RPCA algorithms. In addition, we show that BEAR can perform foregroundbackground separation of calcium imaging data as large as tens of gigabytes. We also demonstrate that two BEARs can be cascaded to perform simultaneous RPCA and nonnegative matrix factorization for the automated extraction of spatial and temporal footprints from calcium imaging data. The source code used in the paper is available at https://github.com/NICALab/BEAR.
Link to paper
DOI: https://doi.org/10.1007/9783030872342_56
SharedIt: https://rdcu.be/cyl8S
Link to the code repository
https://github.com/NICALab/BEAR
Link to the dataset(s)
https://github.com/NICALab/BEAR
https://drive.google.com/file/d/115lCnwIVU0TtKedQ_31FDaOG8wksGmG9/view
Reviews
Review #1
 Please describe the contribution of the paper
A variant formulation of Robust PCA implementation agnostic called BEAR, and 3 different implementations (basic, greedy and cascaded) all based on neural networks. The construct validity of the greedy version, and its application exemplification on calcium imaging.
The rationale of the new RPCA variant BEAR is based on replacing the minimization of the trace norm (here referred to as nuclear norm I suppose in reference to Schatten norms family) with a maximum rank constraint. Then some other additional constraints follow but they are dependent on the implementation.
While the basic implementation requires a prior guess on the rank of L, the greedy version eliminate such demand but still keep a low complexity.
 Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
 Many! A good rationale on the proposed solution, the separation of the formulation of the solution from the implementation, the 3 variant implementations, …
 No need to show off, or overtone… Just sober good experimentation with convincing results.
 Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
If original motivation was computational complexity of RPCA there are stronger competitors to compare to.
 Please rate the clarity and organization of this paper
Excellent
 Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist  for instance, providing code and data is a plus, but not a requirement for acceptance
Sufficient maths and pseudo code supplementary material to guarantee replicability (even if code wouldn’t have been provided…which it has anyway!)
 Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWERGUIDELINES.html
OVERALL COMMENTS It took me two rounds of reading to fall in love with this research. During my first round, I was so driven by the computational complexity that I admit I slightly loss focus on the big picture; the authors were claiming an improvement in computational complexity but they were actually referring to unenhanced versions of PCA, when nowadays optimizations exist that under certain constraints can reach sublinear complexity, see [Ma and Aybat (2018) Proc of IEEE 106(8):14111426], and BEAR also has constraints, so what was the big deal?. But the second reading put things into focus; even without sophisticated optimization BEAR was actually really good in complexity, it is remarkably stable in its error, and the rest of the paper is actually very good in almost every aspect! Not much to say other than congratulations to the authors on a beautiful work! Some minor comments below follow but feel free to ignore them.
SUGGESTIONS TO IMPROVE THE DRAFT
 Fig 3a is excellent to quantitatively report errors. Can it be further accompanied by another figure qualitatively showing good/success and bad/failure examples on the synthetic data?
 Sect 3.2 is perhaps unnecessarily long given that you are already giving the computational complexity analysis… Let me push my luck here; expand Sect 3.3 at the cost of Sect 3.2
 Evaluation is made over the greedy variant but application results are reported with the cascaded variant. Why not complete both ways? Evaluate and validate all 3 variants, and comparatively show the results of the 3 variants on real images.
 Show also results of the another RPCA variant in Sect 3.3. for visual comparison.
 Report exact sizes (number of neurons per layer) and number of parameters of the neural network. If I got this right, since one of the constraints is L=WW^{T}Y then it is likely the size of the network does depend on Y. Correct?
 Please state your overall opinion of the paper
strong accept (9)
 Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Robust PCA is well studied. There are now tens of variants each one trying to optimize here or there. However, to the best of my knowledge the variant in Eq 3 is novel and even if perhaps not as optimal as some other variants for very specific cases, fig 3a justifies compromising a bit of efficiency for a remarkably stable error which means that for practical purposes, the enduser can employ BEAR without worrying much of particularities.
 What is the ranking of this paper in your review stack?
1
 Number of papers in your stack
5
 Reviewer confidence
Confident but not absolutely certain
Review #2
 Please describe the contribution of the paper
The authors proposed BEAR, a simple bilinear neural network for the efficient approximation of RPCA which achieves an order of magnitude speed improvement with GPU acceleration compared to the conventional RPCA algorithms. In addition, BEAR can perform foregroundbackground separation of calcium imaging data as large as tens of gigabytes.
 Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The authors introduce a Bilinear neural network for Efficient Approximation of RPCA which is a computationally efficient implementation of RPCA as a neural network. The work solves the key problem of efficiency in RPCA and it is general for similar tasks.
 Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
But it lacks some reference of similar tasks.
 Please rate the clarity and organization of this paper
Very Good
 Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist  for instance, providing code and data is a plus, but not a requirement for acceptance
The parameters and environment of the experiment is given, it can be easily reproduced as the paper.
 Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWERGUIDELINES.html
Add the reference: [1]Van Luong H, Joukovsky B, Eldar Y C, et al. A DeepUnfolded ReferenceBased RPCA Network For Video ForegroundBackground Separation[C]//2020 28th European Signal Processing Conference (EUSIPCO). IEEE, 2021: 14321436. [2]Hovhannisyan V, Panagakis Y, Parpas P, et al. Multilevel approximate robust principal component analysis[C]//Proceedings of the IEEE International Conference on Computer Vision Workshops. 2017: 536544. [3]Solomon O, Cohen R, Zhang Y, et al. Deep unfolded robust PCA with application to clutter suppression in ultrasound[J]. IEEE transactions on medical imaging, 2019, 39(4): 10511063.
 Please state your overall opinion of the paper
borderline accept (6)
 Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This work give a neural network to solve the problem which is hard to be optimized. The novelty and contribution of this work is clear. But it lacks some reference of similar tasks.
 What is the ranking of this paper in your review stack?
2
 Number of papers in your stack
4
 Reviewer confidence
Confident but not absolutely certain
Review #3
 Please describe the contribution of the paper
The paper casts the problem of background vs. foreground separation of the calcium recordings as a constrained optimization problem that can be efficiently optimized using a neural network.
 Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The problem formulation is interesting, and of potential use for the neuroscience community. The application to multiple datasets is shown (mouse, drosophila, zebrafish).
 Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
See the detailed comments below
 Please rate the clarity and organization of this paper
Poor
 Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist  for instance, providing code and data is a plus, but not a requirement for acceptance
Codes are provided and some hyperparameters are given. Mostly clear but can still be improved (e.g. which 1/3 of the data is used for training).
 Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWERGUIDELINES.html

The idea of using neural nets to solve PCA/RPCA/matrix factorization existed way before this paper, please make sure that the relevant papers are cited.

How does this compare to CNMFE/OASIS? https://elifesciences.org/articles/28728 https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005423
 Eq. 1 is called lowrank+sparse recovery, please cite the relevant papers.

Typo below Eq. 1 > Y, L, S, S 
Reformulated > better wording: Eq. 2 is a surrogate for Eq. 1 and does not necessarily give the same solution

We do have faster SVD methods instead of doing the full decomposition on the whole data matrix https://arxiv.org/pdf/1502.00182.pdf

O(nmr) > how about the number of iterations? How does it interact with m, n, r? This gives a more realistic view of “time complexity”

Eq. 3 again is not a “reformulation”, the solution is different

Shouldn’t L be WM instead of WMY? Why are we multiplying by Y?

M = W^T, what is the interpretation? It finds a lowrank AND symmetric matrix which we might not want.
 What kind of batching is used? Over time, over space?
 Inference only mode can be applied to SVD too, when the singular vectors are known
 The time comparison should include the training of the neural net (training is the same as finding singular vectors & values).
 In Eq. 5, what does “localized” in localized sparse footprints correspond to? How is this condition enforced in the cost?
 How is the nonnegativity enforced? We have specific optimization algorithms for NMF, do the authors simply use backprop for NMF?

Please include a short description of the compared methods. If some methods are based on full SVD on the data matrix how can BEAR outperform them?

Results in Fig. 4 are nice, but videos would be much more helpful, also can the authors show “residual” too (YLS).

The figures on real data as illustrated are not quite informative, please include more informative visualizations (use the papers provided above as examples).

The experiments are limited, since the methods existed before, it’s important to show that the proposed approach existing methods in multiple common datasets

Why training only on 1/3 of data? How is that 1/3 selected? What if there’s a shift in the dynamics of the data in the initial stages of recording vs. later later?

In many cases the low rank component is itself variable in time (there’s overall intensity change in the tissue), how can that be added to the model?
 Simulation experiments (simulated calcium videos) are crucial to see how well the method can recover ground truth signals.

 Please state your overall opinion of the paper
probably reject (4)
 Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Experimental evaluation is limited (on simulations and real data). A clearer empirical understanding of the method is required before sharing it with experimentalists. Literature review seems incomplete, there already exists some literature on sparse+lowrank recovery.
 What is the ranking of this paper in your review stack?
5
 Number of papers in your stack
1
 Reviewer confidence
Confident but not absolutely certain
Primary MetaReview
 Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposed BEAR, a simple bilinear neural network for the problem of background vs. foreground separation of the calcium recordings.
The key strengths include 1) The problem formulation is of potential use for the neuroscience community. 2) The Bilinear neural network for Efficient Approximation (BEAR) is a computationally efficient implementation of robust PCA (RPCA). 3) Multiple species data including mouse and zebrafish are used in the validation of the new method.
The key weaknesses include: 1) lacks some reference 2) some minor issues mentioned in reviewers’ comments.
In sum, this paper has some theory contribution and I would like to recommend “accept” once the authors can address/clarify the concerns from the reviewers.
 What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).
5
Author Feedback
We thank the reviewers for their constructive comments. Below is our response to the major comments.
The manuscript lacks some reference. A) We will add references including the papers suggested by the reviewers.
Include short descriptions of the compared methods. A) We will include a short description in the revised manuscript.
Evaluation is made over the greedy variant but application results are reported with the cascaded variant. Why not complete both ways? A) Cascaded BEAR performs RPCA and NMF simultaneously, whereas the synthetic data is suited only for validation of RPCA. In line with the reviewer’s comment, we will add supplementary videos that show the performance of different algorithms on the real data.
What is the number of parameters of BEAR? A) The total number of parameters in BEAR is n × r, where n is the number of rows in the data matrix and r is the target rank.
How does BEAR compare to CNMFE/OASIS? A) CNMFE is a variant of NMF whereas the main focus of BEAR is to perform scalable and high speed RPCA. OASIS is for deconvolution of spike trains, and hence not related to our work.
There is fast and scalable RPCA paper using faster SVD methods. (https://arxiv.org/pdf/1502.00182.pdf) A) We will cite this paper in the revised version. Unfortunately, we could not make a direct comparison as the source code is not publicly available.
How about the number of iterations in time complexity O(nmr)? A) Each iteration in BEAR and SVDbased algorithms has a time complexity of O(nmr) and O(nm^2+n^2m), respectively. The required number of iterations differs for different algorithms, and therefore, the computation time should be compared as in Section 3.1.
What kind of batching is used? A) Each minibatch is constructed by randomly sampling columns.
The time comparison should include the training of the neural net. A) All computation times reported in the manuscript include all time taken for training. We would like to clarify that even the reported computation time of BEAR with inferenceonly includes the time taken for training.
In Eq. 5, what does “localized” in localized sparse footprints correspond to? How is this condition enforced in the cost? A) It was reported (Yang et al 2007) that projective NMF results in high orthogonality between the learned base vectors (i.e., sparsity). Localization is a sidebenefit that comes with sparsity (i.e., a nonsparse component cannot be localized).
How is nonnegativity enforced? A) Nonnegativity is attained by using projected gradient descent.
How can BEAR outperform some methods based on full SVD? A) The original version of RPCA relies on convex relaxation of the optimization problem which requires that L has low rank and S is sparse. This is not met in the topright regime of the phase diagrams and therefore, fails to converge. We speculate that this left the possibility for an approximate method to outperform.
Add videos for Figure 4 and authors can show “residual (YLS)” too. A) We will add multiple videos in the revised version. Adding the residual may not be particularly suited for comparing BEAR and other methods, as the residual of BEAR is always zero by design.
The experiments are limited, since the methods existed before, it’s important to show that the proposed approach existing methods in multiple common datasets. A) As the “ground truth” low rank component cannot exist in real data, the standard method for evaluating RPCA algorithms is to decompose synthetic data with known L and S, as we demonstrated in Section 3.1.
Why did authors train only on ⅓ of data for inferenceonly mode? How is that ⅓ selected? What if there’s a shift in the lowrank dynamics of the data? A) The portion of the data is a userdefined hyperparameter and the first ⅓ of the video was selected. The inferenceonly mode is not suited if a large shift in low rank component is expected.
Postrebuttal MetaReviews
Metareview # 1 (Primary)
 Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your metareview. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
I think the authors have done a good job in addressing the reviewers’ questions. I would like to recommend to accept.
 After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
 What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).
5
Metareview #2
 Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your metareview. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
A simple bilinear neural network is presented for efficient approximation of robust principal component analysis. It achieves an order of magnitude speed improvement with GPU acceleration compared to conventional RPCA algorithms and can perform foregroundbackground separation of gigabytescale calcium imaging data. The paper makes a solid contribution but especially one reviewer asks for clarification on many aspects of the work. This is provided by the authors in their feedback, which seems sufficiently convincing.
 After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
 What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).
4
Metareview #3
 Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your metareview. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper presents a novel and computationally efficient approach to an important problem that is within the scope of MICCAI. Concerns in the reviews included some missing references, which the authors promised to add, and a number of detailed questions, several of which are convincingly addressed in the rebuttal. The main remaining concern is that R3 would have liked to see an even more extensive experimental comparison. However, I would agree with the other reviewers, who found the presented experiments to be convincing, especially in the light of the page limit at MICCAI.
 After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
 What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).
1