Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews
Back to top List of papers List of papers - by topics Author List
Paper Info Reviews Meta-review Author Feedback Post-Rebuttal Meta-reviews

Authors

Luojie Huang, Gregory N. McKay, Nicholas J. Durr

Abstract

Oblique back-illumination capillaroscopy has recently been introduced as an efficient method for non-invasive blood cell imaging inhuman capillaries. To apply this technique to clinical blood cell counting, solutions for automatic processing of acquired videos are needed. Here, we take the first step towards this goal, by introducing a novel deep learning multi-cell tracking model, named CycleTrack, which achieves accurate blood cell counting from capillaroscopic videos. CycleTrack combines two simple online tracking models, SORT and CenterTrack, and is tailored to features of capillary blood cell flow. Blood cells are tracked by displacement vectors in two opposing temporal directions between consecutive frames. This approach yields accurate tracking despite rapidly moving and deforming blood cells. The proposed model outperforms other baseline trackers, achieving 66.3% MOTA and 75.1% ID F1 score on test videos. CycleTrack achieves an average cell counting error of 3.42% among 8 1000-frame test videos, compared to 6.55% and 22.98% from original CenterTrack and SORT, with negligible time expense. It takes 800s to track and count approximately 8000 blood cells from 9,600 frames captured in a typical one-minute video. Moreover, the blood cell velocity measured by CycleTrack demonstrates a consistent, pulsatile pattern within the physiological range of heart rate. The project code is accessible online at: https://github.com/DurrLab/CycleTrack.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87237-3_40

SharedIt: https://rdcu.be/cymaw

Link to the code repository

https://github.com/DurrLab/CycleTrack

Link to the dataset(s)

N/A

Reviews

Review #1

  • Please describe the contribution of the paper

    Two simple on-line tracking algorithms (SORT and CenterTrack) are combined to create the proposed CycleTrack, where positional information is adopted to track and count multiple cells in oblique back-illumination capillaroscopy. The proposed CenterTrack combines SORT forward tracking and CenterTrack backward tracking

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper makes a probably correct assumption of movement pattern of blood cells in oblique back-illumination capillaroscopy. As a result, the proposed tracking architecture enhances MOTA from 62.4% to 66.3%.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    A couple parts of the paper are not clear:

    1. The notation H^(t) in the Fig. 1 hasn’t been explained.
    2. The direction of displacement derived by CenterTrack hasn’t been explicitly defined that it is backward.
    3. Notation w used in CenterTrack to present both confidence score and the weight of the displacement vector prediction is confusing.
    4. How SORT was used for forward prediction hasn’t been elaborated in details.
    5. There are many visual tracking algorithms in the MOT literature. So why select the combination of SORT and CenterTrack as your solution should be better motivated. Why not, say, apply the best forward tracking method in backward fashion as your combined solution? Also as pointed out in the paper, the detection and tracking in CenterTrack is performed and optimized end-to-end; however, SORT tracking can be performed independent of the given detections; is this one of the reason you choose SORT + CenterTrack to basically improve CenterTrack via SORT??
  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    Training techniques (data augmentation, loss function, and etc.), deep learning models, testing environments (Pytorch, Nvidia GPU) are directly noted, such that these shall be reproducible.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

    In the object association of the method, minimum of the output from the two algorithms is selected as the candidate of the cost matrix. It means that the result would be from one of them, the algorithms could be, therefore, not “combined” as expected.

  • Please state your overall opinion of the paper

    borderline accept (6)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    There’s only 3.9% enhancement from 62.4% of CenterTrack that it is a small improvement of accuracy. Moreover, as mentioned in 4., there are some details to be clarified, so that the improvement can be justified.

  • What is the ranking of this paper in your review stack?

    1

  • Number of papers in your stack

    3

  • Reviewer confidence

    Confident but not absolutely certain



Review #2

  • Please describe the contribution of the paper

    The paper proposes a new method for tracking cells in 2D capillaroscopy images. Specifically they use a tracking by detection approach as previously described with CenterNet/Centertrack that predicts object centers/bounding box sizes and displacement vectors backwards in time. They additionally use a Kalman filter (SORT) to predict displacement vectors forward in time. Both displacements are used to create a combined matching score matrix that is used greedily to produce the final tracking results. They evaluate their method on a real dataset of blood cell images and compare against several competing tracking methods.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • Simple idea, combining two effective methods
    • Seems to work quite well for the specific dataset
    • Broad comparison with competing methods
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • Hard to asses whether the results generalizes to other datasets different from the very special modality the authors have chosen
    • No code available
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    No data and code given, thus hard to reproduce.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

    • Overall the paper is well written and well to follow. It is however hard to asses, whether the relatively simple heuristic of combing SORT and displacement in the presented way (simply taking the minimum) is really the most relevant factor here, or whether its due to the relatively specific dataset of unidrectionally flowing bloodcells. Comparing the method on another dataset (e.g. from the celltracking challenge) would have made this more convincing.

    • It might be good to precisely state what “Cycletrack” refers to during section 3.3.

    • Eq (1): What is the motivation to compute this weighting? It appears that if cells would not all move along a single preferred direction (which is might be the case for blood flow, but not e.g. for cells in expanding tissue), the so obtained displacements would be rather incorrect.

    • Fig 1: Currently its hard to have a good overview of what cells / ground truth bounding boxes are looking like (i.e. what the specific problem is). I would recommend to make that clearer

  • Please state your overall opinion of the paper

    borderline accept (6)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Simple idea which seems to help, not entirely sure whether it might be a heuristic that does not generalize to other datasets.

  • What is the ranking of this paper in your review stack?

    2

  • Number of papers in your stack

    5

  • Reviewer confidence

    Confident but not absolutely certain



Review #3

  • Please describe the contribution of the paper

    This manuscripts presents a bi-directional method for tracking cells from blood cells from non-invasive capillaroscopy videos. This particular data type is novel, which is one of the contributions of this submission. On top of that, the authors combine two different tracking methods, one forward and one backward, for achieving superior performance.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    • Novel data type . • Good performance with respect to cell tracking.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    • The presented approach is essentially an ensemble of two tracking approaches (which happen to work in opposite directions). The authors did not provide any evidence of the benefits using the bi-directional approach.
    • This work is poorly illustrated, as most of the figures are unreadable due to small size and poor quality.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    The authors did not provide any information about public availability of their method and the validation data set. The implementation is mostly clear, but not entirely. The values of the method parameters are reported.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html
    1. The proposed method is an ensemble of two tracking methods and, as expected, outperforms both combined approaches. At the same time, while the authors make emphasis on bi-directionality of their approach, in the manuscript I did not find any evidence of benefits it provides. Meaning that the authors could have as well chosen two approaches that perform tracking in the same direction.
    2. Majority of the figures (Fig. 1A, 2BC, Supp. Fig. 1) is virtually unreadable due to small size and poor quality. Whereas it is, in my opinion, important to present a good visualization (in addition to the Supplemental Movie) of the novel data type being quantified by the authors.
    3. Page 2: “Hemocue (Ängelholm, Sweden) is a portable hemoglobinometers…” → “Hemocue (Ängelholm, Sweden) is a portable hemoglobinometer…”
    4. Page 4, Section 3.1.: variables “W” and “H” should be defined for completeness.
    5. Page 4, Equation 2: there should be a hat symbol above “d”.
    6. Page 4, Section 3.3: it is not entirely clear how the set of detections is constructed. Do the authors use detections returned by one of the methods (which?) or the combined set?
    7. Page 5, Section 3.3: “if all the distances of a detection in the matrix are out of a reasonable range”. Please specify the cutoff value.
    8. Page 6: “… it is also importance to noted…” → “… it is also important to note”.
    9. Page 7: “…as the base vector from CenterTrack gradually stabilize…” → “…as the base vector from CenterTrack gradually stabilizes…”.
    10. Page 7, Section 5.2: “And the average coefficient of variation…”. I find it difficult to connect this sentence to the previous one, as the latter reports the acceptable accuracy values and here the authors switch to the coefficient of variation, which is related only indirectly.
  • Please state your overall opinion of the paper

    borderline accept (6)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This is a fair conference submission, in which the authors present an ensemble tracking approach for tracking cells from a novel data type. The performance of their method is, as expected, superior to both combined approaches. One critique is that, in my opinion, the authors should have put more effort into demonstrating the benefits of using a bi-directional tracking approach, or, alternatively, the “bi-directionality” property should be downplayed.

  • What is the ranking of this paper in your review stack?

    3

  • Number of papers in your stack

    5

  • Reviewer confidence

    Very confident



Primary Meta-Review

  • Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The reviewers unanimously consider the paper borderline acceptable. The problem and proposed solution are interesting and the paper is clear. However, there are quite some points of concern regarding both the method and the experiments that need to be addressed before the paper can be further considered. See the reviewer comments for details.

  • What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

    3



Author Feedback

We thank the reviewers for their insightful comments and suggestions, which we address below.

Reviewers 1 and 3 ask about the motivation for combining SORT and CenterTrack to form the temporally bi-directional CycleTrack. First, we want to emphasize that non-invasive blood analysis requires both high-speed and highly accurate blood cell counting to be clinically-useful. Second, in capillaroscopy images, blood cells look similar to neighbors and are highly deformable frame-to-frame. We did benchmark with two SOTA appearance-based trackers (Tracktor++ and MaskTrack in Table 1), and, as expected, observed both poor accuracy and slow speed when compared to position-based trackers (SORT and CenterTrack in Table 1). As described in Section 2.3, we choose SORT and CenterTrack because they are fast position-based trackers that exploit complementary behaviors observed in capillaroscopy data.

Reviewer 1 suggests that greedy matching may not be ideal, but our experiments found this approach gave the best results, likely due to one tracker occasionally failing.

Reviewer 3 has a reasonable concern regarding our use of “bi-directional” to describe our model. One benefit of combining SORT and CenterTrack is that SORT maintains a long-term memory of flow velocity, whereas CenterTrack allows for short-term changes in velocity while enforcing similar relative positions of detected cells. At OBC frame rates, pulsatile blood flow exhibits periods of both flow behaviors. As predictive models (SORT) are typically implemented in a forward direction, while tracking-by-detection models (CenterTrack) are typically implemented in a backward direction, we call our method “bi-directional” to concisely capture these model differences and source of the performance benefit. Moreover, these two trackers can be combined efficiently in a high-speed bi-directional framework that maintains the capability of online tracking.

Reviewer 1 is concerned that CycleTrack only improves the MOTA by 3.9% from 62.4% of CenterTrack. We believe this is a reasonable improvement. On the MOT17 challenge leaderboard, a reduction of 3.9% MOTA would move the best model down to 4th place. For our cell counting task, we emphasize that CycleTrack reduces ID switches by almost 50% (from 59.8% with CenterTrack to 34.5% with CycleTrack). This improvement is largely responsible for the significant reduction in red blood cell counting error, from 6.55% of CenterTrack to 3.42% of CycleTrack.

Reviewer 2 suggests applying CycleTrack to previous cell tracking challenges to assess generalizability. Though we are tracking cells, the task in our application is quite different from previous challenges, which generally focus on precise slide cell segmentation for cells that exhibit small movements, relatively-constant shapes across frames and unique features compared to other cells. Blood cells in OBC videos do not follow any of these three behaviors, and the clinical goal is simply counting instead of precise segmentation. As mentioned in Section 2.3, we built our model based on two general assumptions from blood cell flow. CycleTrack can generalize to other datasets that meet these two criteria, such as blood flow in different vessels, cells flowing in microfluidic channels, and urine sediment flow analysis. Unfortunately, there are no publicly-available medical datasets from these applications to test our algorithm.

Reviewer 2 and 3 are also concerned about the model reproducibility. We have uploaded our code with the trained model onto Github and will include the link in the camera-ready version, so that readers can use our model on their datasets and explore more applications.

We would like to thank Reviewer 2 and 3 for pointing out the issue of figure quality. We will improve image quality in the camera-ready submission to ensure readability. As for the bounding boxes mentioned by Reviewer 2, we include a step-by-step workflow visualization in the Supplementary Figure 1.



Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    The paper proposes a deep-learning based bidirectional tracking method for automated blood cell counting in oblique back-illumination capillaroscopy videos. It combines two online tracking models to predict back-and-forth cell displacements to achieve optimal matching between newly detected cells and previously tracked cells in two consecutive video frames. Experimental results show the method outperforms several existing multi-object tracking methods and is capable of robust cell counting with close to clinically acceptable accuracy. There are concerns about generalizability beyond the specific private data set used in the experiments. Also, the benefits of using a bidirectional tracking approach are not sufficiently demonstrated. These concerns are addressed to some extent by the authors. Overall the consensus is that this is a fair paper that can be accepted for MICCAI 2021.

  • After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

    Accept

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

    6



Meta-review #2

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    The authors have adequately responded to the major concerns of the reviewers. Indeed, the authors should include these explanations in the final version of the paper and improve the overall quality of the paper.

  • After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

    Accept

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

    5



Meta-review #3

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    This paper uses SORT tracker and CenterTrack to track blood cells, in the forward and backward directions, respectively, and then uses a basic min() operation and greedy matching strategy on the two tracking cost matrices to associate cells between two consecutive frames. It is hard to see the technical contributions from this ensemble method.

  • After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

    Reject

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

    15



back to top