Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews

Authors

Juan P. Vigueras-Guillén, Arijit Patra, Ola Engkvist, Frank Seeliger

Abstract

Capsule Networks (CapsNets) is a machine learning architecture proposed to overcome some of the shortcomings of convolutional neural networks (CNNs). However, CapsNets have mainly outperformed CNNs in datasets where images are small and/or the objects to identify have minimal background noise. In this work, we present a new architecture, parallel CapsNets, which exploits the concept of branching the network to isolate certain capsules, allowing each branch to identify different entities. We applied our concept to the two current types of CapsNet architectures, studying the performance for networks with different layers of capsules. We tested our design in a public, highly unbalanced dataset of acute myeloid leukaemia images (15 classes). Our experiments showed that conventional CapsNets show similar performance than our baseline CNN (ResNeXt-50) but depict instability problems. In contrast, parallel CapsNets can outperform ResNeXt-50, is more stable, and shows better rotational invariance than both, conventional CapsNets and ResNeXt-50.

Link to paper

DOI: https://doi.org/10.1007/978-3-030-87234-2_70

SharedIt: https://rdcu.be/cyl9h

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The contribution of this paper is the introduction of capsule net parallelization. The authors apply the technique to leukocyte classification in a public database of patients with acute myeloid leukemia. In the highly unbalanced dataset, the proposed method with parallel DR-caps improves over other capsule networks or a ResNeXt classification network.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Interesting methodology Sufficient experimentation
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Undoubtedly the proposed method outperforms the baseline and other capsule networks, however, it does so at the cost of a threefold increase in the number of parameters with respect to ResNetXt-50.
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Code will be needed to reproduce these results. There are few implementation details.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Capsule networks have not been successful with respect to their convolutional counterparts. The method and experimentation shown here proves otherwise. To make the statement general, this reviewer would like the comparison in other datasets.
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The idea is interesting, but I am not sure the comparison with the reference ResNeXt-50 is a fair comparison against convolutional networks. I would like to see a head-to-head comparison with a network with the same number of parameters. If the performance levels then, the use of parallel capsule networks would not be justified.
What is the ranking of this paper in your review stack?

3
Number of papers in your stack

4
Reviewer confidence

Somewhat confident

Review #2

Please describe the contribution of the paper

Authors adopt Capsule networks for the classification of (15 classes) white blood cells from a public dataset. A parallel Capsule network is proposed, and different network configurations are compared. The proposed method with selected optimal parameters slightly outperforms a classic CNN (ResNet).
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Authors investigate a relatively less studied deep learning direction – capsule network for medical image classification. The utilized dataset seems to be reasonable, with a relatively large size and multiple classes.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Results just show slight improvement compared to a classic CNN. Single-cell image classification is less attractive.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

fair
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

It seems the CapsNets cannot achieve significant performance improvement compared to classic CNNs, although authors tried to tune the network to some extent. The experimental method is not very clear to me. “Once established the best networks, a 5-fold cross-validation (CV) was performed for comparison.” How to establish the best networks? If the 5-fold CV is used to compare different settings (such as different layers, number of branches), it might not be correct – an independent testing set is needed. Lack of comparison with state-of-the-art methods, which are designed for the same or similar tasks.
Please state your overall opinion of the paper

borderline accept (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

just slightly performance improvement compared to a classic CNN. Lack of comparison with state-of-the-art methods.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

4
Reviewer confidence

Somewhat confident

Review #3

Please describe the contribution of the paper

The authors implemented a network consisting of CNNs and parallel CapNets to classify white blood cell types from a public dataset of single-cel images. They compared it with a baseline network based on ResNeXt-50. Different configurations were setup and tested.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Strengths include:
- Parallel architecture of capsule networks (CapNets).
- Appropriate testing methods using 5-fold cross-validation.
- Results show improvement when compared to baseline method that did not use parallel CapNets.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Weaknesses include:
- (minor) AUC of ~0.75 does not provide a lot of confidence in result, above 0.9-0.95 would be better.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Using the public dataset and the methods described, there is a good chance this work can be reproduced with similar results.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://miccai2021.org/en/REVIEWER-GUIDELINES.html

Very good paper. Not much for me to add.
Please state your overall opinion of the paper

strong accept (9)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Well organized paper with strong methods and discussion.
What is the ranking of this paper in your review stack?

1
Number of papers in your stack

5
Reviewer confidence

Very confident

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

Based on the reviewers’ comments, we think this is a high-quality paper with significant technical contributions. Authors should address reviewers’ comments in the camera-ready submission.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers).

2

Author Feedback

N/A

back to top

Parallel Capsule Networks for Classification of White Blood Cells