[2020 CVPR Efficient DET paper review]
- 1. Presented by ChanHyuk Lee 2021/06/13 Computer Graphics @ Korea University EfficientDet MingxingTan et al. CVPR 2020 517 citation 1/
- 2. CONTENTS Introduction 01 Related work 02 Proposed method 03 Experiments 04 Ablation study 05 Conclusion 05 2
- 3. 3 Background Detection architecture 00 Backbone network FPN Prediction Network Box prediction (Regression) Class prediction (Classification) Backbone network Feature Pyramid Network Prediction network
- 4. Introduction โข Recent detectors have the trade-off between accuracy and efficiency โข Most previous works only focus on a specific or a small range of resource requirements โข This points make hard to apply the recent detection models on industry field โข โIs it possible to build a scalable detection architecture with both higher accuracy and better efficiency across a wide spectrum of resource constraints?โ Motivation 01 4
- 5. Introduction Challenge 1. Efficient multi-scale feature fusion 01 5 โข Feature fusion : The method for combining feature maps โ Normal feature fusion methods donโt care about feature resolution. Challenge 2 : Model scaling โข Model scaling : The method for up-scaling the model architecture โ Limitation of up-scaling by considering one factor Input-image up-scaling Network up-scaling 02
- 7. Related work Multi-scale feature representation 01 Conv Conv Conv Conv Up scaling Up scaling Up scaling 1x1 Conv 1x1 Conv 1x1 Conv 1x1 Conv Prediction Prediction Prediction Prediction Backbone Feature pyramid ๐๐๐๐๐ ๐๐๐๐๐ ๐๐๐๐๐ ๐๐๐๐๐ ๐๐ ๐๐ ๐๐ ๐๐ 7 โข For considering multi-scale object Area Prediction layer
- 8. Related work Model scaling 02 โข EfficientNet (EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks, Mingxing Tan et al, ICML 2019) โข Jointly Scale up the depth, width, resolution (Compound scaling) 8 ๐ ๐ ๐ ๐ ๐ท๐๐๐กโ ๐ผ๐๐๐ข๐ก ๐๐๐ ๐๐๐ข๐ก๐๐๐
- 9. Q&A 9
- 10. Proposed method 01 RetinaNet architecture 10 02 EfficientDet architecture
- 11. BiFPN : Efficient bidirectional cross-scale connections and weighted feature fusion Problem formulation 01 11 โข Delete two blocks (compared to PANet) โข Add skip connection โข Weighted feature fusion โข Repeat BiFPN Layers ๐ค ๐ค ๐ค ๐ค ๐ค ๐ค ๐ค ๐ค ๐ค ๐ค ๐ค ๐ค ๐ค
- 12. BiFPN Weighted Feature Fusion 02 โข The difference of Resolution between Inputs โ Different degrees of contribution to output โข Gave each input feature a weight to learn the contribution of the input feature. ๐ถ๐๐๐๐๐ ๐๐๐๐๐๐๐ ๐พ๐๐๐๐๐๐ ๐ฐ๐๐๐๐ ๐๐๐๐๐๐๐๐ ๐บ๐๐๐๐๐๐ โ ๐๐๐๐๐ ๐๐๐๐๐๐ ๐ญ๐๐๐ ๐๐๐๐๐๐๐๐๐๐ ๐๐๐๐๐๐ (30% Speed Gain in GPU) 12
- 13. EfficientDet EfficientDet Architecture 01 โข Using the efficientNet trained by ImageNet Data as backbone โข The Prediction layer networkโs weights is shared for all Level features 13
- 14. EfficientDet Compound scaling 02 โข Previous works mostly scale up baseline network or using larger image inputs, stacking more FPN layers โข New compound scaling method jointly scale up all dimensions of backbone network, BiFPN network, prediction network and resolution of input. Backbone network 02-1 โข Reuse the same width/depth scaling coefficients of EfficientNet-B0 to B6 BiFPN network 02-2 โข Perform grid search for finding best factor value on a list of values {1.2, 1.25, 1.3, 1.35, 1.4, 1.45} ๐โ๐ ๐๐ข๐๐๐๐ ๐๐ ๐โ๐๐๐๐๐ ๐โ๐ ๐๐ข๐๐๐๐ ๐๐ ๐๐๐ฆ๐๐ 14
- 15. EfficientDet Prediction network 02-3 โข The width of network is same as BiFPN network's width ๐โ๐ ๐๐ข๐๐๐๐ ๐๐ ๐๐๐ฆ๐๐ Input image resolution 02-4 Overall scaling output 02-5 15
- 16. Q&A 16
- 17. Experiments Experiment configuration 01 โข Dataset : COCO 2017 datasets with 118K images โข Optimizer : SGD with momentum 0.9 and weight decay 4e-5 โข Learning Rate : 0 to 0.16 (First epoch), annealed down using cosine decay rule (0~0.16 ๐๐๐๐๐๐ก) โข Batch normalization is used after every convolution layer โข Every convolution layer is depth-wise conv layer โข Activation function : Swish (๐ฅ โ ๐๐๐๐๐๐๐(๐ฝ๐ฅ)) โข Augmentation : Multi-resolution cropping / scaling / flipping 17
- 18. Experiments Loss function 02 โข Using Focal-loss for detection โข Class imbalanced problem is most effected by easy negative samples โข Training by focusing on hard samples โข If ๐๐ก is almost 1 โ โ 1 โ 0.999 ๐ ๐๐๐ ๐๐ก โ 0 โข Else โ โ 1 โ 0.001 ๐ ๐๐๐(๐๐ก) โ โ ๐๐๐๐๐๐๐๐๐๐ก๐ฆ ๐๐ ๐๐๐๐ ๐ ๐๐๐๐๐๐ก๐๐๐ 18
- 19. Experiments Performance on COCO 03 โข Latency is inference latency with batch size 1 โข AA denotes Auto-Augmentation 19
- 20. Experiments Model size and inference latency comparison 04 โข The comparison result of using GPU (Titan-V), CPU (Xeon) 20
- 21. Experiments EfficientDet for Semantic Segmentation 05 โข Use P2 Layer in BiFPN for semantic segmentation in EfficientDet-D4 model DeepLabv3 21
- 22. Ablation study Disentangling Backbone and BiFPN 01 โข The Backbone network and multi-feature network of EfficientDet achieves higher AP and Efficiency than prior networks 22
- 23. Ablation study BiFPN Cross Scale Connection 02 โข For the fair comparison, FPN and PANet are repeated multiple times and change the conv. โข BiFPN achieves the best accuracy with fewer parameters and FLOPs 23
- 24. Ablation study Softmax vs Fast Normalized fusion 03 โข Fast normalized fusion approach achieves similar accuracy as the softmax-based method โข Figure 5 illustrates the learned weights for three feature fusion nodes 24
- 25. Ablation study Compound Scaling 04 โข EfficientDet jointly scale up the networkโs backbone, BiFPN, prediction net, input resolution โข The proposed method achieves the best accuracy than other scaling method 25
- 26. Conclusion Propose the weight bidirectional feature network and customized compound scaling method, in order to improve accuracy and efficiency 01 EfficientDet achieves better accuracy and efficiency than the prior art across a wide spectrum of resource constrains 02 EfficientDet achieves SOTA accuracy with much fewer parameters and FLOPs in object detection and semantic segmentation 03 26
- 27. THANK YOU 27