Object detection with afordable robustness for UAV aerial imagery: model and providing method

Viacheslav Moskalenko, Artem Korobov, Yuriy Moskalenko

Abstract


Neural network object detectors are increasingly being used for aerial video analysis, with a growing demand for onboard processing on UAVs and other limited resources. However, the vulnerability of neural networks to adversarial noise, out-of-distribution data, and fault injections reduces the functionality and reliability of these solutions. The development of detector models and training methods that simultaneously ensure computational efficiency and robustness against disturbances is an urgent scientific task. The research subjects. The model and method for ensuring the robustness of resource-constrained neural network systems for object detection in aerial video surveillance. Objective. Development of a model and method to ensure the robustness of object detectors for aerial image analysis. Methods. Combination of ideas and methods for dynamic neural networks, and methods for robustness and resilience optimization for neural networks. Results. The detector model with a ViT-B/16 backbone modified with gate units for dynamic inference was developed. The model was trained on the VEDAI dataset and meta-trained on the results of adaptation to different types of disturbances. The model with different training methods was tested for robustness against random bit-flip injection where the proportion of the modified weights is determined at a fault rate of 0.1. In addition, the model with different training methods were tested for robustness against a black-box Adversarial Attack with a perturbation level of 3/255 according to the L¥ norm. Conclusions. The object detection model for aerial images with dynamic inference and optimized robustness is developed for the first time. The model includes a transformer-based backbone, gate units, and simplified feature pyramid network with a RetinaNet detection head. Gate units are trained to deactivate transformer encoders that are irrelevant to the input data and disturbances. The proposed model reduces FLOPs by more than 22% without loss of mean Average Precision (mAP) by deactivating some encoders. The detector training method was developed for the first time, combined the RetinaNet loss function with the gate unit loss function and applied meta-learning to the results of adaptation to various types of synthetic disturbances. The analysis of the experimental results demonstrates that the proposed method provides an 11.7 % increase in mAP during testing under fault injection conditions and a 15.1 % increase in mAP during adversarial attack testing.

Keywords


object detection; robustness; adversarial attack; fault injection; meta-learning

Full Text:

PDF

References


Polina, A. M., Suparwito, H., & Kumalasanti, R. A. Aerial object detection analysis: Challenges and preliminary results. E3S Web of Conferences, 2024, vol. 475, article no. 02017. DOI: 10.1051/e3sconf/202447502017.

Marinó, G. C., Petrini, A., Malchiodi, D., & Frasca, M. Deep neural networks compression: A comparative survey and choice recommendations. Neurocomputing, 2023, vol. 520, pp. 152–170. DOI: 10.1016/j.neucom.2022.11.072.

Syed, R., Ulbricht, M., Piotrowski, K., & Krstic, M. A Survey on Fault-Tolerant Methodologies for Deep Neural Networks. Pomiary Automatyka Robotyka, 2023, vol. 27, iss. 2, pp. 89–98. DOI: 10.14313/par_248/89.

Chen, X., Huang, W., Peng, Z., Guo, W., & Zhang, F. Diversity supporting robustness: Enhancing adversarial robustness via differentiated ensemble predictions. Computers & Security, 2024, vol. 142, article no. 103861. DOI: 10.1016/j.cose.2024.103861.

Niu, Z., Chen, Z., Li, L., Yang, Y., Li, B., & Yi, J. On the Limitations of Denoising Strategies as Adversarial Defenses. arXiv, 2020. DOI: 10.48550/ARXIV.2012.09384.

Lust, J., & Condurache, A. P. Efficient detection of adversarial, out-of-distribution and other misclassified samples. Neurocomputing, 2022, vol. 470, pp. 335–343. DOI: 10.1016/j.neucom.2021.05.102.

. Lu, Z., Sun, H., Ji, K., & Kuang, G. Adversarial Robust Aerial Image Recognition Based on Reactive-Proactive Defense Framework with Deep Ensembles. Remote Sensing, 2023, vol. 15, iss. 19, article no. 4660. DOI: 10.3390/rs15194660.

Zaidi, S. S. A., Ansari, M. S., Aslam, A., Kanwal, N., Asghar, M., & Lee, B. A survey of modern deep learning based object detection models. Digital Signal Processing, 2022, vol. 126, article no. 103514. DOI: 10.1016/j.dsp.2022.103514.

Han, Y., Huang, G., Song, S., Yang, L., Wang, H., & Wang, Y. Dynamic Neural Networks: A Survey. arXiv, 2021. DOI: 10.48550/ARXIV.2102.04906.

Meng, L., Li, H., Chen, B.-C., Lan, S., Wu, Z., Jiang, Y.-G., & Lim, S.-N. AdaViT: Adaptive Vision Transformers for Efficient Image Recognition. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, article no. 01199. DOI: 10.1109/cvpr52688.2022.01199.

Tang, G., Ni, J., Zhao, Y., Gu, Y., & Cao, W. A Survey of Object Detection for UAVs Based on Deep Learning. Remote Sensing, 2023, vol. 16, iss. 1, article no. 149. DOI: 10.3390/rs16010149.

Paul, S., & Chen, P.-Y. Vision Transformers Are Robust Learners. Proceedings of the AAAI Conference on Artificial Intelligence, 2022, vol. 36, iss. 2, pp. 2071–2081. DOI: 10.1609/aaai.v36i2.20103..

Li, Y., Fan, Q., Huang, H., Han, Z., & Gu, Q. A Modified YOLOv8 Detection Network for UAV Aerial Image Recognition. Drones, 2023, vol. 7, iss. 5, article no. 304. DOI: 10.3390/drones7050304.

Wu, W., Liu, A., Hu, J., Mo, Y., Xiang, S., Duan, P., & Liang, Q. EUAVDet: An Efficient and Lightweight Object Detector for UAV Aerial Images with an Edge-Based Computing Platform. Drones, 2024, vol. 8, iss. 6, article no. 261. DOI: 10.3390/drones8060261.

Kim, J., Chang, S., & Kwak, N. PQK: Model Compression via Pruning, Quantization, and Knowledge Distillation. arXiv, 2021. DOI: 10.48550/ARXIV.2106.14681.

Hawks, B., Duarte, J., Fraser, N. J., Pappalardo, A., Tran, N., & Umuroglu, Y. Ps and Qs: Quantization-Aware Pruning for Efficient Low Latency Neural Network Inference. Frontiers in Artificial Intelligence, 2021, vol. 4. DOI: 10.3389/frai.2021.676564.

Haque, M., & Yang, W. Dynamic Neural Network is All You Need: Understanding the Robustness of Dynamic Mechanisms in Neural Networks. 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 2023, pp. 1489–1498. DOI: 10.1109/iccvw60793.2023.00163.

Moskalenko, V., Kharchenko, V., Moskalenko, A., & Kuzikov, B. Resilience and Resilient Systems of Artificial Intelligence: Taxonomy, Models and Methods. Algorithms, 2023, vol. 16, iss. 3, article no. 165. DOI: 10.3390/a16030165.

Moskalenko, V., & Moskalenko, A. Neural network based image classifier resilient to destructive perturbation influences – architecture and training method. Radioelectronic and Computer Systems, 2022, iss. 3, pp. 95–109. DOI: 10.32620/reks.2022.3.07.

Lu, Z., Sun, H., Ji, K., & Kuang, G. Adversarial Robust Aerial Image Recognition Based on Reactive-Proactive Defense Framework with Deep Ensembles. Remote Sensing, 2023, vol. 15, iss. 19, article no. 4660. DOI: 10.3390/rs15194660.

Fang, C., Song, Y., Guan, F., Liang, F., & Yang, L. A Robust Complex-Valued Deep Neural Network for Target Recognition of UAV SAR Imagery. IEEE Journal on Miniaturization for Air and Space Systems, 2023, vol. 4, iss. 2, pp. 175–185. DOI: 10.1109/jmass.2023.3247586.

Wang, J., Zhang, Z., Wang, M., Qiu, H., Zhang, T., Li, Q., Li, Z., Wei, T., & Zhang, C. Aegis: Mitigating Targeted Bit-flip Attacks against Deep Neural Networks. arXiv, 2023. DOI: 10.48550/ARXIV.2302.13520.

Haque, M., & Yang, W. Dynamic Neural Network is All You Need: Understanding the Robustness of Dynamic Mechanisms in Neural Networks. 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 2023, pp. 1489-1498. DOI: 10.1109/iccvw60793.2023.00163.

Cavagnero, N., Santos, F. D., Ciccone, M., Averta, G., Tommasi, T., & Rech, P. Transient-Fault-Aware Design and Training to Enhance DNNs Reliability with Zero-Overhead. 2022 IEEE 28th International Symposium on On-Line Testing and Robust System Design (IOLTS), 2022. DOI: 10.1109/iolts56730.2022.9897813.

Dong, J., Qiu, H., Li, Y., Zhang, T., Li, Y., Lai, Z., Zhang, C., & Xia, S.-T. One-bit Flip is All You Need: When Bit-flip Attack Meets Model Training. 2023 IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 4665-4675. DOI: 10.1109/iccv51070.2023.00432.

Cao, H., & Xue, M. Adversarial Training for Better Robustness. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, Springer Nature Switzerland, 2023, pp. 75-84. DOI: 10.1007/978-3-031-35982-8_6.

Bai, T., Luo, J., Zhao, J., Wen, B., & Wang, Q. Recent Advances in Adversarial Training for Adversarial Robustness. Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI-21), 2021, pp. 4312–4321. DOI: 10.24963/ijcai.2021/591.

Athalye, A., Carlini, N., & Wagner, D. Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples. arXiv, 2018. DOI: 10.48550/ARXIV.1802.00420.

Qiu, P., Wang, Q., Wang, D., Lyu, Y., Lu, Z., & Qu, G. Mitigating Adversarial Attacks for Deep Neural Networks by Input Deformation and Augmentation. 2020 25th Asia and South Pacific Design Automation Conference (ASP-DAC), 2020, pp. 157–162. DOI: 10.1109/asp-dac47756.2020.9045107.

Zhang, B., Tondi, B., Lv, X., & Barni, M. Challenging the Adversarial Robustness of DNNs Based on Error-Correcting Output Codes. Security and Communication Networks, 2020, vol. 2020, pp. 1–11. DOI: 10.1155/2020/8882494.

Wang, R., Xu, K., Liu, S., Chen, P.-Y., Weng, T.-W., Gan, C., & Wang, M. On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learning (Version 1). arXiv, 2021. DOI: 10.48550/ARXIV.2102.10454.

Maurício, J., Domingues, I., & Bernardino, J. Comparing Vision Transformers and Convolutional Neural Networks for Image Classification: A Literature Review. Applied Sciences, 2023, vol. 13, iss. 9, article no. 5521. DOI: 10.3390/app13095521.

Wang, L., & Tien, A. Aerial Image Object Detection with Vision Transformer Detector (ViTDet). IGARSS 2023 - 2023 IEEE International Geoscience and Remote Sensing Symposium, 2023. DOI: 10.1109/igarss52108.2023.10282836.

Li, Y., Mao, H., Girshick, R., & He, K. Exploring Plain Vision Transformer Backbones for Object Detection. Lecture Notes in Computer Science, Springer Nature Switzerland, 2022, pp. 280–296. DOI: 10.1007/978-3-031-20077-9_17.

Yuldashev, Y., Mukhiddinov, M., Abdusalomov, A. B., Nasimov, R., & Cho, J. Parking Lot Occupancy Detection with Improved MobileNetV3. Sensors, 2023, vol. 23, iss. 17, article no. 7642. DOI: 10.3390/s23177642.

Nawaz, S. A., Li, J., Bhatti, U. A., Shoukat, M. U., & Ahmad, R. M. AI-based object detection latest trends in remote sensing, multimedia and agriculture applications. Frontiers in Plant Science, 2022, vol. 13. DOI: 10.3389/fpls.2022.1041514.

Assran, M., Caron, M., Misra, I., Bojanowski, P., Bordes, F., Vincent, P., Joulin, A., Rabbat, M., & Ballas, N. Masked Siamese Networks for Label-Efficient Learning. Lecture Notes in Computer Science, Springer Nature Switzerland, 2022, pp. 456–473. DOI: 10.1007/978-3-031-19821-2_26.

Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., & Joulin, A. Emerging Properties in Self-Supervised Vision Transformers. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021, article no. 00951. DOI: 10.1109/iccv48922.2021.00951.

Pinetsuksai, N., Kittichai, V., Jomtarak, R., Jaksukam, K., Tongloy, T., Boonsang, S., & Chuwongin, S. Development of Self-Supervised Learning with Dinov2-Distilled Models for Parasite Classification in Screening. 2023 15th International Conference on Information Technology and Electrical Engineering (ICITEE), 2023, pp. 323-328. DOI: 10.1109/icitee59582.2023.10317719.

Moskalenko, V. V. Model-Agnostic Meta-Learning for Resilience Optimization of Artificial Intelligence System. Radio Electronics, Computer Science, Control, 2023, iss. 2, article no. 79. DOI: 10.15588/1607-3274-2023-2-9.

Kulkarni, U., Meena, S. M., Hallyal, R., Sulibhavi, P., Sunil, V. G., Guggari, S., & Shanbhag, A. R. Optimisation of deep neural network model using Reptile meta learning approach. Cognitive Computation and Systems, Institution of Engineering and Technology (IET), 2023. DOI: 10.1049/ccs2.12096.

Kotyan, S., & Vargas, D. V. Adversarial robustness assessment: Why in evaluation both L0 and L∞ attacks are necessary. PLOS ONE, 2022, vol. 17, iss. 4, article no. e0265723. DOI: 10.1371/journal.pone.0265723.

Li, G., Pattabiraman, K., & DeBardeleben, N. TensorFI: A Configurable Fault Injector for TensorFlow Applications. 2018 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW), 2018. DOI: 10.1109/issrew.2018.00024.

Zhao, Y., Sun, H., & Wang, S. Small Object Detection in Medium–Low-Resolution Remote Sensing Images Based on Degradation Reconstruction. Remote Sensing, 2024, vol. 16, iss. 14, article no. 2645. DOI: 10.3390/rs16142645.

Padilla, R., Passos, W. L., Dias, T. L. B., Netto, S. L., & da Silva, E. A. B. A Comparative Analysis of Object Detection Metrics with a Companion Open-Source Toolkit. Electronics, 2021, vol. 10, iss. 3, article no. 279. DOI: 10.3390/electronics10030279.

Shen, L., Lang, B., Song, Z. Infrared Object Detection Method Based on DBD-YOLOv8. IEEE Access, 2023, vol. 11, pp. 145853–145868. DOI: 10.1109/access.2023.3345889.




DOI: https://doi.org/10.32620/reks.2024.3.04

Refbacks

  • There are currently no refbacks.