Model and training method for aerial image object detector with optimization of both robustness and computational efficiency

Alona Moskalenko, Mykola Zaretskyi, Maksym Vynohradov, Vladyslav Babych

Abstract


The subject of research is Neural network-based object detectors, which are widely used for video image analysis. An increasing number of tasks now demand data processing directly at the source, which limits the available computational resources. However, the vulnerability of neural networks to noise, adversarial attacks, and weight error injections significantly diminishes their robustness and overall effectiveness. The relevant task is to develop models that provide both computational efficiency and robustness against perturbations. This paper investigates a model and method for enhancing the robustness of neural network detectors under limited resources. The objective is to design a model that allocates resources optimally while maintaining stability. To achieve this, the study employs techniques such as dynamic neural networks, robustness optimization, and resilience strategies. The following results were obtained. A detector with a feature extractor based on ViT-S/16, modified with gate modules for dynamic examination was developed. The model was trained on the RSOD dataset and meta-learned on the adaptation results to various perturbations. The model's resistance to random bit inversions in weights (10 % of weights) and to adversarial attacks with perturbation amplitudes up to 3/255 (L∞ norm) was tested. Conclusion. The proposed detector model incorporating dynamic examination and optimized robustness, reduced floating-point operations by over 20 % without loss of accuracy. A novel method for training the detector was developed, combining the RetinaNet loss function with the loss function of gate blocks and applying meta-learning on the adaptation results for various types of synthetic perturbations. Testing demonstrated an increase in accuracy by 11.9 % under the influence of error injection and by 13.2 % under the influence of adversarial attacks.

Keywords


object detection; robustness; adversarial attacks; fault injections; meta-learning

References


Polina, A. M., Suparwito, H., & Kumalasanti, R. A. Aerial object detection analysis: Challenges and preliminary results. E3S Web of Conferences, 2024, vol. 475, article no. 02017. DOI: 10.1051/e3sconf/202447502017.

Marinó, G. C., Petrini, A., Malchiodi, D., & Frasca, M. Deep neural networks compression: A comparative survey and choice recommendations. Neurocomputing, 2023, vol. 520, pp. 152–170. DOI: 10.1016/j.neucom.2022.11.072.

Syed, R., Ulbricht, M., Piotrowski, K., & Krstic, M. A Survey on Fault-Tolerant Methodologies for Deep Neural Networks. Pomiary Automatyka Robotyka, 2023, vol. 27, iss. 2, pp. 89-98. DOI: 10.14313/par_248/89.

Chen, X., Huang, W., Peng, Z., Guo, W., & Zhang, F. Diversity supporting robustness: Enhancing adversarial robustness via differentiated ensemble predictions. Computers & Security, 2024, vol. 142, article no. 103861. DOI: 10.1016/j.cose.2024.103861.

Niu, Z., Chen, Z., Li, L., Yang, Y., Li, B., & Yi, J. On the Limitations of Denoising Strategies as Adversarial Defenses. arXiv, 2020. DOI: 10.48550/ARXIV.2012.09384.

Lust, J., & Condurache, A. P. Efficient detection of adversarial, out-of-distribution and other misclassified samples. Neurocomputing, 2022, vol. 470, pp. 335–343. DOI: 10.1016/j.neucom.2021.05.102.

Lu, Z., Sun, H., Ji, K., & Kuang, G. Adversarial Robust Aerial Image Recognition Based on Reactive-Proactive Defense Framework with Deep Ensembles. Remote Sensing, 2023, vol. 15, iss. 19, article no. 4660. DOI: 10.3390/rs15194660.

Zaidi, S. S. A., Ansari, M. S., Aslam, A., Kanwal, N., Asghar, M., & Lee, B. A survey of modern deep learning based object detection models. Digital Signal Processing, 2022, vol. 126, article no. 103514. DOI: 10.1016/j.dsp.2022.103514.

Han, Y., Huang, G., Song, S., Yang, L., Wang, H., & Wang, Y. Dynamic Neural Networks: A Survey. arXiv, 2021. DOI: 10.48550/ARXIV.2102.04906.

Meng, L., Li, H., Chen, B.-C., Lan, S., Wu, Z., Jiang, Y.-G., & Lim, S.-N. AdaViT: Adaptive Vision Transformers for Efficient Image Recognition. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, article no. 01199. DOI: 10.1109/cvpr52688.2022.01199.

Paul, S., & Chen, P.-Y. Vision Transformers Are Robust Learners. Proceedings of the AAAI Conference on Artificial Intelligence, 2022, vol. 36, iss. 2, pp. 2071-2081. DOI: 10.1609/aaai.v36i2.20103.

Kim, J., Chang, S., & Kwak, N. PQK: Model Compression via Pruning, Quantization, and Knowledge Distillation. arXiv, 2021. DOI: 10.48550/ARXIV.2106.14681.

Hawks, B., Duarte, J., Fraser, N. J., Pappalardo, A., Tran, N., & Umuroglu, Y. Ps and Qs: Quantization-Aware Pruning for Efficient Low Latency Neural Network Inference. Frontiers in Artificial Intelligence, 2021, vol. 4. DOI: 10.3389/frai.2021.676564.

Haque, M., & Yang, W. Dynamic Neural Network is All You Need: Understanding the Robustness of Dynamic Mechanisms in Neural Networks. 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 2023, pp. 1489–1498. DOI: 10.1109/iccvw60793.2023.00163.

Moskalenko, V., Kharchenko, V., Moskalenko, A., & Kuzikov, B. Resilience and Resilient Systems of Artificial Intelligence: Taxonomy, Models and Methods. Algorithms, 2023, vol. 16, iss. 3, article no. 165. DOI: 10.3390/a16030165.

Moskalenko, V., & Moskalenko, A. Neural network based image classifier resilient to destructive perturbation influences – architecture and training method. Radioelectronic and Computer Systems, 2022, no. 3, pp. 95-109. DOI: 10.32620/reks.2022.3.07.

Wang, J., Zhang, Z., Wang, M., Qiu, H., Zhang, T., Li, Q., Li, Z., Wei, T., & Zhang, C. Aegis: Mitigating Targeted Bit-flip Attacks against Deep Neural Networks. arXiv, 2023. DOI: 10.48550/ARXIV.2302.13520.

Haque, M., & Yang, W. Dynamic Neural Network is All You Need: Understanding the Robustness of Dynamic Mechanisms in Neural Networks. 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 2023, pp. 1489-1498. DOI: 10.1109/iccvw60793.2023.00163.

Cavagnero, N., Santos, F. D., Ciccone, M., Averta, G., Tommasi, T., & Rech, P. Transient-Fault-Aware Design and Training to Enhance DNNs Reliability with Zero-Overhead. IEEE 28th International Symposium on On-Line Testing and Robust System Design (IOLTS), 2022. DOI: 10.1109/iolts56730.2022.9897813.

Dong, J., Qiu, H., Li, Y., Zhang, T., Li, Y., Lai, Z., Zhang, C., & Xia, S.-T. One-bit Flip is All You Need: When Bit-flip Attack Meets Model Training. 2023 IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 4665-4675. DOI: 10.1109/iccv51070.2023.00432.

Cao, H., & Xue, M. Adversarial Training for Better Robustness. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, Springer Nature Switzerland, 2023, pp. 75-84. DOI: 10.1007/978-3-031-35982-8_6.

Bai, T., Luo, J., Zhao, J., Wen, B., & Wang, Q. Recent Advances in Adversarial Training for Adversarial Robustness. Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI-21), 2021, pp. 4312-4321. DOI: 10.24963/ijcai.2021/591.

Athalye, A., Carlini, N., & Wagner, D. Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples. arXiv, 2018. DOI: 10.48550/ARXIV.1802.00420.

Qiu, P., Wang, Q., Wang, D., Lyu, Y., Lu, Z., & Qu, G. Mitigating Adversarial Attacks for Deep Neural Networks by Input Deformation and Augmentation. 2020 25th Asia and South Pacific Design Automation Conference (ASP-DAC), 2020, pp. 157-162. DOI: 10.1109/asp-dac47756.2020.9045107.

Zhang, B., Tondi, B., Lv, X., & Barni, M. Challenging the Adversarial Robustness of DNNs Based on Error-Correcting Output Codes. Security and Communication Networks, 2020, vol. 2020, pp. 1–11. DOI: 10.1155/2020/8882494.

Maurício, J., Domingues, I., & Bernardino, J. Comparing Vision Transformers and Convolutional Neural Networks for Image Classification: A Literature Review. Applied Sciences, 2023, vol. 13, iss. 9, article no. 5521. DOI: 10.3390/app13095521.

Moskalenko, V., Korobov, A., & Moskalenko, Y. Object detection with affordable robustness for UAV aerial imagery: model and providing method. Radioelectronic and Computer Systems, 2024, no. 3, pp. 55–66. DOI: 10.32620/reks.2024.3.04.

Wang, L., & Tien, A. Aerial Image Object Detection with Vision Transformer Detector (ViTDet). IGARSS 2023 - 2023 IEEE International Geoscience and Remote Sensing Symposium, 2023. DOI: 10.1109/igarss52108.2023.10282836.

Li, Y., Mao, H., Girshick, R., & He, K. Exploring Plain Vision Transformer Backbones for Object Detection. Lecture Notes in Computer Science, Springer Nature Switzerland, 2022, pp. 280–296. DOI: 10.1007/978-3-031-20077-9_17.

Yuldashev, Y., Mukhiddinov, M., Abdusalomov, A. B., Nasimov, R., & Cho, J. Parking Lot Occupancy Detection with Improved MobileNetV3. Sensors, 2023, vol. 23, iss. 17, article no. 7642. DOI: 10.3390/s23177642.

Nawaz, S. A., Li, J., Bhatti, U. A., Shoukat, M. U., & Ahmad, R. M. AI-based object detection latest trends in remote sensing, multimedia and agriculture applications. Frontiers in Plant Science, 2022, vol. 13. DOI: 10.3389/fpls.2022.1041514.

Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., & Joulin, A. Emerging Properties in Self-Supervised Vision Transformers. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021, article no. 00951. DOI: 10.1109/iccv48922.2021.00951.

Pinetsuksai, N., Kittichai, V., Jomtarak, R., Jaksukam, K., Tongloy, T., Boonsang, S., & Chuwongin, S. Development of Self-Supervised Learning with Dinov2-Distilled Models for Parasite Classification in Screening. 2023 15th International Conference on Information Technology and Electrical Engineering (ICITEE), 2023, pp. 323-328. DOI: 10.1109/icitee59582.2023.10317719.

Kulkarni, U., S M M., Hallyal, R., Sulibhavi, P., G S. V., Guggari, S., & Shanbhag, A. R. Optimisation of deep neural network model using Reptile meta learning approach. Cognitive Computation and Systems, Institution of Engineering and Technology (IET), 2023. DOI: 10.1049/ccs2.12096.

Kotyan, S., & Vargas, D. V. Adversarial robustness assessment: Why in evaluation both L0 and L∞ attacks are necessary. PLOS ONE, 2022, vol. 17, iss. 4, article no. e0265723. DOI: 10.1371/journal.pone.0265723.

Li, G., Pattabiraman, K., & DeBardeleben, N. TensorFI: A Configurable Fault Injector for TensorFlow Applications. 2018 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW), 2018. DOI: 10.1109/issrew.2018.00024.

Zhao, Y., Sun, H., & Wang, S. A Method for Detecting Lightweight Optical Remote Sensing Images Using Improved Yolov5n. Journal of Multimedia Information System, 2023, vol. 10, iss. 3, pp. 215-226. DOI: 10.33851/jmis.2023.10.3.215.

Zou, C., Jeon, W.-S., Rhee, S.-Y., & Cai, M. YOLO-RS: A More Accurate and Faster Object Detection Method for Remote Sensing Images. In Remote Sensing. Remote Sensing, 2023, vol. 15, iss. 15, article no. 3863. DOI: 10.3390/rs15153863.




DOI: https://doi.org/10.32620/aktt.2024.5.11