Impact of distortions in UAV images on quality and accuracy of object localization

Rostyslav Tsekhmystro, Oleksii Rubel, Oleksandr Prysiazhniuk, Vladimir Lukin

Abstract


The localization and classification of objects of different types in images is an important and actively researched topic because the designed methods and tools are exploited in a wide variety of fields, including remote sensing, security systems, and medical diagnostics. Imaging systems installed on-board unmanned aerial vehicles (UAVs) and drones have become popular recently, and they are potentially beneficial for numerous applications like mine detection, traffic control, and crowd control. Images acquired by such systems may suffer from low quality because of the use of rather cheap cameras and the necessity to transfer obtained data via communication lines with limited bandwidth, employing lossy compression. These factors can influence the quality and accuracy of object localization, which is typically negatively performed by trained neural networks. However, the intensity of the noise and distortions that can be considered acceptable, i.e. such that they do not lead to radical reduction of the performance characteristics are unclear. Given this, it is reasonable to investigate the impact of these effects on the quality of object localization and classification using a reliable data size and various noise/distortion intensities. Therefore, the research subject of this paper is the performance of object localization and classification methods for color images acquired by UAV-installed sensors. The primary focus is on the dependence of localization and classification metrics on the noise intensity, where the simulated noise mimics not only noise but also distortions due to lossy compression by modern coders. The aim of this work is to obtain adequate statistics and analyze them to build dependencies of the metrics on the intensity of distortions. The objective is to obtain conditions for which the effects of noise and distortions can be considered negligible or acceptable in practice. The second objective is to analyze the sensitivity of several modern neural network models to noise/distortions.  The result is a statistical assessment of the dependence of model performance on input data quality. The conclusions are based on the statistics characterizing the model performance for the noise/distortion intensity interval. The conclusions allow the selection of the best (most robust) neural networks and the establishment of appropriate performance conditions.

Keywords


object localization; classification; noise and distortions, UAV

Full Text:

PDF

References


Dai, Y., Liu, D., Hu, G. & Yu, X. Radar Target Detection Algorithm Using Convolutional Neural Network to Process Graphically Expressed Range Time Series Signals. Sensors, 2022, no. 18, article no. 6868. DOI: 10.3390/s22186868.

Wen, X., Wang, J., Cheng, C., Zhang, F. & Pan, G. Underwater Side-Scan Sonar Target Detection: YOLOv7 Model Combined with Attention Mechanism and Scaling Factor. Remote Sensing, 2024, no. 16, article no. 2492. DOI: 10.3390/rs16132492.

Zhao, Q., Wu, Y. & Yuan, Y. Ship Target Detection in Optical Remote Sensing Images Based on E2YOLOX-VFL. Remote Sensing, 2024, no. 16, article no. 340. DOI: 10.3390/rs16020340.

Liu, S., Chen, P. & Woźniak, M. Image Enhancement-Based Detection with Small Infrared Targets. Remote Sensing, 2022, no. 13, article no. 3232. DOI: 10.3390/rs14133232.

Vivone, G., Addesso, P. & Ziemann, A. Editorial for Special Issue “Remote Sensing for Target Object Detection and Identification”. Remote Sensing, 2020, no. 1, article no. 196. DOI: 10.3390/rs12010196.

Zou, J., Zheng, H. & Wang, F. Real-Time Target Detection System for Intelligent Vehicles Based on Multi-Source Data Fusion. Sensors, 2023, no. 23, article no. 1823. DOI: 10.3390/s23041823.

Yang, R. & Yu, Y. Artificial Convolutional Neural Network in Object Detection and Semantic Segmentation for Medical Imaging Analysis. Frontiers in Oncology, 2021, vol. 11. DOI: 10.3389/fonc.2021.638182

Mittermayer, J., Younis, M., Metzig, R., Wollstadt, S., Marquez Martinez J. & Meta A. TerraSAR-X System Performance Characterization and Verification. IEEE Transactions on Geoscience and Remote Sensing, 2010, vol. 48, no. 2, pp. 660-676. DOI: 10.1109/TGRS.2009.2026742.

Džunda, M., Dzurovčin, P. & Melníková, L. Anti-Collision System for Small Civil Aircraft. Applied Sciences, 2022, no. 12, article no. 1648. DOI: 10.3390/app12031648.

Hayat, S., Yanmaz E. & Muzaffar R. Survey on Unmanned Aerial Vehicle Networks for Civil Applications: A Communications Viewpoint. IEEE Communications Surveys & Tutorials, 2016, vol. 18, no. 4, pp. 2624-2661. DOI: 10.1109/COMST.2016.2560343.

Dewan, R. & Rahman, K.F. A Survey on Applications of Unmanned Aerial Vehicles (UAVs). Recent Innovations in Computing, 2022, vol. 855. DOI: 10.1007/978-981-16-8892-8_8.

Vasilyeva, I., Lukin, V., Kharchenko, V. & Nereta, A. Combined processing of satellite and UAV data to increase the classification reliability. IntelITSIS’2023: 4th International Workshop on Intelligent Information Technologies and Systems of Information Security, 2023, pp 539-552.

Fedorenko, G., Fesenko, H., Kharchenko, V., Kliushnikov, I. & Tolkunov I. Robotic-biological systems for detection and identification of explosive ordnance: concept, general structure, and models. Radioelectronic and Computer Systems, 2023, vol. 106, no. 2, pp. 143-159. DOI: 10.32620/reks.2023.2.12.

Ramirez-Jaime, A., Pena-Pena, K., Arce, G. R., Harding, D., Stephen M. & MacKinnon J. HyperHeight LiDAR Compressive Sampling and Machine Learning Reconstruction of Forested Landscapes. IEEE Transactions on Geoscience and Remote Sensing, 2024, vol. 62, article no. 4402416, pp. 1-16. DOI: 10.1109/TGRS.2024.3356389.

Ieremeiev, O., Lukin V. & Vozel B. Combined No-Reference Image Quality Metric for UAV Applications. IEEE 2nd Ukrainian Microwave Week (UkrMW), 2022, pp. 638-643. DOI: 10.1109/UkrMW58013.2022.10037120.

Kim, J. H. & Sung, S. M. Quality Analysis of Unmanned Aerial Vehicle Images Using a Resolution Target. Applied Sciences, 2024, vol. 14, no. 5. DOI: 10.3390/app14052154.

Ahmad, A., Amira, K., Mawardy, M., Firdaus, S., Yazid, M., & Rahman, A. B. Noise and Restoration of UAV Remote Sensing Images. International Journal of Advanced Computer Science and Applications, 2020, vol. 11. DOI: 10.14569/IJACSA.2020.0111222.

Papić, V., Šolić, P., Milan, A., Gotovac, S. & Polić, M. High-Resolution Image Transmission from UAV to Ground Station for Search and Rescue Missions Planning. Applied Sciences, 2021, vol. 11, no. 5, article no. 2105. DOI: 10.3390/app11052105.

Wang, Y. Y. & Huang, D. Q. Compression for UAV reconnaissance images. Optics and Precision Engineering, 2014, vol. 22, no. 5, pp. 1363-1370. DOI: 10.3788/OPE.20142205.1363.

Abramova, V., Lukin, V., Abramov, S., Abramov, K. & Bataeva, E. Analysis of Statistical and Spatial Spectral Characteristics of Distortions in Lossy Image Compression. IEEE 2nd Ukrainian Microwave Week (UkrMW), 2022, pp. 644-649. DOI: 10.1109/UkrMW58013.2022.10036949.

Chatterjee, P. & Milanfar, P. Is Denoising Dead?. IEEE Transactions on Image Processing, 2010, vol. 19, no. 4, pp. 895-911. DOI: 10.1109/TIP.2009.2037087.

Chatterjee, P. & Milanfar, P. Patch-based near-optimal image denoising. IEEE Trans Image Process, 2012, vol. 21, no. 4, DOI: 10.1109/TIP.2011.217279.

Lukin, V., Abramov, S., Krivenko, S., Kurekin, A. & Pogrebnyak, O. Analysis of classification accuracy for pre-filtered multichannel remote sensing data. Journal of Expert Systems with Applications, 2013, vol. 40, pp. 6400-6411. DOI: 10.1016/j.eswa.2013.05.061.

Zabala, A. & Pons, X. Effects of lossy compression on remote sensing image classification of forest areas. International Journal of Applied Earth Observation and Geoinformation, 2011, vol. 13, no. 1, pp. 43-51. DOI: 10.1016/j.jag.2010.06.005.

Ozah, N. & Kolokolova, A. Compression Improves Image Classification Accuracy. Advances in Artificial Intelligence, 2019, vol. 11489. DOI: 10.1007/978-3-030-18305-9_55.

Tsekhmystro, R., Rubel, O. & Lukin, V. Study of methods for searching and localizing objects in images from aircraft using convolutional neural networks. Radioelectronic and Computer Systems, 2024, no. 1, pp. 87-98. DOI: 10.32620/reks.2024.1.08.

Adli, T., Bujaković, D., Bondžulić, B., Laidouni, M. Z. & Andrić, M. A Modified YOLOv5 Architecture for Aircraft Detection in Remote Sensing Images. Journal of the Indian Society of Remote Sensing, 2024. DOI: 10.1007/s12524-024-02033-7.

Qi, W. Object detection in high resolution optical image based on deep learning technique. Natural Hazards Research, 2022, vol. 2, no. 4, pp. 384-392. DOI: 10.1016/j.nhres.2022.10.002.

Shorten, C. & Khoshgoftaar, T. M. A survey on Image Data Augmentation for Deep Learning. Journal of Big Data, 2019, vol. 6, no. 60. DOI: 10.1186/s40537-019-0197-0.

Zoph, B., Cubuk, E. D., Ghiasi, G., Lin, T., Shlens, J. & Le, Q. V. Learning Data Augmentation Strategies for Object Detection. Computer Vision – ECCV 2020, 2020, vol. 12372. DOI: 10.1007/978-3-030-58583-9_34.

Zhu, P., Wen, L., Du, D., Bian, X., Fan, H., Hu, Q., & Ling, H. Detection and Tracking Meet Drones Challenge. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, vol. 44, no. 11, pp. 7380-7399. DOI: 10.1109/TPAMI.2021.3119563.

Rezatofighi, H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I., & Savarese, S. Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 658-666. DOI: 10.48550/arXiv.1902.09630.

Sokolova, M., Japkowicz, N. & Szpakowicz, S. Beyond Accuracy, F-Score and ROC: A Family of Discriminant Measures for Performance Evaluation. Advances in Artificial Intelligence, 2006, vol. 4304. DOI: 10.1007/11941439_114.

YOLOv5 by Ultralytics. Available at: https://zenodo.org/records/7347926. (accessed 25 Sep. 2024). DOI: 10.5281/zenodo.7347926.

Ren, S., He, K., Girshick, R., & Sun, J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, vol. 39, pp. 1137-1149. DOI: 10.1109/TPAMI.2016.2577031.

Lin, T. Y., Goyal, P., Girshick, R., He, K. & Dollár, P. Focal Loss for Dense Object Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, vol. 42, no. 2, pp. 318-327. DOI: 10.1109/TPAMI.2018.2858826.

Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C., & Berg, A. SSD: Single Shot MultiBox Detector. Lecture Notes in Computer Science, 2016, vol. 9905, pp. 21-37. DOI: 10.1007/978-3-319-46448-0_2

Tsekhmystro, R., Rubel, O. & Lukin, V. Investigation of the effect of object size on accuracy of human localization in images acquired from unmanned aerial vehicles. Aerospace Technic and Technology, 2024, no. 2, pp. 83-90. DOI: 10.32620/aktt.2024.2.09.

Tsekhmystro, R., Rubel, O. & Lukin, V. Study of the dependence of accuracy in vehicles search on the size of the object using UAV images. Aerospace Technic and Technology, 2024, no. 3, pp. 89-98. DOI: 10.32620/aktt.2024.3.08.

Cong, X., Li, S., Chen, F., Liu, C. & Meng, Y. A Review of YOLO Object Detection Algorithms based on Deep Learning. Frontiers in Computing and Intelligent Systems, 2023, vol. 4, no. 2, pp. 17-20. DOI: 10.54097/fcis.v4i2.9730.

Malakan, Z. M., Anwar, S., Hassan, G. M. & Mian, A. Sequential Storytelling Image Dataset (SSID). IEEE Dataport, 2023, DOI: 10.21227/dbr9-dq51.

Ting, K. M. Precision and Recall. Encyclopedia of Machine Learning, 2010. 781 p. DOI: 10.1007/978-0-387-30164-8_652.

Uss, M., Vozel, B., Chehdi, K. & Lukin, V. Maximum likelihood estimation of spatially correlated signal-dependent noise in hyperspectral images. Optical Engineering, 2012, vol. 51, no. 11, article no. 111712. DOI: 10.1117/1.OE.51.11.111712.

Colom, M. & Buades, A. Analysis and Extension of the PCA Method, Estimating a Noise Curve from a Single Image. Image Processing On Line, 2016, no. 6, pp. 365-390. DOI: 10.5201/ipol.2016.124.

Makarichev, V., Lukin, V. & Brysina, I. On the Impact of Discrete Atomic Compression on Image Classification by Convolutional Neural Networks. Computation, 2024, vol. 12, no. 9, article no. 176. DOI: 10.3390/computation12090176.




DOI: https://doi.org/10.32620/reks.2024.4.05

Refbacks

  • There are currently no refbacks.