Intelligent technologies for target detection and engagement by UAVs based on artificial intelligence and simulation data

Igor Sinitsyn, Anatoliy Doroshenko, Ivan Kyrylov, Iaroslav Omelianenko, Valentyn Smirnov, Olena Yatsenko

Abstract


The subject matter of the article is the methodology for developing integrated intelligent systems that enable automated detection, classification, tracking, and engagement of targets by unmanned aerial vehicles (UAVs) using artificial intelligence, computer vision, and simulation modeling technologies. The goal is to design an architecture and experimental environment for evaluating the efficiency of deep learning models, particularly the YOLO family, in real-time operation across different hardware platforms, considering resource and power constraints, as well as to quantitatively assess improvements in key system characteristics. The tasks to be solved are: developing a microservice-based system architecture using Kubernetes and ROS2; creating specialized datasets for model training; integrating the AirSim, ArduPilot SITL, and Mission Planner simulators into a unified testing environment; and conducting comparative performance analysis of various YOLO versions on platforms ranging from low-power ARM/NPU to high-end GPU. The applied methods are simulation modeling, automated neural network training, model quantization, inference optimization using TensorRT, and statistical analysis of the obtained metrics. The following results were obtained. A unified experimental methodology and a hardware–software platform were developed, providing a complete research cycle—from data generation and annotation to model performance evaluation. Experimental tests demonstrated that YOLOv8n and YOLOv11n offer the best balance of accuracy and speed on low-resource platforms. The most effective onboard solutions were identified as the Google Coral Dev Board Micro and NVIDIA Jetson Orin Nano. A simulation testbed was deployed that successfully models UAV flight, target acquisition, and engagement processes. Conclusions. The results confirmed the feasibility of using simulation data and a microservice-based approach for developing autonomous intelligent UAV control systems, demonstrated measurable improvements in performance, computational efficiency, and energy consumption, as well as provided practical guidelines for selecting the optimal “model–platform” configuration. The scientific novelty of the obtained results lies in the creation of a comprehensive research environment that integrates modern artificial intelligence tools, simulation modeling, and hardware optimization into a unified reproducible structure, enabling objective evaluation of target detection and engagement algorithms in real time.

Keywords


artificial intelligence; automated target detection and engagement; computer vision; deep learning; neural networks; simulation modeling; unmanned aerial vehicles

Full Text:

PDF

References


Tang, G., Ni, J., Zhao, Y., Gu, Y., & Cao, W. A survey of object detection for UAVs based on deep learning. Remote Sensing, 2024, vol. 16, iss. 1, pp. 1-29. DOI: 10.3390/rs16010149.

Valaboju, R., Vaishnavi, Harshitha, C., Kallam, A. R., & Babu, B. S. Drone detection and classification using computer vision. Proceedings of the 7th International Conference on Trends in Electronics and Informatics (ICOEI). IEEE, Piscataway, 2023, pp. 1320-1328. DOI: 10.1109/ICOEI56765.2023.10125737.

Dimmig, C. A., Silano, G., McGuire, K., Gabellieri, C., Hönig, W., Moore, J., & Kobilarov, M. Survey of simulators for aerial robots: an overview and in-depth systematic comparisons. IEEE Robotics & Automation Magazine, 2025, vol. 32, iss. 2, pp. 153-166. DOI: 10.1109/MRA.2024.3433171.

Chan, J. H., Liu, K., Chen, Y., Sagar, A. S. M. S., & Kim Y.-G. Reinforcement learning-based drone simulators: survey, practice, and challenge. Artificial Intelligence Review, 2024, vol. 57, pp. 1-47. DOI: 10.1007/s10462-024-10933-w.

Gallagher, J. E., & Oughton, E. J. Surveying You Only Look Once (YOLO) multispectral object detection advancements, applications and challenges. IEEE Access, 2025, vol. 13, pp. 7366-7395. DOI: 10.1109/access.2025.3526458.

Bochkovskiy, A., Wang, C.-Y., & Liao, H. Y. M. YOLOv4: optimal speed and accuracy of object detection. Available at: https://doi.org/10.48550/arXiv.2004.10934 (аccessed 17 October 2025).

Shah, S., Dey, D., Lovett, C., & Kapoor, A. AirSim: high-fidelity visual and physical simulation for autonomous vehicles. Proceedings of the 11th International Conference on Field and Service Robotics (FSR). Springer Proceedings in Advanced Robotics, 2018, vol. 5, pp. 621-635. DOI: 10.1007/978-3-319-67361-5_40.

SITL Simulator (Software in the Loop). Available at: https://ardupilot.org/dev/docs/sitl-simulator-software-in-the-loop.html (аccessed 17 October 2025).

Mission Planner Overview. Available at: https://ardupilot.org/planner/docs/mission-planner-overview.html (аccessed 17 October 2025).

Zhao, J., Zhang, J., Li, D., & Wang, D. Vision-based anti-UAV detection and tracking. IEEE Transactions on Intelligent Transportation Systems, 2022, vol. 23, iss. 12, pp. 25323-25334. DOI: 10.1109/TITS.2022.3177627.

Bo, C., Wei, Y., Wang, X., Shi, Z., & Xiao, Y. Vision-based anti-UAV detection based on YOLOv7 GS in complex backgrounds. Drones, 2024, vol. 8, iss. 7, pp. 1-21. DOI: 10.3390/drones8070331.

Li, L., Huang, W., Gu, I. Y. H., & Tian, Q. Statistical modeling of complex backgrounds for foreground object detection. IEEE Transactions on Image Processing, 2004, vol. 13, iss. 11, pp. 1459-1472. DOI: 10.1109/TIP.2004.836169.

Lowe, D. G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, vol. 60, pp. 91-110. DOI: 10.1023/B:VISI.0000029664.99615.94.

Bay, H., Tuytelaars, T., & Van Gool, L. SURF: speeded up robust features. Proceedings of the 2006 European Conference on Computer Vision (ECCV). Lecture Notes in Computer Science, 2006, vol. 3951, pp. 404-417. DOI: 10.1007/11744023_32.

Dalal, N., & Triggs, B. Histograms of oriented gradients for human detection. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Piscataway, 2005, pp. 886-893. DOI: 10.1109/CVPR.2005.177.

Jocher, G., & Qiu, J. Ultralytics YOLO11. Available at: https://github.com/ultralytics/ultralytics (аccessed 17 October 2025).

Ni, J., Shen, K., Chen, Y., & Yang, S. X. An improved SSD-Like deep network-based object detection method for indoor scenes. IEEE Transactions on Instrumentation and Measurement, 2023, vol. 72, pp. 1-15. DOI: 10.1109/TIM.2023.3244819.

Girshick, R., Donahue, J., Darrell, T., & Malik, J. Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the 2014 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, Piscataway, 2014, pp. 580-587. DOI: 10.1109/CVPR.2014.81.

Ren, S., He, K., Girshick, R., & Sun, J. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, vol. 39, iss. 6, pp. 1137-1149. DOI: 10.1109/TPAMI.2016.2577031.

Ni, J., Shen, K., Chen, Y., Cao, W., & Yang, S. X. An improved deep network-based scene classification method for self-driving cars. IEEE Transactions on Instrumentation and Measurement, 2022, vol. 71, pp. 1-14. DOI: 10.1109/TIM.2022.3146923.

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., & Houlsby, N. An image is worth 16x16 words: transformers for image recognition at scale. Available at: https://doi.org/10.48550/arXiv.2010.11929 (аccessed 17 October 2025).

Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. End-to-end object detection with transformers. Proceedings of the 2020 European Conference on Computer Vision (ECCV). Lecture Notes in Computer Science, vol. 12346, pp. 213-229. DOI: 10.1007/978-3-030-58452-8_13.

Rasti, B., Hong, D., Hang, R., Ghamisi, P., Kang, X., Chanussot, J., & Benediktsson, J. A. Feature extraction for hyperspectral imagery: the evolution from shallow to deep: overview and toolbox. IEEE Geoscience and Remote Sensing Magazine, 2020, vol. 8, iss. 4, pp. 60-88. DOI: 10.1109/MGRS.2020.2979764.

Hong, D., Gao, L., Yao, J., Zhang, B., Plaza, A., & Chanussot, J. Graph convolutional networks for hyperspectral image classification. IEEE Geoscience and Remote Sensing Magazine, 2021, vol. 59, iss. 7, pp. 5966-5978. DOI: 10.1109/TGRS.2020.3015157.

Shi, J., & Tomasi, C. Good features to track. Proceedings of the 1994 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Piscataway, 1994, pp. 593-600. DOI: 10.1109/CVPR.1994.323794.

Bewley, A., Ge, Z., Ott, L., Ramos, F., & Upcroft, B. Simple online and realtime tracking. Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP). IEEE, Piscataway, 2016, pp. 3464-3468. DOI: 10.1109/ICIP.2016.7533003.

Wojke, N., Bewley, A., & Paulus, D. Simple online and realtime tracking with a deep association metric. Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP). IEEE, Piscataway, 2017, pp. 3645-3649. DOI: 10.1109/ICIP.2017.8296962.

Lazarevich, I. , Grimaldi, M., Kumar, R., Mitra, S., Khan, S., & Sah, S. YOLOBench: benchmarking efficient object detectors on embedded systems. Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). IEEE, Piscataway, 2023, pp. 1161-1170. DOI: 10.1109/ICCVW60793.2023.00126.

Cantero, D., Esnaola Gonzalez, I., Miguel Alonso, J., & Jauregi, E. Benchmarking object detection deep learning models in embedded devices. Sensors, 2022, vol. 22, iss. 11. DOI: 10.3390/s22114205.

Stanley, K.O., Clune, J., Lehman, J., & Miikkulainen, R. Designing neural networks through neuro-evolution. Nature Machine Intelligence, 2019, vol. 1, pp. 24-35. DOI: 10.1038/s42256-018-0006-z.

Omelianenko, I., & Sinitsyn, I. Artificial swarm intelligence. International Scientific Technical Journal “Problems of Control and Informatics”, 2024, vol. 69, iss. 3, pp. 91-103. DOI: 10.34229/1028-0979-2024-3-7. (In Ukrainian).

Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., & Ng, A. Y. ROS: an open-source robot operating system. Proceedings of the 2009 ICRA workshop on open source software. IEEE, Piscataway, 2009. P. 1-6. Available at: http://www.robotics.stanford.edu/~ang/papers/icraoss09-ROS.pdf (аccessed 17 October 2025).

Tan, M., Pang, R., & Le, Q. V. EfficientDet: scalable and efficient object detection. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 10781-10790. DOI: 10.1109/CVPR42600.2020.01079.

Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C. Y., & Berg, A. C. SSD: Single Shot MultiBox Detector. Proceedings of the 2016 European Conference on Computer Vision (ECCV). Lecture Notes in Computer Science, 2016, vol. 9905, pp. 21-37. DOI: 10.1007/978-3-319-46448-0_2.

Minikube. Available at: https://minikube.sigs.k8s.io/docs (аccessed 17 October 2025).

Introduction to CVAT.ai: best image annotation tool explained in simple terms. Available at: https://www.cvat.ai/resources/blog/introduction-to-cvat (аccessed 17 October 2025).

Ultralytics YOLO Docs. Performance Metrics Deep Dive. Available at: https://docs.ultralytics.com/guides/yolo-performance-metrics (аccessed 17 October 2025).

ArduPilot Dev Team. SITL with Webots. Available at: https://ardupilot.org/dev/docs/sitl-with-webots.html (аccessed 17 October 2025).

autopilot_bee_sim. Autopilot with Target Following for FPV Combat Drone (Simulator version). Available at: https://github.com/under0tech/autopilot_bee_sim.git (аccessed 17 October 2025).

DRMS platform for the development of applications and specialized software. Available at: https://it-solutions.ua/it-infrastruktura/fizichna/platforma-drms (аccessed 17 October 2025) (In Ukrainian).

Sinitsyn, I. P., Omelianenko, Ia. V., Tverdokhlib, Ye. M., Stepaniuk, M. Yu., Zhytkevych, O. B., & Verveyko, V.M. Computer program “DRMS Platform”. Certificate of registration of copyright for the work no. 122378 dated 12/26/2023. Available at: https://iss.nas.gov.ua/storage/editor/files/private/Sinitsin/03-svidotstvo_DRMS.pdf (аccessed 17 October 2025) (In Ukrainian).




DOI: https://doi.org/10.32620/reks.2026.1.06

Refbacks

  • There are currently no refbacks.