Method of UAV-based inspection of photovoltaic modules using thermal and RGB data fusion

Andrii Lysyi, Anatoliy Sachenko, Pavlo Radiuk, Mykola Lysyi, Oleksandr Melnychenko, Diana Zahorodnia

Abstract


The Subject matter of the article is the design and experimental evaluation of an intelligent edge-cloud cyber-physical system for automated inspection of photovoltaic (PV) modules in utility-scale solar power plants based on multi-palette thermal infrared and RGB imagery acquired by unmanned aerial vehicles (UAVs). The Goal of this study is to enhance PV defect detection accuracy and operational efficiency by designing a novel method that converts raw, overlapping sensor data into a compact, geo-referenced inventory of actionable defects. The goal is achieved by systematically increasing detection mean Average Precision (mAP) and reducing false positive rates caused by data redundancy, while simultaneously minimizing bandwidth usage. The Tasks to be solved include: developing a palette-invariant thermal representation that suppresses dependence on pseudo-color rendering and camera internal parameters; fusing this robust thermal stream with contrast-enhanced RGB data using an adaptive gated mechanism that can down-weight unreliable modalities; implementing an on-board active perception loop that re-orients the UAV gimbal for additional views of ambiguous, small-area anomalies; designing a geo-spatial clustering and de-duplication module that merges repeated detections into unique fault events suitable for integration with SCADA and GIS tools; and quantifying the benefits of the proposed architecture on public benchmarks and real-world field trials in comparison with single-modality baselines. The Methods employed include deep convolutional neural networks based on a YOLOv11m-seg instance segmentation backbone trained with a palette-consistency regularization term, gated feature-level fusion of thermal and RGB embeddings, Rodrigues-based calculation of corrective gimbal rotations for adaptive re-acquisition, density-based spatial clustering with the haversine distance metric for geographic de-duplication of detections, and statistical performance analysis using mAP, macro-averaged F1-score, recall, and a duplicate-induced false positive indicator. The following Results were obtained: on the PVF-10 benchmark the proposed system achieves mAP@0.5 = 0.903, exceeding thermal-only and RGB-only detectors by 12–16 percentage points; on the STHS-277 dataset it reaches mAP@0.5 = 0.887; palette-invariant training and adaptive re-acquisition together increase small-target recall to 0.86; geo-spatial clustering reduces the duplicate-induced false positive rate by 12–15 percentage points; field validation at rooftop and ground-mounted plants confirms 96% recall with a low root-mean-square deviation between automatic and manual defect counts; and relevance-only telemetry reduces airborne data transmission by 60–67% while preserving diagnostic fidelity. In Conclusion, the scientific novelty of the results obtained lies in a unified palette-invariant, multi-modal edge-cloud cyber-physical architecture that combines UAV sensing, active perception, geo-spatial reasoning, and bandwidth-aware reporting into a single operational method for photovoltaic module inspection, providing a scalable foundation for condition-based maintenance of large solar power plants.

Keywords


photovoltaic modules; UAV inspection; defect detection; thermal infrared imaging; RGB imagery; deep learning; multi-modal fusion; palette invariance; geo-spatial clustering

Full Text:

PDF

References


Tsanakas, J., Ha, L., & Buerhop, C. Faults and infrared thermographic diagnosis in operating c‐Si photovoltaic modules: a review of research and future challenges. Renewable and Sustainable Energy Reviews, 2016, vol. 62, pp. 695–709. DOI: 10.1016/j.rser.2016.04.079.

Grimaccia, F., Leva, S., Dolara, A., & Aghaei, M. Survey on PV modules common faults after an O&M flight extensive campaign over different plants in Italy. IEEE Journal of Photovoltaics, 2017, vol. 7, no. 3, pp. 810–816. DOI: 10.1109/JPHOTOV.2017.2674977.

Kandeal, A., Elkadeem, M., Thakur, A., Abdelaziz, G., Sathyamurthy, R., Kabeel, A., Yang, N., & Sharshir, S. Infrared thermography-based condition monitoring of solar photovoltaic systems: a mini review of recent advances. Solar Energy, 2021, vol. 223, pp. 33–43. DOI: 10.1016/j.solener.2021.05.032.

Gallardo-Saavedra, S., Hernández-Callejo, L., & Duque-Pérez, O. Technological review of the instrumentation used in aerial thermographic inspection of photovoltaic plants. Renewable and Sustainable Energy Reviews, 2018, vol. 93, pp. 566–579. DOI: 10.1016/j.rser.2018.05.027.

Zefri, Y., ElKettani, A., Sebari, I., & Ait Lamallam, S. Thermal infrared and visual inspection of photovoltaic installations by UAV photogrammetry—application case: Morocco. Drones, 2018, vol. 2, no. 4, p. 41. DOI: 10.3390/drones2040041.

Melnychenko, O., Scislo, L., Savenko, O., Sachenko, A., & Radiuk, P. Intelligent integrated system for fruit detection using multi-UAV imaging and deep learning. Sensors, 2024, vol. 24, no. 6, p. 1913. DOI: 10.3390/s24061913.

Michail, A., Livera, A., Tziolis, G., Georghiou, G., Panayiotou, C., Loutsiou, A., & Lilli, C. A comprehensive review of unmanned aerial vehicle-based approaches to support photovoltaic plant diagnosis. Heliyon, 2024, vol. 10, no. 1, p. e023983. DOI: 10.1016/j.heliyon.2024.e23983.

Lee, D., & Park, J. Developing inspection methodology of solar energy plants by thermal infrared sensor on board unmanned aerial vehicles. Energies, 2019, vol. 12, no. 15, article no. 2928. DOI: 10.3390/en12152928.

Aghaei, M., Dolara, A., Leva, S., & Grimaccia, F. Image resolution and defects detection in PV inspection by unmanned technologies. In: Proceedings of the 2016 IEEE Power and Energy Society General Meeting (PESGM). Boston, MA, USA, IEEE, 2016, pp. 1–5. DOI: 10.1109/PESGM.2016.7741605.

Buerhop-Lutz, C., Bommes, L., Schlipf, J., Pickel, T., Fladung, A., & Peters, I. Infrared imaging of photovoltaic modules: A review of the state of the art and future challenges facing gigawatt-scale PV power stations. Progress in Energy, 2022, vol. 4, no. 4, article no. 042010. DOI: 10.1088/2516-1083/ac890b.

Phoolwani, U., Sharma, T., Singh, A., & Gawre, S. IoT based solar panel analysis using thermal imaging. In: Proceedings of the 2020 IEEE International Students’ Conference on Electrical, Electronics and Computer Science (SCEECS). Bhopal, India, IEEE, 2020, pp. 1–6. DOI: 10.1109/SCEECS48394.2020.114.

Lysyi, A., Sachenko, A., Radiuk, P., Lysyi, M., Melnychenko, O., Ishchuk, O., & Savenko, O. Enhanced fire hazard detection in solar power plants: An integrated UAV, AI, and SCADA-based approach. Radioelectronic and Computer Systems, 2025, vol, 2025, no. 2, pp. 99–117. DOI: 10.32620/reks.2025.2.06.

Morando, L., Recchiuto, C., Calla, J., Scuteri, P., & Sgorbissa, A. Thermal and visual tracking of photovoltaic plants for autonomous UAV inspection. Drones, 2022, vol. 6, no. 11, article no. 347. DOI: 10.3390/drones6110347.

Svystun, S., Scislo, L., Pawlik, M., Melnychenko, O., Radiuk, P., Savenko, O., & Sachenko, A. DyTAM: Accelerating wind turbine inspections with dynamic UAV trajectory adaptation. Energies, 2025, vol. 18, no. 7, article no. 1823. DOI: 10.3390/en18071823.

Vlaminck, M., Heidbuchel, R., Philips, W., & Luong, H. Region-based CNN for anomaly detection in PV power plants using aerial imagery. Sensors, 2022, vol. 22, no. 3, article no. 1244. DOI: 10.3390/s22031244.

Di Tommaso, A., Genduso, F., Miceli, R., & Galluzzo, G. A multi-stage model based on YOLOv3 for defect detection in PV panels using IR and visible imaging by UAV. Renewable Energy, 2022, vol. 193, pp. 941–962. DOI: 10.1016/j.renene.2022.04.046.

Su, Y., Tao, F., Jin, J., & Zhang, C. Automated overheated region object detection of photovoltaic module with thermography image. IEEE Journal of Photovoltaics, 2021, vol. 11, no. 2, pp. 535–544. DOI: 10.1109/JPHOTOV.2020.3045680.

Duranay, Z. B. Fault detection in solar energy systems: a deep learning perspective. Electronics, 2023, vol. 12, no. 21, article no. 4397. DOI: 10.3390/electronics12214397.

Meng, S., Yue, Y., & Xu, T. Enhanced YOLOv11 framework for accurate multi-fault detection in UAV photovoltaic inspection. Sensors, 2025, vol. 25, no. 17, article no. 5311. DOI: 10.3390/s25175311.

Tan, M., Pang, R., & Le, Q. EfficientDet: scalable and efficient object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, WA, USA, IEEE, 2020, pp. 10778–10787. DOI: 10.1109/CVPR42600.2020.01079.

Akram, M., Li, G., Jin, Y., Chen, X., Zhu, H., Ahmad, A., & Riaz, F. Automatic detection of photovoltaic module defects in infrared images with deep learning. Solar Energy, 2020, vol. 198, pp. 175–186. DOI: 10.1016/j.solener.2020.01.055.

Zefri, Y., Sebari, I., Hajji, H., & Aniba, G. Developing a deep learning-based layer-3 solution for thermal infrared large-scale photovoltaic module inspection from orthorectified big UAV imagery data. International Journal of Applied Earth Observation and Geoinformation, 2022, vol. 106, article no. 102652. DOI: 10.1016/j.jag.2021.102652.

Jia, Y., Chen, G., & Zhao, L. Defect detection of photovoltaic modules based on improved VarifocalNet. Scientific Reports, 2024, vol. 14, article no. 15170. DOI: 10.1038/s41598-024-66234-3.

Wang, B., Chen, Q., Wang, M., Chen, Y., Zhang, Z., Liu, X., Gao, W., Zhang, Y., & Zhang, H. PVF-10: a high-resolution unmanned aerial vehicle thermal infrared image dataset for fine-grained photovoltaic fault classification. Applied Energy, 2024, vol. 376, article no. 124187. DOI: 10.1016/j.apenergy.2024.124187.

Alfaro-Mejía, E., Loaiza-Correa, H., Franco-Mejía, E., Restrepo-Girón, A., & Nope-Rodríguez, S. Dataset for recognition of snail trails and hot spot failures in monocrystalline Si solar panels. Data in Brief, 2019, vol. 26, article no. 104441. DOI: 10.1016/j.dib.2019.104441.

Qureshi, U., Rashid, A., Altini, N., Bevilacqua, V., & La Scala, M. Explainable intelligent inspection of solar photovoltaic systems with deep transfer learning: considering warmer weather effects using aerial radiometric infrared thermography. Electronics, 2025, vol. 14, no. 4, article no. 755. DOI: 10.3390/electronics14040755.

Oulefki, A., Agaian, S., El Afou, Y., Djahel, S., Zenkouar, K., & Taleb-Ahmed, A. Detection and analysis of deteriorated areas in solar PV modules using unsupervised sensing algorithms and 3D augmented reality. Heliyon, 2024, vol. 10, no. 6, article no. e27973. DOI: 10.1016/j.heliyon.2024.e27973.

Liao, K-C., & Lu, J-H. Using UAV to detect solar module fault conditions of a solar power farm with IR and visual image analysis. Applied Sciences, 2021, vol. 11, no. 4, article no. 1835. DOI: 10.3390/app11041835.

Svystun, S., Melnychenko, O., Radiuk, P., Savenko, O., Sachenko, A., & Lysyi, A. Thermal and RGB images work better together in wind turbine damage detection. International Journal of Computing, 2024, vol. 23, no. 4, pp. 526–535. DOI: 10.47839/ijc.23.4.3752.

Rohith, G., Rajalakshmi, R., Manish, D., & Narasimhan, R. Fusion-Solar-Net for solar panel fault detection. Results in Engineering, 2025, vol. 27, article no. 106513. DOI: 10.1016/j.rineng.2025.106513.

Lai, Y.-S., Hsieh, C.-C., Liao, T.-W., Huang, C.-Y., Yeh, C.-H., & Chen, W.-H. Deep learning-based automatic defect detection of photovoltaic modules in infrared, electroluminescence, and red–green–blue images. Energy Conversion and Management, 2025, vol. 332, article no. 119783. DOI: 10.1016/j.enconman.2025.119783.

Niccolai, A., Grimaccia, F., & Leva, S. Advanced asset management tools in photovoltaic plant monitoring: UAV-based digital mapping. Energies, 2019, vol. 12, no. 24, article no. 4736. DOI: 10.3390/en12244736.

Bommes, L., Pickel, T., Buerhop-Lutz, C., Hauch, J., Brabec, C., & Peters, I. Computer vision tool for detection, mapping, and fault classification of photovoltaics modules in aerial IR videos. Progress in Photovoltaics: Research and Applications, 2021, vol. 29, no. 12, pp. 1236–1251. DOI: 10.1002/pip.3448.

Kolahi, M., Esmailifar, S. M., Moradi Sizkouhi, A. M., & Aghaei, M. Digital-PV: a digital twin-based platform for autonomous aerial monitoring of large-scale photovoltaic power plants. Energy Conversion and Management, 2024, vol. 321, article no. 118963. DOI: 10.1016/j.enconman.2024.118963.

Kishor, I., Mamodiya, U., Patil, V., Naik, N., Kumar, S., & Yadav, R. AI-integrated autonomous robotics for solar panel cleaning and predictive maintenance using drone and ground-based systems. Scientific Reports, 2025, vol. 15, article no. 32187. DOI: 10.1038/s41598-025-17313-6.

DJI. Matrice 300 RTK [Computer program], 2020. Available at: https://www.dji.com/global/support/product/matrice-300 (accessed 3 September 2025).

DJI. Zenmuse H20T [Computer program], 2020. Available at: https://enterprise.dji.com/zenmuse-h20-series (accessed 1 September 2025).

NVIDIA Corporation. Jetson AGX Orin 32GB Module [Computer program], 2022. Available at: https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-orin/ (accessed 1 September 2025).

Canonical Ltd. Ubuntu [Computer program]. Version 20.04.6 LTS (Focal Fossa), 2022. Available at: https://releases.ubuntu.com/focal/ (accessed 4 September 2025).

Open Robotics. ROS Noetic Ninjemys [Computer program], 2020. Available at: https://www.openrobotics.org/ (accessed 3 September 2025).

DJI. DJI Onboard SDK [Computer program]. Version 4.1.0, 2021. Available at: https://github.com/dji-sdk/Onboard-SDK (accessed 30 September 2025).

DJI. DJI Payload SDK [Computer program]. Version 3.12.0, 2025. Available at: https://developer.dji.com/payload-sdk/ (accessed 3 September 2025).

Python Software Foundation. Python 3 Programming Language [Computer program]. Version 3.12.0, 2023. Available at: https://www.python.org/ (accessed 22 September 2025).

OpenCV Development Team. OpenCV [Computer program]. Version 4.9.0, 2023. Available at: https://opencv.org/ (accessed 30 September 2025).

NumPy Developers. NumPy [Computer program]. Version 2.3.0, 2025. Available at: https://numpy.org/ (accessed 2 September 2025).

PyTorch Foundation. PyTorch [Computer program]. Version 2.0, 2023. Available at: https://pytorch.org/ (accessed 1 September 2025).

Ultralytics. YOLOv11m-seg Architecture [Computer program], 2024. Available at: https://docs.ultralytics.com/models/yolo11/ (accessed 31 September 2025).

ZeroMQ Project. ZeroMQ [Computer program]. Version 4.3.5, 2023. Available at: https://zeromq.org/ (accessed 1 September 2025).

Microsoft Corporation. Microsoft Azure Cloud Platform [Computer program], 2010. Available at: https://azure.microsoft.com/ (accessed 1 September 2025).

ZigBee Alliance. ZigBee Specification [Standard]. Document No. 053474r20, 2005. Available at: https://zigbeealliance.org/ (accessed 3 September 2025).

Rainio, O., Teuho, J., & Klén, R. Evaluation metrics and statistical tests for machine learning. Scientific Reports, 2024, vol. 14, no. 1, article no. 6086. DOI: 10.1038/s41598-024-56706-x.




DOI: https://doi.org/10.32620/reks.2025.4.13

Refbacks

  • There are currently no refbacks.