Performance comparison of CNNs on high-resolution multispectral dataset applied to land cover classification problem

Vladyslav Yaloveha, Andrii Podorozhniak, Heorhii Kuchuk, Nataliia Garashchuk

Abstract


Multispectral images acquired by satellites have been used in many fields such as agriculture, urban change detection, finding fire-hazardous forest areas, and real-time surface monitoring. The central issue in remote sensing analysis is land use and land cover classification. Land use and land cover classification (LULC) is the process of classification into meaningful classes based on the spectral characteristics of remote sensing data. Land use and land cover classification is a challenging task due to the complex nature of the Earth's surface. The accuracy of solving the issue using deep learning approaches depends on the quality of the remote sensing data, the choice of the classification algorithm. The ability to obtain high-resolution multispectral images periodically could dramatically improve remote sensing solutions. In this study, we propose a solution for the land cover and land classification problem of high-resolution remote sensing data by applying deep learning methods using EuroPlanet geo-referenced high-quality images with four bands and pixel resolution of 204x204 per image, and acquired by Planet platform in 2020-2022 years. The dataset consists of 25911 images with spatial resolution up to 3.125 meters per pixel and 10 different classes. In the past decade, artificial neural networks have shown great performance in solving complex image classification tasks. For the dataset evaluation, we have taken advantage of state-of-art pretrained convolutional neural network models ResNet50v2, EfficientNetV2, Xception, VGG-16, and DenseNet201 with fine tuning. It has been established that DenseNet201 pretrained neural network outperformed other models. The accuracy of the test data was 92.01 % and the F1 metric was 91.63 %. In addition, bands evaluation for the dataset was carried out. Overall classification accuracy of 93.83 % and F1 score of 93.56 % were achieved by DenseNet201 model. The results could be used for area verification, real-time monitoring, and surface change detection. Nowadays, this is very helpful for Ukrainian territory because of the Russian invasion and the country's recovery in the future.

Keywords


EuroPlanet; pretrained convolutional neural network; multispectral images; spectral indexes; land cover; remote sensing

Full Text:

PDF

References


Hussain, S., Lu, L. Mubeen, M., Nasim, W., Karuppannan, S., Fahad, S., Tariq, A., Mousa, B. G., Mumtaz, F. & Aslam, M. Spatiotemporal variation in land use land cover in the response to local climate change using multispectral remote sensing data. Land, 2022, vol. 11, no. 5, article no. 595. DOI: 10.3390/land11050595.

Mandal, A., Majumder, A., Dhaliwal, S. S., Toor, A. S., Mani, P. K., Naresh, R. K., Gupta, R. K. & Mitran, T. Impact of agricultural management practices on soil carbon sequestration and its monitoring through simulation models and remote sensing techniques: A review. Critical Reviews in Environmental Science and Technology, 2022, vol. 52, no. 1, pp. 1-49. DOI: 10.1080/10643389.2020.1811590.

Yaloveha, V., Hlavcheva, D., Podorozhniak, A. & Kuchuk, H. Fire hazard research of forest areas based on the use of convolutional and capsule neural networks. 2019 IEEE 2nd Ukraine Conference on Electrical and Computer Engineering (UKRCON), 2019, pp. 828-832. DOI: 10.1109/UKRCON.2019.8879867.

Podorozhniak, A., Liubchenko, N., Kvochka, M. & Suarez, I. Usage of intelligent methods for multispectral data processing in the field of environmental monitoring. Advanced Information Systems, 2021, vol. 5, no. 3, pp 97-102. DOI: 10.20998/2522-9052.2021.3.13.

Radočaj, D., Jurišić, M. & Gašparović, M. The Role of Remote Sensing Data and Methods in a Modern Approach to Fertilization in Precision Agriculture. Remote Sensing, 2022, vol. 14, no. 3, article no. 778. DOI: 10.3390/rs14030778.

Munawar, H. S., Hammad, A. W. A. & Waller, S. T. Remote Sensing Methods for Flood Prediction: A Review. Sensors, 2022, vol. 22, no. 3, article no. 960. DOI: 10.3390/s22030960.

Atik, S. O. & Ipbuker, C. Integrating convolutional neural network and multiresolution segmentation for land cover and land use mapping using satellite imagery. Applied Sciences, 2021, vol. 11, no. 12, article no. 5551. DOI: 10.3390/app11125551.

Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A. C. & Fei-Fei, L. ImageNet large scale visual recognition challenge. International journal of computer vision, 2015, vol. 115, pp. 211-252. DOI: 10.1007/s11263-015-0816-y.

Helber, P., Bischke, B., Dengel, A. & Borth, D. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019, vol. 12, no. 7, pp. 2217-2226. DOI: 10.1109/JSTARS.2019.2918242.

Barabash, O., Bandurka, O., Svynchuk, O. & Tverdenko, H. Method of identification of tree species composition of forests on the basis of geographic information database. Advanced Information Systems, 2022, vol. 6, no. 4, pp 5-10. DOI: 10.20998/2522-9052.2022.4.01.

Zhang, L., Dong. R., Yuan, S., Li, W., Zheng, J. & Fu, H. Making low-resolution satellite images reborn: a deep learning approach for super-resolution building extraction. Remote Sensing, 2021, vol. 13, no. 15, article no. 2872. DOI: 10.3390/rs13152872.

Alganci, U., Soydas, M. & Sertel, E. Comparative research on deep learning approaches for airplane detection from very high-resolution satellite images. Remote Sensing, 2020, vol. 12, no. 3, article no. 458. DOI: 10.3390/rs12030458.

Yang, Y. & Newsam, S. Bag-of-visual-words and spatial extensions for land-use classification. Proceedings of the 18th SIGSPATIAL international conference on advances in geographic information systems, 2010, pp. 270-279. DOI: 10.1145/1869790.1869829.

Zou, Q., Ni, L., Zhang, T. & Wang, Q. Deep learning based feature selection for remote sensing scene classification. IEEE Geoscience and Remote Sensing Letters, 2015, vol. 12, no. 11, pp. 2321-2325. DOI: 10.1109/LGRS.2015.2475299.

Xia, G.-S., Hi, J., Hu, F., Shi, B., Bai, X., Zhong, Y., Zhang, L. & Lu, X. AID: A benchmark data set for performance evaluation of aerial scene classification. IEEE Transactions on Geoscience and Remote Sensing, 2017, vol. 55, no. 7, pp. 3965-3981. DOI: 10.1109/TGRS.2017.2685945.

Li, H. Dou, X., Tao, C., Wuet, Z., Chen, J., Peng, J., Deng, M. & Zhao, L. RSI-CB: A large scale remote sensing image classification benchmark via crowdsource data. Sensors, 2020, vol. 20, no. 6, article no. 1594. DOI: 10.3390/s20061594.

Zhou, W., Newsam, S., Li, C. & Shao, Z. PatternNet: A benchmark dataset for performance evaluation of remote sensing image retrieval. ISPRS journal of photogrammetry and remote sensing, 2018, vol. 145, no. 6, pp. 197-209. DOI: 10.1016/j.isprsjprs.2018.01.004.

Cheng, G., Han, J. & Lu, X. Remote sensing image scene classification: Benchmark and state of the art. Proceedings of the IEEE, 2017, vol. 105, no. 10, pp. 1865-1883. DOI: 10.1109/JPROC.2017.2675998.

Günen, M. A. Performance comparison of deep learning and machine learning methods in determining wetland water areas using EuroSAT dataset. Environmental Science and Pollution Research, 2022, vol. 29, pp. 21092-21106. DOI: 10.1007/s11356-021-17177-z.

Yaloveha, V., Hlavcheva, D. & Podorozhniak, A. Spectral Indexes Evaluation for Satellite Images Classification using CNN. Journal of Information and Organizational Sciences, 2021, vol. 45, no. 2, pp. 435-449. DOI: 10.31341/jios.45.2.5.

Wulder, M., A., Roy, D. P., Radeloff, V. C., Loveland, T. R., Anderson, M. C., Johnson, D. M., Healey, S., Zhu, Z., Scambos, T. A., Pahlevan, N., Hansen, M., Gorelick, N., Crawford, C. J., Masek, J. G., Hermosilla, T., White, J. C., Belward, A. S., Schaaf, C., Woodcock, C. E., Huntington, J. L., Lymburner, L., Hostert, P., Gao, F., Lyapustin, A., Pekel, J.-F., Strobl, P. & Cook, B. D. Fifty years of Landsat science and impacts. Remote Sensing of Environment, 2022, vol. 280, article no. 113195. DOI: 10.1016/j.rse.2022.113195.

Maxar Resources: WorldView-3. Available at: https://resources.maxar.com/data-sheets/worldview-3 (Accessed 15 January 2023).

Planet Scope Ortho Tiles. Available at: https://developers.planet.com/docs/data/psorthotile (Accessed 15 January 2023).

Ghaffarian, S., Valente, J., Van Der Voort, M. & Tekinerdogan, B. Effect of attention mechanism in deep learning-based remote sensing image processing: A systematic literature review. Remote Sensing, 2021. vol. 13, is. 15, article no. 2965. DOI: 10.3390/rs13152965.

Moskalenko, V. & Moskalenko, A. Neural network based image classifier resilient to destructive perturbation influences – architecture and training method. Radioelectronic and computer systems, 2022, no. 3 (103), pp. 95-109. DOI: 10.32620/reks.2022.3.07.

Weiss, K., Khoshgoftaar, T. M. & Wang, D. D. A survey of transfer learning. Journal of Big data, 2016, vol. 3, article no. 9. DOI: 10.1186/s40537-016-0043-6.

Hlavcheva, D., Yaloveha, V., Podorozhniak, A. & Kuchuk, H. Tumor nuclei detection in histopathology images using R – CNN. CEUR Workshop Proceedings, 2020. vol. 2740, pp. 63-74. Available at: https://ceur-ws.org/Vol-2740/20200063.pdf (Accessed 15 January 2023).

Simonyan, K. & Zisserman, A. Very deep convolutional networks for large-scale image recognition. ArXiv (Cornell University), preprint arXiv:1409.1556, 2014. DOI: 10.48550/arXiv.1409.1556.

He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778. DOI: 10.1109/CVPR.2016.90.

He, K., Zhang, X., Ren, S. & Sun, J. Identity mappings in deep residual networks. European conference on computer vision, 2016, pp. 630-645. DOI: 10.1007/978-3-319-46493-0_38.

Chollet, F. Xception: Deep learning with depthwise separable convolutions. Proceedings of the IEEE conference on computer vision and patternrecognition, 2017, pp. 1800-1807. DOI: 10.1109/CVPR.2017.195.

Huang, G., Liu, Z., Van Der Maaten, L. & Weinberger, K. Q. Densely connected convolutional networks. Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2261-.2269. DOI: 10.1109/CVPR.2017.243.

Tan, M. & Le, Q. V. Efficientnetv2: Smaller models and faster training. ArXiv (Cornell University), Preprint arXiv:2104.00298, 2021. DOI: 10.48550/arXiv.2104.00298.

Carneiro, T., Da Nóbrega, R. V. M., Nepomuceno, T., Bian, G.-B., De Albuquerque, V. H. C. & Filho, P. P. R. Performance analysis of google colaboratory as a tool for accelerating deep learning applications. IEEE Access, 2018, vol. 6, pp. 61677-61685. DOI: 10.1109/ACCESS.2018.2874767.

Lacaux, J., Tourre, Y., Vignolles, C., Ndione, J. A. & Lafaye, M. Classification of ponds from high-spatial resolution remote sensing: Application to Rift Valley Fever epidemics in Senegal. Remote Sensing of Environment, 2007, vol. 106, is. 1, pp. 66-74. DOI: 10.1016/j.rse.2006.07.012.

Yaloveha, V., Podorozhniak, A. & Kuchuk, H. Convolutional neural network hyperparameter optimization applied to land cover classification. Radioelectronic and computer systems, 2022, no. 1 (101), pp. 115-128. DOI: 10.32620/reks.2022.1.09.

Hlavcheva, D., Yaloveha, V., Podorozhniak, A. & Kuchuk, H. Comparison of CNNs for Lung Biopsy Images Classification. 2021 IEEE 3rd Ukraine Conference on Electrical and Computer Engineering, UKRCON 2021 – Proceedings, 2021, pp. 1–5. DOI: 10.1109/UKRCON53503.2021.9575305.

de Carvalho, O. L. F., de Carvalho Júnior, O. A., O. de Albuquerque, A., Santana, N. C., Borges, D. L., Luiz, A. S., Gomes, R. A. T. & Guimarães, R. F. Multispectral panoptic segmentation: Exploring the beach setting with worldview-3 imagery. International Journal of Applied Earth Observation and Geoinformation, 2022, vol. 112, article no. 102910. DOI: 10.1016/j.jag.2022.102910.

Rouse, J. W., Haas, R. H., Deering, D. W., Schell, J. A. & Harlan, J. C. Monitoring the Vernal Advancement and Retrogradation (Green Wave Effect) of Natural Vegetation. Texas A. & M. University. Remote Sensing Centre, 1974, 120 p. Available at: https://ntrs.nasa.gov/api/citations/19730017588/downloads/19730017588.pdf (Accessed 15 January 2023).

Gitelson, A. A. & Merzlyak, M. N. Signature analysis of leaf reflectance spectra: algorithm development for remote sensing of chlorophyll. Journal of plant physiology, 1996, vol. 148, no. 3-4, pp. 494-500. DOI: 10.1016/S0176-1617(96)80284-7.




DOI: https://doi.org/10.32620/reks.2023.2.09

Refbacks

  • There are currently no refbacks.