RADIOELECTRONIC AND COMPUTER SYSTEMS

A machine learningsemi-supervised method was developed for the classification analysis of defects on the surface of the sewer pipe based on CCTV video inspection images. The aim of the research is the process of defect detection on the surface of sewage pipes. The subject of the research is a machine learning method for the classification analysis of sewage pipe defects on video inspection images under conditions of a limited and unbalanced set of labeled training data. A five-stage algorithm for classifier training is proposed. In the first stage, contrast training occurs using the instance-prototype contrast loss function, where the normalized Euclidean distance is used to measure the similarity of the encoded samples. The second step considers two variants of regularized loss functions – a triplet NCA function and a contrast-center loss function. The regularizing component in the second stage of training is used to penalize the rounding error of the output feature vector to a discrete form and ensures that the principle of information bottlenecking is implemented. The next step is to calculate the binary code of each class to implement error-correcting codes, but considering the structure of the classes and the relationships between their features. The resulting prototype vector of each class is used as a label of image for training using the cross-entropy loss function.  The last stage of training conducts an optimization of the parameters of the decision rules using the information criterion to consider the variance of the class distribution in Hamming binary space. A micro-averaged metric F1, which is calculated on test data, is used to compare learning outcomes at different stages and within different approaches. The results obtained on the Sewer-ML open dataset confirm the suitability of the training method for practical use, with an F1 metric value of 0.977. The proposed method provides a 9 % increase in the value of the micro-averaged F1 metric compared to the results obtained using the traditional method.

sewer pipes; inspection; classification analysis; convolutional neural network; self-learning; loss function; regularisation; information-extreme machine learning

Cheng, J. C. P., Wang, M. Automated detection of sewer pipe defects in closed-circuit television images using deep learning techniques. Automation in Construction, 2018, vol. 95, pp. 155-171. DOI: 10.1016/j.autcon.2018.08.006.

Moradi, S., Zayed, T., Golkhoo, F. Review on Computer Aided Sewer Pipeline Defect Detection and Condition Assessment. Infrastructures, 2019, vol 4, no. 1, article id: 10. DOI: 10.3390/infrastructures 4010010.

Haurum, J. B., Moeslund, T. B. A Survey on image-based automation of CCTV and SSET sewer inspections. Automation in Construction, 2020, vol. 111, article id: 103061. DOI: 10.1016/j.autcon.2019.103061.

Czimmermann, T., Ciuti, G., Milazzo, M., Chiurazzi, M.,Roccella, S., Oddo, C. M., Dario, P. Visual-Based Defect Detection and Classification Approaches for Industrial Applications – A SURVEY. Sensors, 2020, vol. 20, article Id: 1459. DOI: 10.3390/s20051459.

Li, Dawei., Xie, Qian., Yu, Zhenghao., Wu, Qiaoyun., Zhou, Jun., Wang, Jun. Sewer pipe defect detection via deep learning with local and global feature fusion. Automation in Construction, 2021, vol. 129, article id: 103823. DOI: 10.1016/j.autcon.2021.103823.

Li, Duanshun., Cong, A., Guo, S. Sewer damage detection from imbalanced CCTV inspection data using deep convolutional neural networks with hierarchical classification. Automation in Construction, 2019, vol. 101, pp. 199-208. DOI: 10.1016/j.autcon.2019.01.017.

Kumar, S. S., Mingzhu, W., Abraham, D. M., Jahanshahi, M. R., Tom, I., Cheng, J. C. Deep Learning–Based Automated Detection of Sewer Defects in CCTV Videos. Journal of Computing in Civil Engineering, 2020, vol. 34, iss. 1, Article Id: 4019047. DOI: 10.1061/(asce)cp.1943-5487.0000866.

Haurum, J. B., Moeslund, T. B. Sewer-ML: A Multi-Label Sewer Defect Classification Dataset and Benchmark. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 13456-13467.

Zhang, B., Tondi, B., Lv, X., Barni, M. Challenging the Adversarial Robustness of DNNs Based on Error-Correcting Output Codes. Security and Communication Networks, 2020, vol. 2020, article id: 8882494. DOI: 10.1155/2020/8882494.

Kolchinsky, A., Tracey, B. D., Wolpert, D. H. Nonlinear Information Bottleneck. Entropy, 2019, vol. 21, iss. 2, article id: 1181. DOI: 10.3390/e21121181.

Yang, L., Wang, Y., Miao, Z., Wang, J., Zhang, R. Contrastive Self-Supervised Hashing With Dual Pseudo Agreement. IEEE Access, 2020, vol. 8, pp. 165034 – 165043. DOI: 10.1109/ACCESS.2020.3022672.

Moskalenko, A., Moskalenko, V., Lysyuk, V., Zaretskyi, M. Sewer Pipe Defects Classification Based on Deep Convolutional Network with Information-extreme Error-correction Decision. Communications in Computer and Information Science, 2020, vol. 1158, pp. 253-263. DOI: 10.1007/978-3-030-61656-4_16.

Liu, X., Zhang, F., Hou, Z., Mian, L., Wang, Z., Zhang, J., Tang, J. Self-supervised Learning: Generative or Contrastive. IEEE Transaction on Knowledge and Data Engineering, 2021. DOI: 10.1109/TKDE.2021.3090866.

Dovbysh, A., Shelechov, I., Khibovska, Ju., Маtiash, O. Informatsiyno-analitychna systema otsinyuvannya vidpovidnosti suchasnym vymoham navchal'noho kontentu spetsial'nosti kiberbezpeka [Information and analytical system for assessing the compliance of educational content specialties ciber security with modern requirements]. Radioelectronic and computer systems, 2021, vol. 1, pp. 70-80. DOI: 10.32620/reks.2021.1.06