Road Pavement Damage Detection using Computer Vision Techniques: Approaches, Challenges and Opportunities

Authors

DOI:

https://doi.org/10.22456/2175-2745.129787

Keywords:

Smart Cities, Road Pavement Damage Detection, Computer Vision, Convolutional Neural Networks, Object Detection, Survey

Abstract

The work presented in this paper is the result of a preliminary research aimed at using computer vision techniques for road pavement damage detection in the context of a smart city. It first introduces the related concepts. Then, it surveys the state of the art and existing solutions, presenting their main features, strengths and limitations. The most promising solutions are identified. Finally, it discusses open challenges and research directions in this area.

Downloads

Download data is not yet available.

References

YIN, C. et al. A literature survey on smart cities. Science China Information Sciences, Pequim, China, v. 58, n. 10, p. 1–18, ago. 2015.

CONFEDERAÇÃO Nacional do Transporte. CNT, 2023. Disponível em: <https://www.cnt.org.br/>. Acedido em 26 Julho 2023.

BORGES, C. Problemas estruturais em rodovias dobram chances de acidentes graves, aponta cnt. Agência iNFRA, 2018. Disponível em: <https://www.agenciainfra.com/blog/problemas-estruturais-em-rodovias-dobram-chances-de-acidentes-graves-aponta-cnt/>. Acedido em 26 Julho 2023.

ESTATÍSTICAS, gráficos e indicadores. PORDATA, 2023. Disponível em: <https://www.pordata.pt/>. Acedido em 26 Julho 2023.

DADOS sobre acidentes de viação em Portugal. Pordata, 2023. Disponível em: <https://www.pordata.pt/portugal/acidentes+de+viacao+em+portugal+continental+acidentes+com+vitimas++feridos+e+mortos-326>. Acedido em 26 Julho 2023.

CONSERVAÇÃO. Infraestruturas de Portugal, 2021. Disponível em: <https://www.infraestruturasdeportugal.pt/pt-pt/conservacao>. Acedido em 26 Julho 2023.

COMPUTER Vision Techniques. javaTpoint, 2021. Disponível em: <https://www.javatpoint.com/computer-vision-techniques>. Acedido em 26 Julho 2023.

OZGON, D. Top 10 computer vision techniques with deep learning. Becoming Human: Artificial Intelligence Magazine, 2021. Disponível em: <https://becominghuman.ai/top-10-computer-vision-techniques-with-deep-learning-124fcbd3c20>. Acedido em 26 Julho 2023.

XU, S. et al. Computer vision techniques in construction: A critical review. Archives of Computational Methods in Engineering, Barcelona, Espanha, v. 28, n. 5, p. 3383–3397, out. 2020.

GOODFELLOW, I.; BENGIO, Y.; COURVILLE, A. Deep Learning. Cambridge: MIT Press, 2016.

SMEDA, K. Understand the architecture of CNN. 2019. Disponível em: <https://towardsdatascience.com/understand-the-architecture-of-cnn-90a25e244c7>. Acedido em 26 Julho 2023.

AWATI, R. What are convolutional neural networks? | definition from techtarget. TechTarget. Disponível em: <https://www.techtarget.com/searchenterpriseai/definition/convolutional-neural-network>. Acedido em 26 Julho 2023.

LE, J. Convolutional neural networks: The biologically-inspired model. Codementor Community, 2018. Disponível em: <https://www.codementor.io/@james_aka_yale/convolutional-neural-networks-the-biologically-inspired-model-iq6s48zms>. Acedido em 26 Julho 2023.

O que é overfitting? 2023. Disponível em: <https://aws.amazon.com/pt/what-is/overfitting/>. Acedido em 26 Julho 2023.

WHAT is a convolutional neural network (CNN). Arm®, 2023. Disponível em: <https://www.arm.com/glossary/convolutional-neural-network>. Acedido em 26 Julho 2023.

ISLAM, M. M.; SADI, M. S. Path hole detection to assist the visually impaired people in navigation; path hole detection to assist the visually impaired people in navigation. In: 2018 4th International Conference on Electrical Engineering and Information Communication Technology (iCEEiCT). Nova Iorque, EUA: IEEE, 2018. p. 268–273.

ZHAO, Z.-Q. et al. Object detection with deep learning: A review. Arxiv preprint arXiv: 1807.05511, [S. l.], p. 1–21, jul. 2018.

CONVOLUTIONAL Neural Network. javatpoint, 2021. Disponível em: <https://www.javatpoint.com/pytorch-convolutional-neural-network>. Acedido em 26 Julho 2023.

WHAT Is Object Detection? MATLAB Simulink, 2023. Disponível em: <https://www.mathworks.com/discovery/object-detection.html?s_tid=srchtitle_object%20detection_1>. Acedido em 26 Julho 2023.

14.4. Anchor Boxes. Dive into Deep Learning, 2023. Disponível em: <https://d2l.ai/chapter_computer-vision/anchor.html>. Acedido em 26 Julho 2023.

CHRISTIANSEN, A. Anchor boxes - the key to quality object detection | towards data science. Towards Data Science, 2018. Disponível em: <https://towardsdatascience.com/anchor-boxes-the-key-to-quality-object-detection-ddf9d612d4f9>. Acedido em 26 Julho 2023.

14.3. Object Detection and Bounding Boxes. Dive into Deep Learning, 2023. Disponível em: <https://d2l.ai/chapter_computer-vision/bounding-box.html>. Acedido em 26 Julho 2023.

AN Overview of One-Stage Object Detection Models. Papers With Code. Disponível em: <https://paperswithcode.com/methods/category/one-stage-object-detection-models>. Acedido em 26 Julho 2023.

LOHIA, A. et al. Bibliometric analysis of one-stage and two-stage object bibliometric analysis of one-stage and two-stageobject detection detection. Library Philosophy and Practice (e-journal), EUA, v. 4910, p. 1–33, fev. 2021.

REDMON, J.; FARHADI, A. Yolov3: An incremental improvement. Arxiv preprint arXiv: 1804.02767, [S. l.], p. 1–6, abr. 2018.

ZVORNICANIN, E. What is yolo algorithm? Baeldung on Computer Science, 2023. Disponível em: <https://www.baeldung.com/cs/yolo-algorithm>. Acedido em 26 Julho 2023.

PEDOEEM, J.; HUANG, R.; CHEN, C. Yolo-lite: A real-time object detection algorithm optimized for non-gpu computers. In: 2018 IEEE International Conference on Big Data (Big Data). Los Alamitos, EUA: IEEE Computer Society, 2018. p. 2503–2510.

YOLO: Real-Time Object Detection. Joseph Chet Redmon. Disponível em: <https://pjreddie.com/darknet/yolo/>. Acedido em 26 Julho 2023.

LIN, T.-Y. et al. Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, Nova Iorque, EUA, v. 42, n. 2, p. 318 – 327, fev. 2020.

LIN, T.-Y. et al. Feature pyramid networks for object detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Nova Iorque, EUA: IEEE, 2017. p. 936–944.

TIAN, H.; ZHENG, Y.; JIN, Z. Improved retinanet model for the application of small target detection in the aerial images. IOP Conference Series: Earth and Environmental Science, Reino Unido, v. 585, p. 1–8, out. 2020.

RETINANET Explained. Papers With Code. Disponível em: <https://paperswithcode.com/method/retinanet>. Acedido em 26 Julho 2023.

LIU, W. et al. Ssd: Single shot multibox detector. In: Computer Vision – ECCV 2016. Cham, Suíça: Springer Berlin Heidelberg, 2016. p. 21–37.

PING, P.; YANG, X.; GAO, Z. A deep learning approach for street pothole detection. In: 2020 IEEE 6th International Conference on Big Data Computing Service and Applications, BigDataService 2020. Nova Iorque, EUA: IEEE, 2020. p. 198–204.

KHANDELWAL, R. Ssd : Single shot detector for object detection using multibox. Towards Data Science, 2019. Disponível em: <https://towardsdatascience.com/ssd-single-shot-detector-for-object-detection-using-multibox-1818603644ca>. Acedido em 26 Julho 2023.

REN, S. et al. Faster r-cnn: Towards real-time object detection with region proposal networks. Arxiv preprint arXiv: 1506.01497, [S. l.], p. 1–14, jun. 2015.

GIRSHICK, R. et al. Rich feature hierarchies for accurate object detection and semantic segmentation. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos, EUA: IEEE, 2014. p. 580–587.

GIRSHICK, R. Fast r-cnn. Arxiv preprint arXiv: 1504.08083, [S. l.], p. 1–9, abr. 2015.

WENG, W.; ZHU, X. Inet: Convolutional networks for biomedical image segmentation. IEEE Access, Nova Iorque, EUA, v. 9, p. 16591 – 16603, jan. 2021.

RONNEBERGER, O.; FISCHER, P.; BROX, T. U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI). Cham, Suíça: Springer, 2015. p. 234 – 241.

DALAL, N.; TRIGGS, B. Histograms of oriented gradients for human detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05). Nova Iorque, EUA: IEEE, 2005. v. 1, p. 886–893.

TRIBALDOS, P. et al. People detection in color and infrared video using hog and linear svm. In: Natural and Artificial Computation in Engineering and Medical Applications. Berlim: Springer Berlin Heidelberg, 2013. p. 179 – 189.

MITTAL, K. A gentle introduction into the histogram of oriented gradients. Medium, 2020. Disponível em: <https://medium.com/analytics-vidhya/a-gentle-introduction-into-the-histogram-of-oriented-gradients-fdee9ed8f2aa>. Acedido em 26 Julho 2023.

CORTES, C.; VAPNIK, V. Support-vector networks. Machine Learning, EUA, v. 20, p. 273–297, set. 1995.

SUPPORT Vector Machine Algorithm. javaTpoint. Disponível em: <https://www.javatpoint.com/machine-learning-support-vector-machine-algorithm>. Acedido em: 26 Julho 2023.

BAI, K. et al. Hog-svm-based image feature classification method for sound recognition of power equipments. Energies, Basileia, Suíça, v. 15, n. 12, p. 1–12, jun. 2022.

SAXENA, S. Beginner’s guide to support vector machine(svm). Analytics Vidhya, 2021. Disponível em: <https://www.analyticsvidhya.com/blog/2021/03/beginners-guide-to-support-vector-machine-svm/>. Acedido em 26 Julho 2023.

WHAT is Image Thresholding? MathWorks. Disponível em: <https://www.mathworks.com/discovery/image-thresholding.html>. Acedido em 26 Julho 2023.

POINT Operations - Adaptive Thresholding. Image Processing Learning Resources, 2003. Disponível em: <https://homepages.inf.ed.ac.uk/rbf/HIPR2/adpthrsh.htm>. Acedido em 26 Julho 2023.

WELLNER, P. D. Adaptive thresholding for the digitaldesk. EuroPARC Technical Report EPC1993-110, Cambridge, p. 1–17, jul. 1993.

IMAGE Thresholding. OpenCV. Disponível em: <https://docs.opencv.org/4.x/d7/d4d/tutorial_py_thresholding.html>. Acedido em 26 Julho 2023.

HOWARD, A. G. et al. Mobilenets: Efficient convolutional neural networks for mobile vision applications. Arxiv preprint arXiv: 1704.04861, [S. l.], p. 1–9, abr. 2017.

BROWNLEE, J. A gentle introduction to batch normalization for deep neural networks. Machine Learning Mastery, 2019. Disponível em: <https://machinelearningmastery.com/batch-normalization-for-training-of-deep-neural-networks/>. Acedido em 26 Julho 2023.

MEHTA, V. Object detection using ssd mobilenet v2. Medium - Vidish Mehta, 2021. Disponível em: <https://vidishmehta204.medium.com/object-detection-using-ssd-mobilenet-v2-7ff3543d738d>. Acedido em 26 Julho 2023.

BROWNLEE, J. A gentle introduction to object recognition with deep learning. Machine Learning Mastery, 2019. Disponível em: <https://machinelearningmastery.com/object-recognition-with-deep-learning/>. Acedido em 26 Julho 2023.

WANG, W. et al. Road damage detection and classification with faster r-cnn. In: 2018 IEEE International Conference on Big Data, Big Data 2018. Nova Iorque, EUA: IEEE, 2018. p. 5220–5223.

HIROYAMAEDA. Rdd2022: A multi-national image dataset for automatic road damage detection. Github, 2022. Disponível em: <https://github.com/sekilab/RoadDamageDetector>. Acedido em 28 Julho 2023.

AWAN, A. A. A complete guide to data augmentation. DataCamp, 2022. Disponível em: <https://www.datacamp.com/tutorial/complete-guide-data-augmentation>. Acedido em 26 Julho 2023.

MAHENGE, S. F.; WAMBURA, S.; JIAO, L. A modified u-net architecture for road surfaces cracks detection. In: 8th International Conference on Computing and Artificial Intelligence. Nova Iorque, EUA: Association for Computing Machinery, 2022. p. 464–471.

UNIVERSITESI, O. D. T. Concrete crack images for classification metu. Mendeley Data, 2019. Disponível em: <https://data.mendeley.com/datasets/5y9wdsg2zt/2>. Acedido em 28 Julho 2023.

TENSORFLOW Release Notes. NVIDIA Docs Hub. Disponível em: <https://docs.nvidia.com/deeplearning/frameworks/tensorflow-release-notes/rel_22-01.html>. Acedido em 26 Julho 2023.

SCHLEMPER, J. et al. Attention gated networks: Learning to leverage salient regions in medical images. Medical Image Analysis, Inglaterra, v. 53, p. 197–207, abr. 2019.

AGRAWAL, R. et al. Road surface classification and subsequent pothole detection using deep learning. In: 2021 2nd International Conference for Emerging Technology, INCET 2021. Nova Iorque, EUA: IEEE, 2021. p. 1–6.

PATEL, S. Pothole image data-set. Kaggle, 2019. Disponível em: <https://www.kaggle.com/datasets/sachinpatel21/pothole-image-dataset>. Acedido em 28 Julho 2023.

ROAD Traversing Knowledge (RTK) Dataset. LAPIX - Image Processing and Computer Graphics Lab, 2019. Disponível em: <https://lapix.ufsc.br/pesquisas/projeto-veiculo-autonomo/datasets/?lang=en>. Acedido em 28 Julho 2023.

ROAD/LANE Detection Dataset. Kitti Vision Benchmark Suite, 2023. Disponível em: <https://www.cvlibs.net/datasets/kitti/eval_road.php>. Acedido em 28 Julho 2023.

CARINA Dataset: Benchmark for Road Detection in Challenging Urban Scenarios. CaRINA Dataset, 2015. Disponível em: <http://www.lrm.icmc.usp.br/dataset/>. Acedido em 28 Julho 2023.

POTHOLE dataset compiled at Electrical and Electronic Department, Stelllenbosch University. Google Drive - Potholes dataset, 2015. Disponível em: <https://drive.google.com/drive/folders/1vUmCvdW3-2lMrhsMbXdMWeLcEz__Ocuy>. Acedido em 28 Julho 2023.

OMAR, M.; KUMAR, P. Detection of roads potholes using yolov4. In: 2020 International Conference on Information Science and Communications Technologies, ICISCT 2020. Nova Iorque, EUA: IEEE, 2020. p. 1–6.

KOECH, K. E. Object detection metrics with worked example. Towards Data Science, 2020. Disponível em: <https://towardsdatascience.com/on-object-detection-metrics-with-worked-example-216f173ed31e>. Acedido em 27 Julho 2023.

JAVED, A. et al. Pothole detection system using region-based convolutional neural network. In: 2021 IEEE 4th International Conference on Computer and Communication Engineering Technology, CCET 2021. Nova Iorque, EUA: IEEE, 2021. p. 6–11.

POTHOLE Detection Dataset. Pothole Detection Dataset, 2017. Disponível em: <https://people.etf.unsa.ba/~aakagic/pothole_detection/>. Acedido em 28 Julho 2023.

OCHOA-RUIZ, G. et al. An asphalt damage dataset and detection system based on retinanet for road conditions assessment. Applied Sciences, Basileia, Suíça, v. 10, n. 11, p. 1–16, jun. 2020.

SARI, Y.; PRAKOSO, P. B.; BASKARA, A. R. Road crack detection using support vector machine (svm) and otsu algorithm. In: 2019 6th International Conference on Electric Vehicular Technology (ICEVT). Nova Iorque, EUA: IEEE, 2019. p. 349–354.

FAN, R. et al. Road crack detection using deep convolutional neural network and adaptive thresholding. In: 2019 IEEE Intelligent Vehicles Symposium (IV). Nova Iorque, EUA: IEEE, 2019. p. 474–479.

RUIRANGERFAN (Rui (Ranger) Fan). GitHub. Disponível em: <https://github.com/ruirangerfan/>. Acedido em 26 Julho 2023.

FAN, R.; LIU, M. Road damage detection based on unsupervised disparity map segmentation. IEEE Transactions on Intelligent Transportation Systems, Nova Iorque, EUA, v. 21, n. 11, p. 4906–4911, nov. 2020.

KITTI Stereo Evaluation Dataset. KITTI Vision Benchmark Suite, 2015. Disponível em: <https://www.cvlibs.net/datasets/kitti/eval_scene_flow.php?benchmark=stereo>. Acedido em 28 Julho 2023.

APOLLO Scapestereo Dataset. ApolloScape, 2020. Disponível em: <https://apolloscape.auto/stereo.html>. Acedido em 28 Julho 2023.

EISATS Stereo Dataset. Concise Computer Vision - Wordpress, 2014. Disponível em: <https://ccv.wordpress.fos.auckland.ac.nz/eisats/>. Acedido em 28 Julho 2023.

NVIDIA Maxine. NVIDIA Developer, 2023. Disponível em: <https://developer.nvidia.com/maxine>. Acedido em: 26 Julho 2023.

VASQUEZ, A. How machine learning can drive change in traffic-packed L.A. Google - The Keyword, 2019. Disponível em: <https://blog.google/technology/ai/how-machine-learning-can-drive-change-traffic-packed-l/>. Acedido em 26 Julho 2023.

Downloads

Published

2023-10-05

How to Cite

Gonçalves, M., Marques, T., Gaspar, P. D., Soares, V. N. G. J., & Caldeira, J. M. L. P. (2023). Road Pavement Damage Detection using Computer Vision Techniques: Approaches, Challenges and Opportunities. Revista De Informática Teórica E Aplicada, 30(2), 22–35. https://doi.org/10.22456/2175-2745.129787

Issue

Section

Regular Papers