Can we trust machine learning predictions of fire-induced concrete spalling?

Abstract

Spalling is the detachment of layers or fragments from the concrete surface during exposure to elevated temperatures. This study proposes and examines a method for assessing the pointwise reliability of spalling predictions produced by machine learning. The post-prediction procedure combines a density filter with the CL (local agreement) and AL (local fit) criteria, considering both neighborhood support for the predicted class and classifier performance on similar observations. A dataset containing 855 samples from 46 publications and described by 17 attributes was used. XGBoost was adopted as the classifier, and its performance was compared using a hold-out split, cross-validation, and external validation with distinct source publications assigned to training and evaluation, simulating real-world cases. External validation yielded lower performance than the other strategies. When applied jointly, CL and AL classified 84 of the 210 external predictions as reliable, corresponding to 40.0% coverage. Under the adopted thresholds, no errors occurred within the reliable subset. The proposed method complements global metrics by individually identifying predictions with greater empirical support and flagging those that do not meet the adopted reliability criteria.