CoSegXAI: A Pipeline for Context vs. Segmented ROI Images in Lung Disease Classification through Saliency Map Insights

Graciella dos Santos Favoreto; Mariana Aya Suzuki Uchida; Pedro Augusto Luiz; Erikson Julio de Aguiar; Marcel Koenigkam Santos; Agma Juci Machado Traina

doi:10.22456/2175-2745.150955

Authors

Graciella dos Santos Favoreto Universidade de São Paulo (USP) https://orcid.org/0000-0002-8995-6265
Mariana Aya Suzuki Uchida Universidade de São Paulo (USP) https://orcid.org/0009-0001-9770-0817
Pedro Augusto Luiz Universidade de São Paulo (USP) https://orcid.org/0009-0001-7086-3106
Erikson Julio de Aguiar Universidade de São Paulo (USP) https://orcid.org/0000-0001-8563-8051
Marcel Koenigkam Santos Universidade de São Paulo (USP) https://orcid.org/0000-0002-7160-4691
Agma Juci Machado Traina Universidade de São Paulo (USP) https://orcid.org/0000-0003-4929-7258

DOI:

https://doi.org/10.22456/2175-2745.150955

Keywords:

saliency maps, chest x-ray, deep learning, explainable AI

Abstract

This research proposed CoSegXAI, an experimental pipeline to investigate the impact of lung segmentation on the explainability and performance of deep neural networks applied to multiclass chest X-ray classification. We employed the COVID-19 Radiography Database and trained ResNet50, DenseNet201, and VGG19 models on both contextual (original) and ROI-based (segmented) images across four diagnostic classes: COVID-19, lung opacity, viral pneumonia, and normal cases. To evaluate model decisions, we generate the saliency maps using Grad-CAM and assess them through visual analysis and quantitative metrics, including Insertion Correlation (IC), Deletion Correlation (DC), and Sparsity. Our findings show that models trained on contextual images achieved superior classification performance. DenseNet201 achieves an accuracy of 92.80% and 88.08% trained on contextual and ROI-based images, respectively. However, explainability analysis reveals that models trained on ROI-based (segmented important regions) images produce more focused attention patterns with better calibration, achieving superior IC, DC, and Sparsity scores. These results suggest that models are more reliable and provide clinicians with greater explainability. The source code is available at https://github.com/graciellafavoreto/CoSegXAI.git.

Downloads

Download data is not yet available.

References

[1] BRIN, D. et al. Assessing GPT-4 multimodal performance in radiological image analysis. European Radiology, Springer, v. 35, n. 4, p. 1959–1965, 2025.

[2] WEI, J. et al. Deep learning model for diagnosing and classifying subtypes of chronic pulmonary aspergillosis in chest CT. Mycoses, Wiley Online Library, v. 68, n. 4, p. e70061, 2025.

[3] FU, X. et al. Explainable hybrid transformer for multi-classification of lung disease using chest X-rays. Scientific Reports, Nature Publishing Group UK London, v. 15, n. 1, p. 6650, 2025.

[4] CHEN, C.; ISA, N. A. M.; LIU, X. A review of convolutional neural network based methods for medical image classification. Computers in Biology and Medicine, Elsevier, v. 185, p. 109507, 2025.

[5] PLESNER, L. L. et al. Commercially available chest radiograph AI tools for detecting airspace disease, pneumothorax, and pleural effusion. Radiology, Radiological Society of North America, v. 308, n. 3, p. e231236, 2023.

[6] PESSOA, D. et al. Ensemble deep learning model for dimensionless respiratory airflow estimation using respiratory sound. Biomedical Signal Processing and Control, Elsevier, v. 87, p. 105451, 2024.

[7] SELVARAJU, R. R. et al. Grad-CAM: Visual explanations from deep networks via gradient-based localization. In: 2017 IEEE International Conference on Computer Vision (ICCV). [S.l.: s.n.], 2017. p. 618–626.

[8] VENKATESH, K. et al. Gradient-based saliency maps are not trustworthy visual explanations of automated AI musculoskeletal diagnoses. Journal of Imaging Informatics in Medicine, Springer, v. 37, n. 5, p. 2490–2499, 2024.

[9] HERTEL, R.; BENLAMRI, R. A deep learning segmentation-classification pipeline for X-ray-based COVID-19 diagnosis. Biomedical Engineering Advances, Elsevier, v. 3, p. 100041, 2022.

[10] RAJARAMAN, S. et al. Iteratively pruned deep learning ensembles for COVID-19 detection in chest X-rays. IEEE Access, IEEE, v. 8, p. 115041–115050, 2020.

[11] RAHMAN, T. et al. Exploring the effect of image enhancement techniques on COVID-19 detection using chest X-ray images. Computers in Biology and Medicine, Elsevier, v. 132, p. 104319, 2021.

[12] CHOWDHURY, M. E. H. et al. Can AI help in screening viral and COVID-19 pneumonia? IEEE Access, v. 8, p. 132665–132676, 2020. ISSN 2169-3536.

[13] AZAD, R. et al. Medical image segmentation review: The success of U-Net. IEEE Transactions on Pattern Analysis and Machine Intelligence, v. 46, n. 12, p. 10076–10095, 2024. ISSN 1939-3539.

[14] WANG, R. et al. Medical image segmentation using deep learning: A survey. IET Image Processing, v. 16, n. 5, p. 1243–1267, 2022. ISSN 1751-9667.

[15] KUIRY, S. et al. GA-RISE: Posthoc model agnostic explanations of black-box classifiers using genetic algorithm-based optimized masks—a case study on chest X-ray images. International Journal on Artificial Intelligence Tools, World Scientific, v. 34, n. 01, p. 2550006, 2025.

[16] RODRIGUES, A. V. d. M.; OLIVEIRA, R. B. Segmentation of skin lesions and their attributes in dermatoscopic images based on convolutional neural networks. v. 32, p. 99–106, 2025. ISSN 2175-2745.

[17] MANOEL, L. A. V.; PONTI, M. A. Leveraging optimal methods for diverse skin types classification in images for reduced bias. Revista de Informática Teórica e Aplicada, v. 32, n. 1, p. 250–256, 2025. ISSN 2175-2745.

[18] RODRIGUES, L. et al. Evaluating convolutional neural networks for COVID-19 classification in chest X-ray images. In: XVI Workshop de Visão Computacional (WVC). SBC, 2020. p. 52–57. ISSN 0000-0000. Disponível em: https://sol.sbc.org.br/index.php/wvc/article/view/13480.

[19] ARUN, N. et al. Assessing the trustworthiness of saliency maps for localizing abnormalities in medical imaging. Radiology: Artificial Intelligence, v. 3, n. 6, p. e200267, 2021. Disponível em: https://doi.org/10.1148/ryai.2021200267.

[20] SAPORTA, A. et al. Benchmarking saliency methods for chest X-ray interpretation. Nature Machine Intelligence, v. 4, n. 10, p. 867–878, 2022. ISSN 2522-5839. Disponível em: https://doi.org/10.1038/s42256-022-00536-x.

[21] ZENG, Y. et al. Inconsistency between human observation and deep learning models: Assessing validity of postmortem computed tomography diagnosis of drowning. Journal of Imaging Informatics in Medicine, v. 37, n. 3, p. 1–10, 2024. ISSN 2948-2933.

[22] KUIRY, S. et al. GA-RISE: Posthoc model agnostic explanations of black-box classifiers using genetic algorithm-based optimized masks — a case study on chest X-ray images. International Journal on Artificial Intelligence Tools, World Scientific Publishing Company, 2025. Disponível em: https://www.worldscientific.com/worldscinet/ijait.

[23] HE, K. et al. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. [S.l.: s.n.], 2016. p. 770–778.

[24] HUANG, G. et al. Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. [S.l.: s.n.], 2017. p. 4700–4708.

[25] SIMONYAN, K.; ZISSERMAN, A. Very Deep Convolutional Networks for Large-Scale Image Recognition. 2015. Disponível em: https://arxiv.org/abs/1409.1556.

[26] GOMEZ, T.; FRÉOUR, T.; MOUCHÈRE, H. Metrics for Saliency Map Evaluation of Deep Learning Explanation Methods. Emerging Topics in Pattern Recognition and Artificial Intelligence, p. 101–121, 2022.

[27] PASZKE, A. et al. PyTorch: An Imperative Style, High-Performance Deep Learning Library. 2019. Disponível em: https://arxiv.org/abs/1912.01703.