Assessing Deep Hybrid Representations for Colorectal Polyp Classification in Histopathology Images

Luan B. Guerra; Ana Beatriz S. Zerati; Ana Catarina M. Vicentim; Lucas C. Ribas

doi:10.22456/2175-2745.150896

Authors

Luan B. Guerra Universidade Estadual Paulista (Unesp) https://orcid.org/0009-0004-3278-9306
Ana Beatriz S. Zerati Universidade Estadual Paulista (Unesp) https://orcid.org/0009-0005-9051-6522
Ana Catarina M. Vicentim Universidade Estadual Paulista (Unesp) https://orcid.org/0000-0001-8017-0226
Lucas C. Ribas Universidade Estadual Paulista (Unesp) https://orcid.org/0000-0003-2490-180X

DOI:

https://doi.org/10.22456/2175-2745.150896

Keywords:

feature extraction, computer vision, histopathological images, convolutional neural networks, vision transformers

Abstract

This study investigates the use of Convolutional Neural Networks (CNNs) and Vision Transformers (ViT) backbones for feature extraction from histopathological images, and further evaluates the effectiveness of combining features from different depths of both architectures. The backbones were employed to extract features from colorectal polyp images in the MHIST dataset. Features obtained from each model individually, across different deep layers, were also combined and subjected to a dimensionality reduction process. Subsequently, both the individual and combined feature sets, with and without dimensionality reduction, were evaluated using linear classification models. The combined features achieved an accuracy of 85.55%, surpassing all results obtained from individual models and other approaches. These findings show that features from the first and middle layers, and feature fusion enhances robustness by leveraging the complementary strengths of each architecture.

Downloads

Download data is not yet available.

References

[1] FAMITHA, S.; MOORTHI, M. Intelligent and novel multi-type cancer prediction model using optimized ensemble learning. Computer Methods in Biomechanics and Biomedical Engineering, Taylor & Francis, 2022.

[2] LABIANCA, R. et al. Colorectal carcinoma: A general overview and future perspectives in colorectal cancer. International Journal of Molecular Sciences, v. 18, n. 1, p. 197, 2017.

[3] METTER, D. M. et al. Trends in the us and canadian pathologist workforces from 2007 to 2017. JAMA Network Open, v. 2, n. 5, p. e194337, 2019.

[4] WEI, J. et al. A petri dish for histopathology image analysis. In: SPRINGER. International Conference on Artificial Intelligence in Medicine. [S.l.], 2021. p. 11–24.

[5] FICHTINGER, G.; RUECKERT, D.; ZHOU, S. K. (Ed.). Handbook of Medical Image Computing and Computer Assisted Intervention. London, United Kingdom; Boston, Massachusetts, USA: Academic Press, Elsevier, 2022. ISBN 9781787859289.

[6] LI, J. et al. Transforming medical imaging with transformers? a comparative review of key properties, current progresses, and future perspectives. Medical Image Analysis, v. 85, p. 102762, 2023.

[7] HOSSEINI, M. S. et al. Computational pathology: A survey review and the way forward. Journal of Pathology Informatics, v. 15, p. 100357, Jan 2024.

[8] DOSOVITSKIY, A. et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. 2021. Disponível em: ⟨https://arxiv.org/abs/2010.11929⟩.

[9] SCABINI, L. et al. A comparative survey of vision transformers for feature extraction in texture analysis. Journal of Imaging, MDPI, v. 11, n. 9, p. 304, 2025.

[10] RIBAS, L. C.; CASACA, W.; FARES, R. T. Conditional generative adversarial networks and deep learning data augmentation: A multi-perspective data-driven survey across multiple application fields and classification architectures. AI, MDPI, v. 6, n. 2, p. 32, 2025.

[11] KANG, M. et al. Benchmarking self-supervised learning on diverse pathology datasets. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). [S.l.: s.n.], 2023. p. 3344–3354.

[12] WANG, X. et al. Transformer-based unsupervised contrastive learning for histopathological image classification. Medical Image Analysis, v. 79, p. 102430, 2022.

[13] NERGIZ, M. Classification of precancerous colorectal lesions via convnext on histopathological images. Balkan Journal of Electrical & Computer Engineering, v. 11, n. 2, p. 129–135, April 2023. Disponível em: ⟨https://dergipark.org.tr/en/pub/bajece/issue/77274/1240284⟩.

[14] FARRIS, A. et al. Sessile serrated adenoma: challenging discrimination from other serrated colonic polyps. The American Journal of Surgical Pathology, v. 32, n. 1, p. 30–35, 2008.

[15] KHALID, O. et al. Reinterpretation of histology of proximal colon polyps called hyperplastic in 2001. World Journal of Gastroenterology, v. 15, n. 30, p. 3767–3770, 2009.

[16] LIU, Z. et al. A ConvNet for the 2020s. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). [S.l.: s.n.], 2022. p. 11976–11986.

[17] HE, K. et al. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). [S.l.: s.n.], 2016. p. 770–778.

[18] RIDNIK, T. et al. ImageNet-21K Pretraining for the Masses. 2021. Disponível em: ⟨https://arxiv.org/abs/2104.10972⟩.

[19] VASWANI, A. et al. Attention Is All You Need. 2023. Disponível em: ⟨https://arxiv.org/abs/1706.03762⟩.

[20] JOLLIFFE, I. T. Principal Component Analysis. 2nd. ed. [S.l.]: Springer, 2002. (Springer Series in Statistics). ISBN 978-0387954424.

[21] CORTES, C.; VAPNIK, V. Support-vector networks. Machine Learning, v. 20, n. 3, p. 273–297, 1995.

[22] GUO, Y.; ZHAO, G.; PIETIKÄINEN, M. Texture classification using a linear configuration model based descriptor. In: CITESEER. BMVC. [S.l.], 2011. p. 1–10.

[23] SONG, T. et al. Letrist: Locally encoded transform feature histogram for rotation-invariant texture classification. IEEE Transactions on Circuits and Systems for Video Technology, v. 28, n. 7, p. 1565–1579, 2018.

[24] RIBAS, L. C. et al. Color-texture classification based on spatio-spectral complex network representations. Physica A: Statistical Mechanics and its Applications, p. 129518, 2024. ISSN 0378-4371.

[25] FARES, R. T.; RIBAS, L. C. Volumetric color-texture representation for colorectal polyp classification in histopathology images. In: Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - VISAPP (Vol. 3). [S.l.]: SciTePress, 2025. p. 210–221.

[26] ALALI, M. H. et al. Enabling intelligent IoTs for histopathology image analysis using convolutional neural networks. Micromachines, v. 13, n. 8, p. 1364, 2022. Disponível em: ⟨https://www.mdpi.com/2072-666X/13/8/1364⟩.

[27] SIMONYAN, K.; ZISSERMAN, A. Very deep convolutional networks for large-scale image recognition. 2014.

[28] HUANG, G. et al. Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. [S.l.: s.n.], 2017. p. 4700–4708.