Machine Learning and Explainable Artificial Intelligence in Carbon Nanotubes Predictions

Moacir Guedes Oliveira; Carolina Paula Almeida; Sandra Mara Guse Scós Venske

doi:10.22456/2175-2745.146905

Authors

Moacir Guedes Oliveira Universidade Estadual do Centro-Oeste (UNICENTRO) https://orcid.org/0000-0003-2035-4693
Carolina Paula Almeida Universidade Estadual do Centro-Oeste (UNICENTRO) https://orcid.org/0000-0003-4939-6432
Sandra Mara Guse Scós Venske Universidade Estadual do Centro-Oeste (UNICENTRO) https://orcid.org/0000-0001-8602-3008

DOI:

https://doi.org/10.22456/2175-2745.146905

Keywords:

Carbon Nanotubes, Multi-target Regression, Machine Learning, Automated Machine Learning, Explainable Artificial Intelligence

Abstract

Nanotechnology is an area that can be applied in all fields of science and nanomaterials, such as Carbon Nanotubes (CNT), have been the subject of studies in order to reduce costs, time for experiments and development. CNTs have diverse applications because they are highly rigid, have high strength, and have low density. Because of this, the evaluation of the properties of carbon nanotubes and identification of their coordinates are important fields of research for science. Many real problems predictions, including those on carbon nanotubes, are Multi-Target Regression (MTR) tasks, where multiple continuous target variables are predicted. Machine learning techniques are well suited to address these types of problem. However, the literature reports approaches composed of complex models, which are manually configured and do not present any level of explainability. The goal of this work is to use an Automated Machine Learning (AutoML) technique, for which 10 classic machine learning algorithms are available. The proposal, named OurAlg, uses ensemble learning and AutoML to choose an ensemble of algorithms and their hyperparameters to handle MTR problems. Three CNT data sets were used in the experiments, with 3 and 4 targets, to predict the coordinates and properties of the carbon nanotubes. We also tested our approach on a set of 18 real-world data sets, with different numbers of targets, to evaluate its performance. In addition, an explainable artificial intelligence method was applied to CNT problems. Statistical tests were used and the OurAlg algorithm was compared favorably with classical algorithms and also with the literature.

Downloads

Download data is not yet available.

References

[1] VIVANCO-BENAVIDES, L. E. et al. Machine learning and materials informatics approaches in the analysis of physical properties of carbon nanotubes: A review. Computational Materials Science, Elsevier, v. 201, p. 110939, 2022.

[2] RAMAKRISHNA, S. et al. Materials informatics. J. Intell. Manuf., v. 30, n. 6, p. 2307–2326, 2019. Disponível em: ⟨https://doi.org/10.1007/s10845-018-1392-0⟩.

[3] GERON, A. Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems. 2nd ed. Sebastopol, CA: O’Reilly Media, Inc., 2019. ISBN 1492032646.

[4] ARASHLOO, S. R.; KITTLER, J. Multi-target regression via non-linear output structure learning. Neurocomputing, Elsevier, v. 492, p. 572–580, 2022.

[5] HUSMEIER, D. Automatic relevance determination (ard). In: . Neural Networks for Conditional Probability Estimation: Forecasting Beyond Point Predictions. London: Springer London, 1999. p. 221–227. ISBN 978-1-4471-0847-4. Disponível em: ⟨https://doi.org/10.1007/978-1-4471-0847-4_15⟩.

[6] KETKAR, N. Stochastic gradient descent. In: . Deep Learning with Python: A Hands-on Introduction. Berkeley, CA: Apress, 2017. p. 113–132. Disponível em: ⟨https://doi.org/10.1007/978-1-4842-2766-4_8⟩.

[7] FÜRNKRANZ, J. Decision tree. In: . Encyclopedia of Machine Learning. Boston, MA: Springer US, 2010. p. 263–267. ISBN 978-0-387-30164-8. Disponível em: ⟨https://doi.org/10.1007/978-0-387-30164-8_204⟩.

[8] KRAMER, O. K-nearest neighbors. In: . Dimensionality Reduction with Unsupervised Nearest Neighbors. Berlin, Heidelberg: Springer Berlin Heidelberg, 2013. p. 13–23. ISBN 978-3-642-38652-7. Disponível em: ⟨https://doi.org/10.1007/978-3-642-38652-7_2⟩.

[9] STEINWART, I.; CHRISTMANN, A. Support Vector Machines. Hanover, Pennsylvania: Springer New York, 2008. (Information Science and Statistics). ISBN 9780387772424. Disponível em: ⟨https://books.google.com.br/books?id=HUnqnrpYt4IC⟩.

[10] RASMUSSEN, C. E.; WILLIAMS, C. K. I. Gaussian processes for machine learning. Cambridge: MIT Press, 2006. I-XVIII, 1-248 p. (Adaptive computation and machine learning). ISBN 026218253X.

[11] LIU, Y.; WANG, Y.; ZHANG, J. New machine learning algorithm: Random forest. In: LIU, B.; MA, M.; CHANG, J. (Ed.). Information Computing and Applications. Berlin, Heidelberg: Springer Berlin Heidelberg, 2012. p. 246–252. ISBN 978-3-642-34062-8.

[12] GEURTS, P.; ERNST, D.; WEHENKEL, L. Extremely Randomized Trees. Machine Learning, Springer Verlag, v. 36, p. 3–42, 2006. Disponível em: ⟨https://hal.science/hal-00341932⟩.

[13] FREUND, Y.; SCHAPIRE, R. E. Experiments with a new boosting algorithm. In: INTERNATIONAL CONFERENCE ON MACHINE LEARNING. International Conference on Machine Learning. 1996. p. 148–156. Disponível em: ⟨http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.51.6252⟩.

[14] FRIEDMAN, J. H. Greedy function approximation: A gradient boosting machine. The Annals of Statistics, Institute of Mathematical Statistics, v. 29, n. 5, p. 1189–1232, 2001. Disponível em: ⟨https://doi.org/10.1214/aos/1013203451⟩.

[15] BRESKVAR, M.; KOCEV, D.; DZEROSKI, S. Ensembles for multi-target regression with random output selections. Mach. Learn., v. 107, n. 11, p. 1673–1709, 2018.

[16] ZHOU, Z.-H. Ensemble Methods: Foundations and Algorithms. 1st ed. Boca Raton, Fla.: Chapman & Hall/CRC, 2012. ISBN 1439830037.

[17] HARRISON, M. Machine Learning – Guia de Referência Rápida: Trabalhando com dados estruturados em Python. São Paulo: Novatec Editora, 2019.

[18] RIJN, J. N. van et al. The online performance estimation framework: heterogeneous ensemble learning for data streams. Machine Learning, Springer, v. 107, n. 1, p. 149–176, 2018.

[19] HUTTER, F.; KOTTHOFF, L.; VANSCHOREN, J. Automated Machine Learning: Methods, Systems, Challenges. 1st ed. Hanover, Pennsylvania: Springer Publishing Company, Incorporated, 2019. ISBN 3030053172.

[20] DAS, S.; CAKMAK, U. Hands-On Automated Machine Learning: A beginner’s guide to building automated machine learning systems using AutoML and Python. Birmingham, UK: Packt Publishing, 2018. ISBN 9781788622288. Disponível em: ⟨https://books.google.com.br/books?id=Jd9YDwAAQBAJ⟩.

[21] FEURER, M. et al. Efficient and robust automated machine learning. In: ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS. Advances in Neural Information Processing Systems 28 (2015). [S.l.], 2015. p. 2962–2970.

[22] CARUANA, R. et al. Ensemble selection from libraries of models. In: PROCEEDINGS OF THE TWENTY-FIRST INTERNATIONAL CONFERENCE ON MACHINE LEARNING. Proceedings of the twenty-first international conference on Machine learning. [S.l.], 2004. p. 18.

[23] ALI, S. et al. Explainable artificial intelligence (xai): What we know and what is left to attain trustworthy artificial intelligence. Information Fusion, p. 101805, 2023. ISSN 1566-2535. Disponível em: ⟨https://www.sciencedirect.com/science/article/pii/S1566253523001148⟩.

[24] LUNDBERG, S. M.; LEE, S.-I. A unified approach to interpreting model predictions. Advances in neural information processing systems, v. 30, 2017.

[25] ZHANG, C. A.; CHO, S.; VASARHELYI, M. Explainable artificial intelligence (xai) in auditing. International Journal of Accounting Information Systems, Elsevier, v. 46, p. 100572, 2022.

[26] ACI, M.; AVCI, M. Artificial neural network approach for atomic coordinate prediction of carbon nanotubes. Applied Physics A: Materials Science and Processing, v. 122, n. 7, 6 2016. Disponível em: ⟨https://doi.org/10.1007/s00339-016-0153-1⟩.

[27] SAUL, C. J.; KAMIS, A. Z.; SAHIN, E. Carbon nanotube coordinate prediction with deep learning. In: 4TH INTERNATIONAL CONFERENCE ON COMPUTER AND COMMUNICATION SYSTEMS (ICCCS). 2019 IEEE 4th International Conference on Computer and Communication Systems (ICCCS). [S.l.], 2019. p. 1–6.

[28] ATES, M.; EKER, A. A.; EKER, B. Carbon nanotube-based nanocomposites and their applications. Journal of Adhesion Science and Technology, Taylor & Francis, v. 31, n. 18, p. 1977–1997, 2017. Disponível em: ⟨https://doi.org/10.1080/01694243.2017.1295625⟩.

[29] CANADIJA, M. Deep learning framework for carbon nanotubes: Mechanical properties and modeling strategies. Carbon, Elsevier, v. 184, p. 891–901, 2021.

[30] BRESKVAR, M.; DZEROSKI, S. Multi-target regression rules with random output selections. IEEE Access, v. 9, p. 10509–10522, 2021.

[31] LARGE, J.; LINES, J.; BAGNALL, A. A probabilistic classifier ensemble weighting scheme based on cross-validated accuracy estimates. Data Min. Knowl. Discov., Kluwer Academic Publishers, USA, v. 33, n. 6, p. 1674–1709, nov. 2019. ISSN 1384-5810. Disponível em: ⟨https://doi.org/10.1007/s10618-019-00638-y⟩.

[32] XAVIER-JÚNIOR, J. C. et al. An evolutionary algorithm for automated machine learning focusing on classifier ensembles: An improved algorithm and extended results. Theoretical Computer Science, v. 805, p. 1–18, 2020. ISSN 0304-3975. Disponível em: ⟨https://www.sciencedirect.com/science/article/pii/S0304397519307649⟩.

[33] ZHANG, X.; MAHADEVAN, S. Ensemble machine learning models for aviation incident risk prediction. Decision Support Systems, v. 116, p. 48–63, 2019. ISSN 0167-9236. Disponível em: ⟨https://www.sciencedirect.com/science/article/pii/S0167923618301660⟩.

[34] DUMANCAS, G.; ADRIANTO, I. A stacked regression ensemble approach for the quantitative determination of biomass feedstock compositions using near infrared spectroscopy. Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy, Elsevier, v. 276, p. 121231, 2022.

[35] CARNEIRO, T. C. et al. Ridge regression ensemble of machine learning models applied to solar and wind forecasting in Brazil and Spain. Applied Energy, Elsevier, v. 314, p. 118936, 2022.

[36] WANG, J. et al. Multi-target regression via target specific features. Knowledge-Based Systems, Elsevier, v. 170, p. 70–78, 2019.

[37] SPYROMITROS-XIOUFIS, E. et al. Multi-target regression via input space expansion: treating targets as inputs. Machine Learning, Springer, v. 104, n. 1, p. 55–98, 2016.

[38] MELKI, G. et al. Multi-target support vector regression via correlation regressor chains. Information Sciences, Elsevier, v. 415, p. 53–69, 2017.

[39] ZHEN, X. et al. Multi-target regression via robust low-rank learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, IEEE, v. 40, n. 2, p. 497–504, 2017.

[40] FACELI, K. et al. Inteligência artificial: uma abordagem de aprendizado de máquina. [S.l.]: Grupo Gen - LTC, 2011.

[41] MÜLLER, A. C.; GUIDO, S. Introduction to machine learning with Python: a guide for data scientists. Sebastopol, CA: O’Reilly Media, Inc., 2016.

[42] JASTRZEBSKI, Z. Nature and Properties of Engineering Materials. Hoboken, NJ: Wiley, 1959. (Wiley international edition). ISBN 9780471440888. Disponível em: ⟨https://books.google.com.br/books?id=ucY3zQEACAAJ⟩.

[43] HALL, L. J. et al. Sign change of Poisson’s ratio for carbon nanotube sheets. Science, v. 320, n. 5875, p. 504–507, 2008. Disponível em: ⟨https://www.science.org/doi/abs/10.1126/science.1149815⟩.

[44] SCAFFARO, R.; BOTTA, L. Chapter 5 - Nanofilled thermoplastic–thermoplastic polymer blends. In: THOMAS, S.; SHANKS, R.; CHANDRASEKHARAKURUP, S. (Ed.). Nanostructured Polymer Blends. Oxford: William Andrew Publishing, 2014. p. 133–160. ISBN 978-1-4557-3159-6. Disponível em: ⟨https://www.sciencedirect.com/science/article/pii/B9781455731596000055⟩.

[45] HATZIKOS, E. V. et al. An empirical study on sea water quality prediction. Knowledge-Based Systems, Elsevier, v. 21, n. 6, p. 471–478, 2008.

[46] GROVES, W.; GINI, M. On optimizing airline ticket purchase timing. ACM Transactions on Intelligent Systems and Technology (TIST), ACM New York, NY, USA, v. 7, n. 1, p. 1–28, 2015.

[47] KARALIČ, A.; BRATKO, I. First order regression. Machine Learning, Springer, v. 26, n. 2, p. 147–176, 1997.

[48] TSANAS, A.; XIFARA, A. Accurate quantitative estimation of energy performance of residential buildings using statistical machine learning tools. Energy and Buildings, Elsevier, v. 49, p. 560–567, 2012.

[49] GOOVAERTS, P. Geostatistics for Natural Resources Evaluation. Northamptonshire: Oxford University Press, 1997. (Applied geostatistics series). ISBN 9780195115383. Disponível em: ⟨https://books.google.com.br/books?id=CW-7tHAaVR0C⟩.

[50] CHUDZICKI, D. Online product sales. 2012. Online. Disponível em: ⟨https://www.kaggle.com/c/online-sales⟩.

[51] CUKIERSKI, W. See Click Predict Fix. 2013. Online. Disponível em: ⟨https://www.kaggle.com/c/see-click-predict-fix⟩.

[52] BACHE, K.; LICHMAN, M. UCI Machine Learning Repository. 2013. Online. Disponível em: ⟨http://archive.ics.uci.edu/ml⟩.

[53] YEH, I.-C. Modeling slump flow of concrete using second-order regressions and artificial neural networks. Cement and concrete composites, Elsevier, v. 29, n. 6, p. 474–480, 2007.

[54] DŽEROSKI, S.; DEMŠAR, D.; GRBOVIĆ, J. Predicting chemical parameters of river water quality from bioindicator data. Applied Intelligence, Springer, v. 13, n. 1, p. 7–17, 2000.

[55] SHAHHOSSEINI, M.; HU, G.; PHAM, H. Optimizing ensemble weights and hyperparameters of machine learning models for regression problems. Machine Learning with Applications, Elsevier, v. 7, p. 100251, 2022.

[56] DEMŠAR, J. Statistical comparisons of classifiers over multiple data sets. The Journal of Machine Learning Research, JMLR.org, v. 7, p. 1–30, 2006.

[57] DUNN, O. J. Multiple comparisons among means. Journal of the American Statistical Association, Taylor & Francis, v. 56, n. 293, p. 52–64, 1961.