Efeitos dos sistemas de avaliação de pesquisa de CAPES e CNPQ nos padrões de publicação dos pesquisadores das ciências da saúde no Brasil

Alejandro Caballero Rivero; Raimundo Nonato Macedo dos Santos; Piotr Trzesniak

doi:10.1590/1808-5245.30.138437

Authors

Alejandro Caballero Rivero National Institute of the Atlantic Forest (INMA) https://orcid.org/0000-0003-1061-0534
Raimundo Nonato Macedo dos Santos Federal University of Pernambuco https://orcid.org/0000-0002-9208-3266
Piotr Trzesniak Federal University of Pernambuco (UFPE) https://orcid.org/0000-0002-2833-1923

DOI:

https://doi.org/10.1590/1808-5245.30.138437

Keywords:

research evaluation systems, publication patterns, health sciences, Brazil

Abstract

Introduction: despite the increasing role of research evaluation and funding systems in evaluating the performance of researchers, there is a lack of empirical studies that analyze whether these systems influence their publication patterns. Objective: to analyze whether the evaluation criteria used by the Coordination for the Improvement of Higher Education Personnel and the National Council for Scientific and Technological Development influence the publication patterns of Health Sciences researchers in Brazil. Methodology: a semi-structured questionnaire is applied to a random stratified sample made up of two homogeneous strata: (a) researchers who are permanent professors in graduate programs and research productivity fellows; and (b) researchers who are not permanent professors or scholarship holders. Results: criteria more directly related to the evaluations, such as the score received by the articles in the evaluations, the demanded number of articles, the indexing of the journals in the main databases, or the value of the journal's bibliometric indicators, influence the choices of the evaluated researchers more than that of the non-evaluated ones. Factors not directly related to the evaluations, such as the importance of the articles for increasing the researcher’s reputation, the degree of dissemination of this type of publication, the journal’s prestige, or the quality peer review of the journal, also influence the choices of evaluated researchers. Conclusions: Researchers’ responses to assessments depend on several factors; therefore, it is not possible to establish a deterministic cause-and-effect relationship between the evaluation criteria more directly related to the evaluations and the publication patterns of the researchers.

Downloads

Download data is not yet available.

Author Biographies

Alejandro Caballero Rivero, National Institute of the Atlantic Forest (INMA)

PhD in Information Science, Graduate Program in Information Science at the Federal University of Pernambuco (PPGCI/UFPE), Recife, Brazil. Scholarship holder of the Institutional Training Program of the National Institute of the Atlantic Forest (INMA).

Raimundo Nonato Macedo dos Santos, Federal University of Pernambuco

PhD in Information Stratégique Et Critique Veille Technol -Université Paul Cézanne Aix Marseille III (AMU). Professor of the Graduate Program in Information Science (PPGCI) at the Federal University of Pernambuco (UFPE), Recife, Brazil.

Piotr Trzesniak, Federal University of Pernambuco (UFPE)

PhD in Physics, Graduate Program in Physics at the Physics Institute of the University of São Paulo (IFUSP). Professor of the Professional Master's Program in Public Management at the Federal University of Pernambuco (UFPE), Recife, Brazil.

References

AAGAARD, K. How incentives trickle down: local use of a national bibliometric indicator system. Science and Public Policy, Oxford, v. 42, n. 5, p. 725-737, 2015. Disponível em: https://doi.org/10.1093/scipol/scu087. Acesso em: 25 jun. 2021.

AAGAARD, K.; SCHNEIDER, J. W. Some considerations about causes and effects in studies of performance-based research funding systems. Journal of Informetrics, Netherlands, v. 11, n. 3, p. 923-926, 2017. Disponível em: https://doi.org/10.1016/j.joi.2017.05.018. Acesso em: 25 jun. 2021.

BAL, R. Playing the Indicator Game: reflections on strategies to position an sts group in a multi-disciplinary environment. Engaging Science, Technology, and Society, Evansville, v. 3, p. 41-52, 2017. Disponível em: https://doi.org/10.17351/ests2017.111. Acesso em: 25 jun. 2021.

BOURDIEU, P. Science of science and reflexivity. Chicago: Chicago University Press, 2004.

BUELA-CASAL, G.; ZYCH, I. What do the scientists think about the impact factor? Scientometrics, New York, v. 92, n. 2, p. 281-292, 2012. Disponível em: https://doi.org/10.1007/s11192-012-0676-y. Acesso em: 25 jun. 2021.

CABALLERO RIVERO, A.; SANTOS, R. N. M.; TRZESNIAK, P. Associação entre os sistemas de avaliação da pesquisa e os padrões de publicação nas Ciências da Saúde no Brasil. Informação & Informação, Londrina, v. 27, n. 3, p. 288-316, 2022. Disponível em: https://doi.org/10.5433/1981-8920.2022v27n3p288. Acesso em: 25 jun. 2021.

CARVALHO, K.; ODDONE, N. E.; CAFÉ, A. L. P.; MENEZES, V. Aspectos gerenciais da política científica brasileira: um olhar sobre a produção científica do campo da sociologia face aos critérios de avaliação do CNPq e da CAPES. Em Questão, Porto Alegre, v.19, n. 1, p. 187-212, 2013. Disponível em: https://www.redalyc.org/articulo.oa?id=465645972011. Acesso em: 25 jun. 2021.

FRY, J.; OPPENHEIM, C.; CREASER, C.; JOHNSON, W.; SUMMERS, M.; WHITE, S.; BUTTERS, G.; CRAVEN, J.; GRIFFITHS, J.; HARTLEY, D. Communicating knowledge: how and why researchers publish and disseminate their findings. The Research Information Network, London, Sept. 2009. Disponível em: http://www.rin.ac.uk/system/files/attachments/Communicating-knowledge-report.pdf. Acesso em: 29 jun. 2021.

HICKS, D. Performance-based university research funding systems. Research Policy, Netherlands, v. 41, n. 2, p. 251-261, 2012. Disponível em: https://doi.org/10.1016/j.respol.2011.09.007. Acesso em: 25 jun. 2021.

GÉNOVA, G.; ASTUDILLO, H.; FRAGA, A. The scientometric bubble considered Harmful. Zaher, s.l.,v. 22, n. 1, p. 227-235, 2016. Disponível em: https://doi.org/10.1007/s11948-015-9632-6. Acesso em: 25 jun. 2021.

GIBBONS, M.; LIMOGES, C.; SCHWARTZMAN, S.; SCOTT, P.; TROW, M. The new production of knowledge: the dynamics of science and research in contemporary societies. London: Sage, 1994.

GLÄSER, J. A fight on epistemological quicksand: comment on the dispute between Van den Besseelaar et al. and Butler. Journal of Informetrics, Netherlands, v. 11, n. 3, p. 927-932, 2017. Disponível em: https://doi.org/10.1016/j.joi.2017.05.019. Acesso em: 25 jun. 2021.

GLÄSER, J.; LAUDEL, G. Evaluation without evaluators: the impact of funding formulae on Australian University Research. In: WHITLEY, R.; GLÄSSER, J. (ed.). The changing governance of the sciences: the advent of Research Evaluation Systems. Dordrecht: Springer, 2007. p. 127-152.

GLÄSER, J.; LAUDEL, G. Governing Science. European Journal of Sociology, Cambridge, v. 57, n. 1, p. 117-168, 2016. Disponível em: https://doi.org/10.1017/S0003975616000047. Acesso em: 25 jun. 2021.

GLÄSER, J.; LANGE, S.; LAUDEL, G.; SCHIMANK, U. The limits of universality: how field-specific epistemic conditions affect authority relations and their consequences. In: WHITLEY, R.; GLÄSER, J.; ENGWALL, L. (ed.), Reconfiguring knowledge production: changing authority relationships in the sciences and their consequences for intellectual innovation. Oxford: Oxford University Press, 2010. p. 291-325.

HAMMARFELT, B. Recognition and reward in the academy: Valuing publication oeuvres in biomedicine, economics and history. Aslib Journal of Information Management, Leeds, v. 69, n. 5, p. 607-623, 2017. Disponível em: https://doi.org/10.1108/AJIM-01-2017-0006. Acesso em: 25 jun. 2021.

HAMMARFELT, B.; RICKJE, S. Accountability in context: effects of research evaluation systems on publication practices, disciplinary norms, and individual working routines in the faculty of Arts at Uppsala University. Research Evaluation, Oxford, v. 24, n. 1, p. 63-77, 2015. Disponível em: https://doi.org/10.1093/reseval/rvu029. Acesso em: 25 jun. 2021.

JIMÉNEZ-CONTRERAS, E.; MOYA-ANEGÓN, F.; LÓPEZ-CÓZAR, E. D. The evolution of research activity in Spain: the impact of the National Commission for the Evaluation of Research Activity (CNEAI). Research Policy, Netherlands, v. 32, n. 1, p. 123-142, 2003. Disponível em: https://doi.org/10.1016/S0048-7333(02)00008-2. Acesso em: 25 jun. 2021.

JOHNSON, R.; ATKINSON, A.; MABE, M. The STM report: an overview of scientific and scholarly publishing. Netherlands: International Association of Scientific, Technical and Medical Publishers, 2018. Disponível em: https://www.stm-assoc.org/2018_10_04_STM_Report_2018.pdf. Acesso em: 18 mai. 2021.

KELLOGG, D. Toward a post-academic science policy: scientific communication and the collapse of the mertonian norms. International Journal of Communications Law & Policy, London, special issue, Fall 2006. Disponível em: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=900042. Acesso em: 9 jul. 2019.

KNELLER, R. Prospective and retrospective evaluation systems in context: insights from Japan. In: WHITLEY, R.; GLÄSSER, J. (ed.). The changing governance of the sciences: the advent of Research Evaluation Systems. Dordrecht: Springer, 2007. p. 51-74.

KORYTKOWSKI, P., KULCZYCKI, E. Examining how country-level science policy shapes publication patterns: the case of Poland. Scientometrics, New York, v. 119, n. 3, p. 1519-1543, 2019. Disponível em: https://doi.org/10.1007/s11192-019-03092-1. Acesso em: 25 jun. 2021.

MARQUES, M.; POWELL, J. J. W.; ZAPP, M.; BIESTA, G. How does research evaluation impact educational research? Exploring intended and unintended consequences of research assessment in the United Kingdom, 1986-2014. European Educational Research Journal, New Jerey, v. 16, n. 6, p. 820-842, 2017. Disponível em: https://doi.org/10.1177/1474904117730159. Acesso em: 25 jun. 2021.

MENEZES, V., ODDONE, N. E., CAFÉ, A. L. P. Aspectos reputacionais dos sistemas de avaliação da produção científica no campo da ciência da informação. Tendências da Pesquisa Brasileira em Ciência da Informação, Rio de Janeiro, v. 5, n. 1, 2012.

O’MEARA, K. A. Inside the panopticom: studying academic reward systems. In: SMART, J.; PAULSEN, M. (ed.). Higher Education: Handbook of Theory and Research. Dordrecht: Springer, 2011. p. 161-220.

OSSENBLOK, T. L.; ENGELS, T. C.; SIVERTSEN, G. The representation of the social sciences and humanities in the Web of Science: a comparison of publication patterns and incentive structures in Flanders and Norway (2005-9), Research Evaluation, Oxford, v. 21, n. 4, p. 280-290, 2012. Disponível em: https://doi.org/10.1093/reseval/rvs019. Acesso em: 25 jun. 2021.

RICKJE, S.; WOUTERS, P. F.; RUSHFORTH, A. D.; FRANSSEN, T. P.; HAMMARFELT, B. Evaluation practices and effects of indicator use a literature review. Research Evaluation, Oxford, v. 25, n. 2, p. 161-169, 2016. Disponível em: https://doi.org/10.1093/reseval/rvv038. Acesso em: 25 jun. 2021.

SCHNEIDER, J. W.; AAGAARD, K.; BLOCH, C. W. What happens when national research funding is linked to differentiated publication counts? A comparison of the Australian and Norwegian publication-based funding models. Research Evaluation, Oxford, v. 25, n. 3, p. 244-256, 2016. Disponível em: https://doi.org/10.1093/reseval/rvv036. Acesso em: 25 jun. 2021.

SILE, L.; VANDERSTRAETEN, R. Measuring changes in publication patterns in a context of performance based research funding systems: the case of educational research in the University of Gothenburg (2005-2014). Scientometrics, New York, v. 118, p. 71-91, 2019. Disponível em: https://doi.org/10.1007/s11192-018-2963-8. Acesso em: 25 jun. 2021.

SOUZA, C. D.; FILIPO, D.; SANZ CASADO, E. Crescimento da atividade científica nas universidades federais brasileiras: análise por áreas temáticas. Avaliação, Campinas, v. 23, n. 1, p. 126-156, 2018. Disponível em: http://dx.doi.org/10.1590/s1414-40772018000100008. Acesso em: 25 jun. 2021.

STATISTICAL PACKAGE FOR THE SOCIAL SCIENCES (SPSS). Software IBM SPSS. Disponível em: https://www.ibm.com/br-pt/spss. Acesso em: 30 jun. 2021.

VAN SELM, M.; JANKOWSKI, N. W. Conducting online surveys. Quality & Quantity, New York, v. 40, n. 3, p. 435-456, 2006. Disponível em: https://doi.org/10.1007/s11135-005-8081-8. Acesso em: 25 jun. 2021.

WHITLEY, R. The intellectual and social organization of the sciences. 2 nd ed. Oxford: Oxford University Press, 2000.

WHITLEY, R. Changing governance of the public sciences: the consequences of establishing research evaluation systems for knowledge production in different countries and scientific fields. In: WHITLEY, R.; GLÄSER, J. (ed.). The changing governance of the sciences: the advent of Research Evaluation Systems. Dordrecht: Springer, 2007. p. 3-30.

WHITLEY, R.; GLÄSER, J. (ed.). The changing governance of the sciences: the advent of Research Evaluation Systems. Dordrecht: Springer, 2007.