Acessibilidade / Reportar erro

Impact of PARFOR on Public Elementary Schools

Abstract:

This study assesses the impact of the Plano Nacional de Formação de Professores da Educação Básica (PARFOR) in public schools. Operationally, the universe of PARFOR was reduced to Pedagogia courses and, its results, to the elementary school. The differences in differences (DD) method was used to assess whether there was an increase in the number of pedagogos, working in the benefited public schools, as well as an improvement in the Índice de Desenvolvimento da Educação Básica (IDEB) of those same schools. The results showed that PARFOR promoted a small increase in the number of pedagogos in school units, but about IDEB, the results were not conclusive.

Keywords:
PARFOR; Teacher Education; Impact Assessment; Differences in Differences

Resumo:

Este estudo avalia o impacto do Plano Nacional de Formação de Professores da Educação Básica (Parfor) nas escolas públicas. Operacionalmente, reduziu-se o universo do Parfor aos cursos de Pedagogia e, seus resultados, aos anos iniciais do ensino fundamental. Utilizou-se o método diferenças em diferenças (DD) para avaliar se houve aumento no número de pedagogos em atuação nas escolas públicas beneficiadas, bem como melhoria no Índice de Desenvolvimento da Educação Básica (Ideb) dessas mesmas escolas. Os resultados apontaram que o Parfor promoveu um pequeno aumento na quantidade de pedagogos nas unidades escolares, mas no que se refere ao Ideb os resultados não foram conclusivos.

Palavras-chave:
Parfor; Formação de Professores; Avaliação de Impacto; Diferenças em Diferenças

Introduction

This article presents the results of the impact assessment of the Plano Nacional de Formação de Professores da Educação Básica (PARFOR)1 1 Translation note: In Brazil, in comparison with the American model, Educação Básica refers to: Educação Infantil (Kindergarten), to anos iniciais do Ensino Fundamental (Elementary School), to anos finais do Ensino Fundamental (Middle School) and Ensino Médio (High School). Therefore, we translate Ensino Fundamental as Elementary and Middle Schools and we do not translate Educação Básica, keeping the use of the term in Portuguese. , assessing the existence of causality between the actions of the Plan, the improvement of the profile of the teaching staff and the quality of public schools in elementary school - early years. This is a quantitative study, based on the differences in differences (DD) method, which seeks to identify the effects promoted by PARFOR’s Pedagogia2 2 Translation note: We do not translate Pedagogia as Pedagogy, keeping the term in Portuguese, considering that, in Brazil, the Pedagogia course allows the exercise of teaching in Educação Infantil and anos iniciais do Ensino Fundamental - in American system, the Bachelors or Master’s Degree in Early Childhood and Elementary Education is a training requirement for acting in Kindergarten and Elementary School. Following the same reasoning, we do not translate pedagogos as educators with diploma in teaching, also keeping the term in Portuguese. courses on the number of pedagogos, working in the benefited public schools, as well as on the Índice de Desenvolvimento da Educação Básica (IDEB3 3 Official quality indicator of Brazilian education by the National Education Plan (PNE), Law No. 13,005, of June 24, 2014 (Brazil, 2014). ) of those same schools.

This study is part of the discussions that signal the importance of developing research on the relationship between teacher education and student learning (Cochran-Smith; Zeichner, 2005COCHRAN-SMITH, Marilyn; ZEICHNER, Kenneth (Org.). Studying Teacher Education: the report of the AERA panel on research and teacher education. Mahwah: Lawrence Erlbaum Associates, 2005.; Yoon et al., 2007YOON, Kwang Suk et al. Reviewing the Evidence on How Teacher Professional Development Affects Student Achievement. Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southwest. Issues & Answers, n. 33, 2007. 62 p. Disponível em: <https://files.eric.ed.gov/fulltext/ED498548.pdf>. Acesso em: 19 nov. 2019.
https://files.eric.ed.gov/fulltext/ED498...
; Goldhaber, 2013GOLDHABER, Dan. What Do Value-Added Measures of Teacher Preparation Programs Tell Us? carnegie foundation for the advancement of teaching, Stanford, CA: Carnegie Knowledge Network. What We Know Series: Value-Added Methods and Applications, p. 14, nov. 2013. Disponnível em: <http://www.carnegieknowledgenetwork.org/wp-content/uploads/2013/11/CKN-Goldhaber-TeacherPrep_Final_11.7.pdf>. Acesso em: 01 jun. 2020.
http://www.carnegieknowledgenetwork.org/...
; Sleeter, 2014SLEETER, Christine. Toward Teacher Education Research that Informs Policy. Educational Researcher, Washington, v. 43 n. 3, p. 146-153, abr. 2014.; AERA, 2015AERA. Statement on Use of Value-Added Models (VAM) for the Evaluation of Educators and Educator Preparation Programs. Educational Researcher, Washington, v. 44, n. 8, p. 448-452, nov. 2015.; Rowan et al., 2015ROWAN, Leonie et al. Investigating the Effectiveness of Teacher Education for Early Career Teachers in Diverse Settings: the longitudinal research we have to have. The Australian Educational Researcher, v. 42, n. 3, p. 273-298, jul. 2015.; Cochran-Smith et al., 2016COCHRAN-SMITH, Marilyn et al. Holding Teacher Preparation Accountable: a review of claims and evidence. Boulder: National Education Policy Center, 2016. Disponível em: <https://nepc.colorado.edu/sites/default/files/pb_cochran-smith_teacher_prep_0.pdf>. Acesso em: 19 nov. 2019.
https://nepc.colorado.edu/sites/default/...
). Despite the relevance that teacher education has gained for educational reforms developed by several countries (OECD, 2011OCDE. Building a High-Quality Teaching Profession: lessons from around the world. Paris: OECD Publishing, 2011. 65 p., 2013OCDE. Learning standards, teaching standards, and standards for school principals: a comparative study. OECD Education Working Papers, Chile, Centre of Study for Policies and Practices in Education (CEPPE), n. 99, 19 dez. 2013., 2018OCDE. Effective Teacher Policies: insights from Pisa. Paris: OECD Publishing, 2018, 163 p.), it is relatively common to find divergences in the area’s literature on the best way to develop this formation, justifying the realization of evaluations that can contribute to the gathering of evidence to support the theme. In addition, the most appropriate methodological paths to ascertain such evidence are not consolidated, encouraging the realization of experiments to generate knowledge in this aspect as well.

PARFOR4 4 Translation note: We prefer not to translate terms like PARFOR and IDEB and others that refer to Brazilian public policies. As the text explains what they are, we therefore chose to allow the reader to understand their meaning in the Brazilian context, and not by inference from English words that do not have the same meaning. Therefore, we do not translate, but we emphasize in italics: Plano Nacional de Formação de Professores da Educação Básica (PARFOR), Índice de Desenvolvimento da Educação Básica (IDEB) e Política Nacional de Formação de Profissionais do Magistério da Educação Básica. was created in 2009 to promote the initial and continuing formation of public-school teachers, with a view to overcoming the deficit of qualified teachers working in schools and, at the same time, expand the conditions for improving the quality of the education offered. In 2016, the edition of Decree No. 8,752, of May 9, 2016 (Brazil, 2016BRASIL. Decreto nº 8.752, de 9 de maio de 2016. Revoga o Decreto 6.755/2009 e dispõe sobre a Política Nacional de Formação dos Profissionais da Educação Básica. Diário Oficial [da] República Federativa do Brasil, Brasília, DF, 10 mai. 2016. Disponível em: <http://www.planalto.gov.br >. Acesso em: 30 jun. 2019.
http://www.planalto.gov.br...
) changed the Política Nacional de Formação de Profissionais do Magistério da Educação Básica5 5 Decree No. 6,755, of January 29, 2009 (Brazil, 2009a). and, consequently, revoked PARFOR’s legal basis6 6 Normative Ordinance No. 9, of June 30, 2009 (Brazil, 2009b). . In this sense, as a turning point, the new decree catalyzed the relevance of evaluating the results generated by the Plan until the moment of the change.

From a methodological point of view, the universe of this analysis was restricted to the initial training offered by PARFOR, more precisely to Pedagogia courses, with a view to: i) the relative importance of initial formation in all PARFOR’s actions; ii) the centrality of Pedagogia as the main area covered by PARFOR during the analyzed period; iii) the great demand of public education networks for qualified pedagogos; and finally, iv) the need to locate the impacts generated by PARFOR by type of formation, separating the interpretive argument and enabling the evidence generated by it to be valid and reliable, as advocated by the American Educational Research Association (AERA, 2015AERA. Statement on Use of Value-Added Models (VAM) for the Evaluation of Educators and Educator Preparation Programs. Educational Researcher, Washington, v. 44, n. 8, p. 448-452, nov. 2015.).

It should also be noted that, conceptually, evaluation is understood, in this work, as a legitimate value judgment on a given object, aiming to subsidize the decision making about it. Based on this concept, it is assumed that the value judgment arising from the PARFOR evaluation process is legitimate when based on criteria that reflect the fulfillment (or not) of its objectives, such as those defined in Art. 1 of the Normative Ordinance No. 9, of June 30, 2009 (Brazil, 2009bBRASIL. Ministério da Educação. Conselho Nacional de Educação. Resolução CNE/CP nº 1, de 15 de maio de 2006, que institui Diretrizes Curriculares Nacionais para o Curso de Graduação em Pedagogia, licenciatura. Brasília, DF, 16 maio. 2006. Disponível em: <http://meclegis.mec.gov.br>. Acesso em 8 out. 2019.
http://meclegis.mec.gov.br...
), which establishes as the purpose of the Plan to meet the demand for training of teachers of public education networks, as well as in Item I of Art. 3 of Decree No. 6,755, of January 29, 2009 (Brazil, 2009aBRASIL. Ministério da Educação. Conselho Nacional de Educação. Resolução CNE/CP nº 1, de 18 de fevereiro de 2002, que institui Diretrizes Curriculares Nacionais para a Formação de Professores da Educação Básica, em nível superior, curso de licenciatura, de graduação plena. Diário Oficial [da] República Federativa do Brasil, Brasília, DF, 19 fev. 2002. Disponível em: <http://portal.mec.gov.br/conselho-nacional-de-educacao/atos-normativos--sumulas-pareceres-e-resolucoes?id=12816>. Acesso em 8 out. 2019.
http://portal.mec.gov.br/conselho-nacion...
), which indicates the improvement of the quality of public Educação Básica as one of the objectives to be pursued by the actions of PARFOR in the period used here as a reference for analysis.

Operationally, these purposes were associated, in this work, with the proxies variables: a) a number of qualified pedagogos, working in public elementary schools - initial years, referring to meeting the need for teacher training; and, b) IDEB calculated for these same public schools, referring to the improvement of the quality of education. With this, we sought to maintain coherence, within the scope of the study under study, between the cause variable (degree in Pedagogia by PARFOR) and the effect variables (number of qualified pedagogos; and IDEB of the initial years of the schools).

Based on the chosen paths, this work was organized in four parts, excluding this Introduction and Final Considerations. In the first, general information about PARFOR and the main numbers involved in its execution are presented. In the second, PARFOR is analyzed in the context of the theoretical discussion that involves teacher education and the assessment of its impact. In the third, the methodology used in the study is described, and finally, in the fourth, the results found and some recommendations from them are presented. In the Final Considerations, the main findings are highlighted, as well as aspects that could be considered in other PARFOR evaluations.

About PARFOR

PARFOR is managed by the Coordination for the Improvement of Higher Education Personnel (CAPES). Until 2016, PARFOR’s execution was divided into two: face-to-face and distance. Face-to-face PARFOR, concentrated courses aimed at initial formation in the face-to-face modality, for teachers from state and municipal public networks of Educação Básica without formation at a higher level (first degree), who work in areas other than their initial formation (second degree) or who do not have a degree in education (pedagogical formation). Distance learning courses for initial and continuing formation of teachers who are part of PARFOR were promoted within the scope of the Open University of Brazil (UAB).

As of 2016, the new regulation of the Política Nacional de Formação dos Profissionais da Educação Básica (Brazil, 2016BRASIL. Decreto nº 8.752, de 9 de maio de 2016. Revoga o Decreto 6.755/2009 e dispõe sobre a Política Nacional de Formação dos Profissionais da Educação Básica. Diário Oficial [da] República Federativa do Brasil, Brasília, DF, 10 mai. 2016. Disponível em: <http://www.planalto.gov.br >. Acesso em: 30 jun. 2019.
http://www.planalto.gov.br...
) established some changes in the guidelines that had been in place since 2009 (Brazil, 2009aBRASIL. Ministério da Educação. Conselho Nacional de Educação. Resolução CNE/CP nº 1, de 18 de fevereiro de 2002, que institui Diretrizes Curriculares Nacionais para a Formação de Professores da Educação Básica, em nível superior, curso de licenciatura, de graduação plena. Diário Oficial [da] República Federativa do Brasil, Brasília, DF, 19 fev. 2002. Disponível em: <http://portal.mec.gov.br/conselho-nacional-de-educacao/atos-normativos--sumulas-pareceres-e-resolucoes?id=12816>. Acesso em 8 out. 2019.
http://portal.mec.gov.br/conselho-nacion...
). Among them, there was a change in the objectives and actions of the aforementioned Policy, with emphasis on the creation of Programa Nacional de Formação de Profissionais da Educação Básica, possibly a substitute for PARFOR, and a national test for teachers. However, it is worth noting that the current policy outlined under Decree No. 8,752, of May 9, 2016 (Brazil, 2016BRASIL. Decreto nº 8.752, de 9 de maio de 2016. Revoga o Decreto 6.755/2009 e dispõe sobre a Política Nacional de Formação dos Profissionais da Educação Básica. Diário Oficial [da] República Federativa do Brasil, Brasília, DF, 10 mai. 2016. Disponível em: <http://www.planalto.gov.br >. Acesso em: 30 jun. 2019.
http://www.planalto.gov.br...
) is not yet in force due to lack of regulation, despite having already revoked the previous rule.

Considering the numbers of PARFOR since its installation in 2009 and until 2014, there are more than 400 thousand enrollments of teachers of Educação Básica (Figure 1), notably concentrated in the distance modality (about 80% of enrollments). Regarding the type of course, more than half of PARFOR’s enrollments took place in undergraduate courses, confirming its purpose of contributing to the fulfillment of the training requirements at a higher level for all teachers, as provided for in the Law of Guidelines and Bases of National Education/LDB (Brazil, 1996).

Figure 1
Enrolled in PARFOR considering Type of Course - Classes from 2009 to 2014

According to data in Table 1, which shows the distribution of enrollments in the Plan by degree area, it is observed that Pedagogia accounts for 28% of these enrollments. Second, comes the Mathematics area, with about 13% of the number of enrollments in the Plan. Taking only face-to-face undergraduate degrees, Pedagogia alone accounts for almost 40% of enrollments and, next, comes the area of Languages, with 11% of enrollments in this course modality.

Table 1
Enrollments in PARFOR Degrees with a teaching diploma divided by bachelor's degrees, Classes 2009 to 2014

Apparently, this concentration of courses offered by PARFOR in the initial training of teachers and, more specifically, in the area of Pedagogia is justified by the data presented by the Statistical Synopsis of the Educação Básica Teacher (INEP, 2009INEP. Sinopse Estatística do Professor da Educação Básica 2009. Brasília, 2009. Disponível em: <http://www.inep.gov.br>. Acesso em: 20 jun. 2020.
http://www.inep.gov.br...
). According to this Synopsis, when PARFOR was created in 2009, about 637,000 teachers in schools in the country did not have higher education (30% of the total universe of almost two million teachers). Specifically in Kindergarten and in Elementary Education, the formation indicators were even worse than the general average: 52% and 38% of teachers at these levels of school performance, respectively, did not have the legally required professional training. In this way, it can be learned that PARFOR worked precisely at school levels with the greatest deficiency of teachers with adequate formation.

From the point of view of student completion by type of courses, Figure 2 confirms the largest number of students at PARFOR in distance learning courses. However, this numerical superiority of graduated students is explained by the ones of distance specialization, improvement, and extension courses. In regard to degrees with a teaching diploma, the face-to-face offer is responsible for most of the graduates in the period under analysis, even though it received less than half of the enrollments of the distance offer.

Figure 2
PARFOR graduated students considering Course Type, Classes from 2009 to 2014

Considering that PARFOR is intended for the training of public-school teachers, a crossover was carried out between the PARFOR base and the 2015 School Census. The findings are systematized in Figure 3. They point out that about a third of the graduated students or taking the classroom course and two thirds of the graduated students or taking the distance course were not listed as Educação Básica teachers. Even considering the possibility that teachers may resign from state and municipal education networks during or after the conclusion of the course, the percentages of PARFOR participants outside the 2015 School Census seem much higher than expected for such cases. This fact indicates the need for greater control of the Plan regarding the selection of the student profile, defined as public school teachers.

Figure 3
Registration in the School Census / 2015 of Graduated Students or Studying Degree in Pedagogia by PARFOR

Thus, in general terms, the data presented here allow an overview of PARFOR’s actions between 2009 and 2014. Taking these data as a reference, notably those related to Pedagogia courses, we sought to identify the impacts generated by PARFOR’s actions, respectively, on the number of pedagogos working in schools and the quality presented by them. The theoretical-methodological reference used in this analysis, as well as the results found are specified in the sections that follow.

Teacher Education and its Impact Evaluation

Considering the contributions of Cochran-Smith and Zeichner (2005)COCHRAN-SMITH, Marilyn; ZEICHNER, Kenneth (Org.). Studying Teacher Education: the report of the AERA panel on research and teacher education. Mahwah: Lawrence Erlbaum Associates, 2005. and the objectives of this study, the present analysis was structured from two theoretical aspects articulated with each other. First, in the recognition that the different approaches to teacher education have an impact not only in academic discussions on the topic, but also in the formulation of public policies. Second, in the verification of the lack of studies on the impact of teacher education, highlighting the need to discuss, also, the research designs most appropriate to the identification of said evidence.

From the point of view of academic discussions on teacher training and its repercussions, Darling-Hammond and Youngs (2002)DARLING-HAMMOND, Linda; YOUNGS, Peter. Defining ‘highly qualified teachers’: what does ‘scientifically-based research’ actually tell us. Educational Researcher, Washington, v. 31, n. 9, p. 13-25, dez. 2002. illustrate divergences in conceptions and empirical analyzes that have occurred in the USA around the relationship between teacher education, certification, and its effectiveness in the classroom (characteristics of teachers versus characteristics of training programs); on the definition of the most important components for teacher effectiveness (skills), as well as on the benefits (or not) of alternative teacher training programs, when compared to traditional training programs for preparing teachers for work.

In Brazil, over the past few years, it has been possible to observe divergences between conceptions about what is most suitable for Teacher Education. According to Araújo (2009)ARAÚJO, Viviane Patricia Colloca. A Multiculturalidade nas Políticas Educacionais e a Formação de Professores: Brasil e Portugal. 2009. 436 f. Tese (Doutorado em Educação) -. Universidade Federal de São Carlos, São Carlos, 2009., such conceptions orbit around three theoretical approaches, namely: technical rationality (emphasizes theoretical knowledge to instrumentalize problem solving), practical rationality (emphasizes knowledge arising from practice, through reflection on the action and focused on concrete reality) and critical rationality (emphasizes knowledge focused on reflections for understanding and social transformation).

However, there are authors such as Rios (2010)RIOS, Terezinha. Compreender e Ensinar: por uma docência da melhor qualidade. 8. ed. São Paulo: Cortez, 2010., who argue for a plural perspective in teacher education, in which the technical, aesthetic, or creative, ethical, and political solidarity dimensions are presented as a whole articulated with each other. Libâneo (2008)LIBÂNEO, José Carlos. Reflexividade e Formação de Professores: outras oscilações do pensamento pedagógico brasileiro? In: PIMENTA, Selma Garrido; GHEDIN, Evandro (Org.). Professor Reflexivo no Brasil: gênese e crítica de um conceito. São Paulo: Cortez, 2008. P. 53-79. also shares an integrating vision of teacher education approaches, pointing out that the best program to educate teachers is the one that articulates, as a whole, scientific culture, instrumental content, learning spaces, in addition to ethical and political convictions, with no room for any reductionism in its approach.

Such differences in conceptions reverberated in recurring changes in national guidelines that guide initial teacher education (Brazil, 2002, 2015BRASIL. Ministério da Educação. Conselho Nacional de Educação. Resolução CNE/CP nº 2, de 1º de julho de 2015, que define as Diretrizes Curriculares Nacionais para a formação inicial em nível superior (cursos de licenciatura, cursos de formação pedagógica para graduados e cursos de segunda licenciatura) e para a formação continuada. Diário Oficial [da] República Federativa do Brasil, Brasília, DF, 02 jul. 2015. Disponível em: <http://portal.mec.gov.br/conselho-nacional-de-educacao/atos-normativos--sumulas-pareceres-e-resolucoes?id=12816>. Acesso em 8 out. 2019.
http://portal.mec.gov.br/conselho-nacion...
, 2019BRASIL. Ministério da Educação. Conselho Nacional de Educação. Resolução CNE/CP nº 2, de 20 de dezembro de 2019 - Define as Diretrizes Curriculares Nacionais para a Formação Inicial de Professores para a Educação Básica e institui a Base Nacional Comum para a Formação Inicial de Professores da Educação Básica (BNC-Formação). Diário Oficial [da] República Federativa do Brasil, Brasília, DF, 20 dez. 2019. Disponível em: <http://portal.mec.gov.br/conselho-nacional-de-educacao/atos-normativos--sumulas-pareceres-e-resolucoes?id=12816>. Acesso em: 2 fev. 2020.
http://portal.mec.gov.br/conselho-nacion...
). Specifically with regard to the guidelines that guide the training of pedagogos, it was also possible to observe conflicts around the teaching base7 7 The proposal of the teaching base emerged from the movement of educators organized in the National Association for the Education Professionals Formation (Anfope), from the conception of teaching as a central element in the education of the pedagogue, with a view to overcoming the dichotomy between undergraduate and bachelor’s degrees , then identified. This proposal was incorporated into the Curricular Guidelines for the Pedagogy course by Resolution CNE / CP No. 1, of May 15, 2006 (Brazil, 2006). (Franco; Libâneo; Pimenta, 2007FRANCO, Maria Amélia Santoro; LIBÂNEO, José Carlos; PIMENTA, Selma Garrido. Elementos para a Formulação de Diretrizes Curriculares para Cursos de Pedagogia. Cadernos de Pesquisa, São Paulo, v. 37, n. 130, p. 63-97, jan./abr. 2007.; Scheibe, 2007SCHEIBE, Leda. Diretrizes Curriculares para o Curso de Pedagogia: trajetória longa e inconclusa. Cadernos de Pesquisa, v. 37, n. 130, p. 43-62, 2007.), reflecting on different opinions about the best way to educate these teachers to work in early childhood education and in the early years of elementary school, as well as in the organization of courses.

Researchers such as Gatti (2010)GATTI, Bernardete. Formação de Professores no Brasil: características e problemas. Educação & Sociedade, Campinas, v. 31, n. 113, p. 1355-1379, out./dez. 2010. and Mascarenhas and Franco (2017)MASCARENHAS, Aline Daiane Nunes; FRANCO, Maria Amélia Santoro. De Pedagogos a Professores: balanço de uma década das Diretrizes Curriculares dos Cursos de Pedagogia no Brasil. Revista Internacional de Formação de Professores, Itapetininga, v. 2, n. 1, p. 41-55, jan./mar. 2017. show that Brazilian degrees in general, and Pedagogia in particular, present problems related to the content covered in training courses, which can compromise the performance of teachers in the classroom, and consequently, the students’ own learning. With regard to the specific offer of Pedagogia courses by PARFOR, the study presented by Souza (2017)SOUZA, Valdinei Costa. Qualidade da formação de pedagogos na perspectiva da oferta do Parfor Presencial. Educação e Pesquisa, São Paulo, v. 43, n. 1, p. 82-96, jan./mar. 2017. signaled the absence of a formative model that can characterize face-to-face PARFOR as a unit. According to the author, different conceptions of teacher training coexist in the execution of the Plan, without articulations around basic foundations, shared by the courses as a formative unit. It was identified that the courses offered by PARFOR made use of both new pedagogical projects, specially designed to attend the training of lay teachers, as well as regular projects, aimed at training teachers who never practiced the profession. In addition, there was a diversity in the definition of the profile of the graduate, the practice of internship and complementary activities, among other aspects, which do not allow associating PARFOR’s offer with a specific and unitary teacher training proposal.

This fragmented view of PARFOR as a training project can be considered sui generis within the range of studies reviewed for this research, the vast majority of which were carried out abroad. In general, the teacher education programs analyzed have their own characterization, aiming to differentiate the experiences from each other and locate what is most efficient in terms of student learning. In the case of this analysis of the impact of PARFOR, due to the Plan’s own characteristics, it was not possible to signal, among the diversity of training proposals, which of them have more efficient approaches in relation to the observed effect variables.

As for methodological research designs with a focus on the impact of teacher training, it was found in Yoon et al. (2007)YOON, Kwang Suk et al. Reviewing the Evidence on How Teacher Professional Development Affects Student Achievement. Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southwest. Issues & Answers, n. 33, 2007. 62 p. Disponível em: <https://files.eric.ed.gov/fulltext/ED498548.pdf>. Acesso em: 19 nov. 2019.
https://files.eric.ed.gov/fulltext/ED498...
the idea that sample randomization is one of the fundamental elements for the validity of the results found. According to them, the systematic observation carried out from randomly chosen cases is a powerful tool to discard competing explanations around the same characteristics of training programs. The authors also argue that research designs should separately quantify the added value of teacher training in curricula, teaching methodologies, in the development of innovative teaching materials, as well as in student learning. In addition, as recommended, a rigorous research project must have externally valid findings; adequate statistical power to detect real effects and sufficient time between professional development; and the measurement of the results resulting from it.

Boyd et al. (2006)BOYD, Donald et al. Complex by Design: investigating pathways into teaching in New York city schools. Journal of Teacher Education, v. 57, n. 2, p. 155-166, mar./abr. 2006. also detailed the methodological challenges that need to be overcome so that research like this can be validated. Among these aspects, three in particular help to illuminate the present study, namely: the issue of selection bias, the difficulties in documenting programmatic resources of training courses and the challenges of estimating the effects of teachers on student performance.

The selection bias is related to the ability of the cases under study to effectively represent the investigated population. It occurs when the choice of the sample favors characteristics that interfere in the relationship between the studied variables and end up conditioning, by itself, the results found. An example of this can be observed in the case of an imbalance, between groups being compared, of variables that can contaminate the calculated result, such as performance scores in the training course subjects, among others. To avoid situations like this, it is important that the samples selected for the studies are chosen probabilistically, so that the cases to be compared have heterogeneous characteristics, without the privilege of specific aspects that may influence the final result found.

Concerning to the challenge of avoiding selection bias, it is important to mention the study by Mihaly et al. (2012)MIHALY, Kata et al. Where You Come from or Where You Go? Distinguishing Between School Quality and the Effectiveness of Teacher Preparation Program Graduates. Center for Analysis of Longitudinal Data in Education Research, Washington, D.C. Working Paper 63, 41 p, jan. 2012 Disponível em: <https://files.eric.ed.gov/fulltext/ED529178.pdf>. Acesso em: 01 jun. 2020.
https://files.eric.ed.gov/fulltext/ED529...
, who sought to control the school’s contextual bias, by inserting it as fixed effects in the performance models used to estimate the controls of the effects of the preparation program for the school environment. Such control is established by studying the differences between the results of students from the same school, and not from any school. This care allows for a closer approximation of the results found, removing the contextual component of the school environment as one of the intervening factors of the results obtained. In this analysis of PARFOR, we sought to locate the unit of analysis in the school to precisely meet this recommendation, avoiding the interventions of the school context in the results obtained.

In relation to the difficulties in documenting programmatic resources of courses, the problem is mainly related to the question of the existence of reliable databases on the structure of teacher training courses, as well as the definition of key elements that differentiate one formative course from others. In the particular case of PARFOR, there is no database that catalogs and differentiates the approaches of the degree courses offered. Such absence, therefore, constitutes an important barrier to assess the causal relationship between specific characteristics of training programs and their impact on student learning. Thus, due to lack of data, the differences between formative experiences of PARFOR’s Pedagogia courses were not considered.

Regarding the challenges of estimating the effects of teachers on student performance, it is necessary to overcome the measurement bias. This type of bias occurs when the measure used does not guarantee the comparability of responses between groups. In general, it is associated with the use of non-standardized tests to measure student learning. If the performance tests are not comparable, their results will not be comparable and, therefore, would jeopardize the analyzes carried out based on them. In the specific case of this study, when choosing to measure the impact of PARFOR on IDEB calculated for schools, the comparability between the indicators generated both in and between schools was considered over time. Roughly speaking, the IDEB value considers the performance averages of students in the 4th grade/5th year of elementary school at Prova Brasil, in relation to the average time it takes students to complete that series/year, by reference unit (Fernandes, 2007FERNANDES, Reynaldo. Índice de Desenvolvimento da Educação Básica (Ideb). Brasília: Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira, 2007. (Série Documental. Textos para Discussão 26).). Prova Brasil considers student performance as measured by techniques such as Item Response Theory (IRT), this allows the degree of difficulty between tests applied over time and between different groups to remain comparable and therefore, it contributes to the validity of the analyzes presented in this article.

Finally, it is worth noting that, in 2015, AERA, representing the educational researchers associated with it, published a statement that analyzed the context of the use of statistical models to ascertain the relationship between the quality of teacher training and students’ school performance (AERA, 2015AERA. Statement on Use of Value-Added Models (VAM) for the Evaluation of Educators and Educator Preparation Programs. Educational Researcher, Washington, v. 44, n. 8, p. 448-452, nov. 2015.). This statement considered the main psychometric problems involved, as well as the validity of inferences made based on them, in view of the challenge of isolating the contributions of teachers and school leaders from many other factors that shape student learning. In summary, this statement specified eight technical requirements that must be met for such measurements to be accurate, reliable, and valid, namely:

  • (1) The students’ performance measures used to explain the influence (or not) of the training program in this result must be anchored in assessments that meet the professional standards of reliability and validity.

  • (2) The measurements made must treat each interpretive argument separately (teachers, leaders, programs, schools), constituting valid and reliable evidence for each statement and interpretation made based on them.

  • (3) Measurements should be based on several years of data from a sufficient number of students.

  • (4) Measurements should only be calculated from test scores, which need to be comparable over time.

  • (5) Measurements should not be calculated based on grades or subjects where there are no standardized assessments accompanied by evidence of their reliability and validity.

  • (6) Measurements should never be used alone or in isolation in educator assessment systems or programs.

  • (7) Evaluation systems that use statistical-based measurements of causal relationships between training programs and student academic performance should include continuous monitoring of technical quality and validity of use.

  • (8) Assessment reports and determinations based on statistical measurements of causality should include estimates of error associated with student performance.

In the case of this impact assessment of PARFOR and the proxies indicators considered here, we sought to comply with such recommendations since the analysis developed is based on reliable and valid measures to determine the performance of students and their measurement for the school unit, as well as the possibility of comparing the evolution of the number of educators informed in the School Census and the IDEB, calculated for the school over time. In addition, it sought to evaluate the program without isolating the results obtained only in a single variable, avoiding using the student’s learning in isolation, out of context of the school unit. Finally, to ensure the measurement of valid statistics, which included the probability of associated error, the differences in differences (DD) method was chosen, in order to minimize the complications of uncontrolled variables in the verified impact, as explained in detail in the section that follow.

Methodological Design

An impact analysis involves the need to establish experimental designs or, in this case of PARFOR, quasi-experimental designs, which search for to statistically control the differences between the results obtained in the group that suffered the action (treatment group) and the results presented by the group that did not suffer the action (control group). As explained by Schneider et al. (2007)SCHNEIDER, Barbara et al. Estimating Causal Effects Using Experimental and Observational Designs: a think tank white paper. Washington: American Educational Research Association, 2007., in these drawings, statistical techniques are used to make causal inferences about the effect caused by a given action, searching for removing the interference from other factors that can compete with the calculated effect. Thereby, the results achieved are always a probability that the effect has been achieved, not the effect itself. In order to clarify the impact of the training of pedagogos promoted by PARFOR on the performance of public elementary schools, it was decided to use the DD statistical method. This technique allows some of the concerns with measuring the impact of training programs to be overcome: it is the same schools, both in terms of control and treatment, that are observed at different times. In addition, it allows to circumvent biases arising from different contextual conditions, through the random choice of participating schools, as recommended by Yoon et al. (2007)YOON, Kwang Suk et al. Reviewing the Evidence on How Teacher Professional Development Affects Student Achievement. Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southwest. Issues & Answers, n. 33, 2007. 62 p. Disponível em: <https://files.eric.ed.gov/fulltext/ED498548.pdf>. Acesso em: 19 nov. 2019.
https://files.eric.ed.gov/fulltext/ED498...
.

The reduction of contextual biases is possible because the DD is based on the differences observed between the increase in the changes that occurred in the treatment group (schools that received the action of PARFOR/degree in Pedagogia) in relation to what occurred in the control group (schools that do not receive any action from PARFOR). Therefore, the trajectories of both groups are observed, projecting as the trajectory of the control group what would be the trajectory of the treatment group in the absence of the program, in order to adequately address the biases promoted by other factors that may intervene in the effect over time.

The first record in the use of DD is in the study by Card and Krueger (1994)CARD, David; KRUEGER, Alan. Minimum Wages and Employment: a case study of the fast food industry in New Jersey and Pennsylvania. The American Economic Review, Pittsburgh, v. 84, n. 4, p.772-793, set. 1994.. This methodology has been increasingly used in studies that seek to measure the impacts of public policies, notably those carried out abroad (Hanushek; Wöbmann, 2006HANUSHEK, Eric; WÖßMANN, Ludger. Does Educational Tracking Affect Performance and Inequality? Differences‐ in‐Differences Evidence Across Countries. The Economic Journal, Oxford, v. 116, n. 510, p. C63-C76, mar. 2006.; Heckman; Ichimura; Todd, 1997HECKMAN, James; ICHIMURA, Hidehiko; TODD, Petra. Maching as an Econometric Evaluation Estimator: evaluating a job training programme. The Review of Economic Studies, London, v. 64, n. 4, p. 605-654, out. 1997.; Dimick; Ryan, 2014DIMICK, Justin; RYAN, Andrew. Methods for Evaluating Changes in Health Care Policy: the difference-in-differences Approach. Journal of the American Medical Association, Chicago, v. 312, n. 22, p. 2401-2402, dez. 2014.; Solé-Ollé; Sorribas -Navarro, 2008SOLÉ-OLLÉ, Albert; SORRIBAS-NAVARRO, Pilar. The Effects of Partisan Alignment on the Allocation of Intergovernmental Transfers: differences-in-differences estimates for Spain. Journal of Public Economics, Amsterdam, v. 92, n. 12, p. 2302-2319, dez. 2008.), but also in Brazil (Firpo; Pieri, 2012FIRPO, Sérgio; PIERI, Renan. Avaliando os Efeitos da Introdução de Computadores em Escolas Públicas Brasileiras. Revista Brasileira de Inovação, Campinas, v. 11, n. especial, p. 153-190, jul. 2012.; Silva Júnior; Pedrosa; Silva, 2013SILVA JÚNIOR, Luiz Honorato da; PEDROSA, Beatriz Mesquita Jardim; SILVA, Márcio Francisco da. Avaliação dos Impactos do ICMS Socioambiental na Criação de Unidades de Conservação e Unidades de Tratamento de Resíduos Sólidos em Pernambuco: uma análise a partir do método de diferenças-em-diferenças. Revista Econômica do Nordeste, Fortaleza, v. 44, n. 2, p. 559-574, abr./jun. 2013.). DD is particularly indicated for cases in which there is no random selection of the components of the treatment group, there is heterogeneity between its components in relation to unobservable characteristics and there is information about the components of the control and treatment groups both in the previous period and in the period after the program.

Mathematically, DD performs the analysis of a double difference: first, between the results presented before and after the action, both for the control group and for the treatment group; and then, the difference between the two differences found to verify the net impact of the action on the treatment group, according to the expression y=α+βdD+βtT+βdtDT, illustrated in Figure 4.

Figure 4
Mathematical Model Differences in Differences (DD)

Definitions:

D defines the type of group. The value equal to 1 (treatment group). The value equal to 0 (control group).

T defines the time to which the measure refers. T = 0 for the moment before the action; T = 1 for the moment after the action.

α represents the “intercept” of the model, that is, the global average of the response in the control group before the application of the policy.

β is the coefficient generated by the regression model for each of the main points of the function, ascertained by calculating the difference in difference, namely:

(a) For the difference between the beginning and the end of the period in the control group: α+βα=βt

(b) For the difference between the beginning and end of the period in the treatment group: αβd+βt+βdtα+βd=βt+βdt

(c) For difference in difference: βt+βdtβt=βdt


Figure 4 describes that the first difference (a) refers to the control group. It is obtained by comparing the change found at the end of the period with that existing at the beginning of the evaluated period. Similarly, the change in the treatment group (b) is determined. The difference in difference (c), then, refers to the impact obtained by the action to be measured. This last difference refers to the impact that is sought, since the effect found in the treatment group is discounted from that effect identified in the control group.

For application of the DD method in the universe of PARFOR analysis, a database was created with information collected from various sources, as specified in Table 1.

Chart 1
Research Database

When assembling the research database, some treatments were adopted to ensure data consistency, such as: a) exclude from the Plataforma Freire data on teachers in formation, whose enrollment situation was empty (teachers enrolled, but not actually enrolled); b) discard all data related to distance courses registered on the Plataforma Freire, prioritizing the records coming from SisUAB for courses offered in this modality; and c) separate the records at SisUAB among those specific to PARFOR from the other courses, excluding from the analysis the records of bachelor’s students, sequential courses and technological courses. In order to identify the continuing education courses belonging to PARFOR, it was used as a rule to consider only areas8 8 Arts, Performing Arts, Arts and Media, Plastic Arts, Visual Arts, Biology, Educational Science, Sciences, Biological Sciences, Religious Sciences, Physical Sciences, Natural Sciences, Social Sciences, Education ..., Religious Studies, Studies Social, Philosophy, Physics, Teacher Education ..., Geography, Education Management, History, History of Culture, Informatics, Educational Informatics, Language ..., Sign Language, Linguistics ..., Literature, Mathematics ..., Music, Pedagogy, Psychology, Psychopedagogy, Chemistry, Public Health, Sociology, Theater, Education Technology and Theology. of knowledge directly linked to undergraduate degrees, in particular, and education in general, registered in the systems.

Once the records of teachers/students in training benefiting from PARFOR were identified, they were crossed with the data from the 2015 School Census and, subsequently, with the IDEB base (Elementary School). As a result of the crossings, two tables were set up. The first, containing 400,622 records of PARFOR students enrolled between 2009 and 2014, providing information about students, courses taken or in progress, as well as about the schools that these PARFOR students were serving as a teacher in 2015. The second, containing 26,918 school records listed in the 2015 School Census, for which the IDEB calculations of schools in the early years of elementary school, referring to the base years 2009 (before PARFOR) and 2015 (after PARFOR) were identified, as well as the number of teachers trained in the area of Pedagogia, according to data from the 2009 and 2015 School Census. In this second file, other totals referring to PARFOR were added, such as the number of graduates of PARFOR Education in each participating school, as well as the number of enrolled in initial and continuing education who acted as teachers. Based on this second file, schools were selected to form the treatment group (if they had at least one teacher trained in Pedagogia by PARFOR by 2014) and the control group (if they did not have a record of teachers enrolled in PARFOR classes, either initial or continuing training).

The treatment group was formed by all 1388 schools active in the 2015 school census that counted teachers trained in Pedagogia by PARFOR. These schools had between 1 and 10 teachers in this situation. The highest frequency found was that of a teacher qualified in PARFOR Pedagogia per school (of the 1388 schools in the treatment group, 1120 had only 1 teacher who completed the course in focus).

Once the treatment group was defined, another 1388 schools were randomly selected to compose the control group, as long as there were no teachers benefited by PARFOR in any of its modalities. The distribution of the schools selected in the research sample by geographic region is shown in Table 2.

Table 2
Distribution of the Research Sample by Region

Once the necessary data for the analysis was selected, the impact of PARFOR on public schools was calculated using the SAS Enterprise Guide (SAS) - statistical package of the Statistical Analysis System. For this purpose, the proc genmod and data in the long format were used (time - before/after in lines, duplicating the records under analysis). The DD estimate for the impact under analysis is provided in the section that follows.

Impact of PARFOR on Basic Education

The evaluation of the impact of the training offered by PARFOR to teachers of public Educação Básica networks, through Pedagogia courses in classroom and distance modalities, revealed that the Plan managed to increase, even if in a relatively small way, the number of educators working in the benefited schools. However, with regard to its impact on the performance of these same schools in IDEB, it was not possible to find any statistically valid relationship.

According to the data, the actions of PARFOR contributed positively to change the situation of lack of teachers with the minimum degree required by law to act in early childhood education and in the early years of elementary school. As can be seen in Table 3, the DD estimates indicated that the schools that participated in PARFOR have more educators around them than they would have if they did not participate in the Plan (estimate = 1.0893), with an approximate probability of only 0.1% (Pr> | z |) that this estimate is wrong.

Table 3
Effect of PARFOR on the Number of Pedagogos - 2009 vs. 2015

It is also important to note that the positive impact of PARFOR on the number of pedagogos working in schools is significant even in the context in which all public schools, participating or not in PARFOR, had an increase in the average number of licensed teachers in Pedagogia at their disposal. As can be seen in Figure 5, the schools in the control group left, in 2009, an average level of almost five pedagogos hired per school unit to slightly more than 10, in 2015. Among the schools benefited by PARFOR, the number of pedagogos increases from a level lower than the average of four pedagogos per school unit, in 2009, to the average of almost 11, in 2015.

Figure 5
Graphical Representation Effect of PARFOR on the Average Number of Pedagogos Working in Public Schools - 2009 vs. 2015

Thus, it is important to note that the evaluation carried out here revealed that PARFOR not only acted more directly in schools that, on average, had a greater shortage of educators, but also contributed to the result presented by these schools, in relation to the profile of their teaching staff, was very close to that presented by the schools in the control group.

As Oliveira and Passador (2019)OLIVEIRA, Lilian Ribeiro de; PASSADOR, Claudia Souza. Ensaio Teórico Sobre as Avaliações de Políticas Públicas. Cadernos EBAPE.BR, Rio de Janeiro, v. 17, n. 2, p. 324-337, jun. 2019. argue, the evidence presented by the impact assessment of public policies allows to separate what works from what does not, subsidizing future interventions on the same policy or program. In the case of the effect of PARFOR on the number of educators working in public schools, the evidence seems to point to a successful experience of the Plan, considering its objectives. However, despite the lack of similar studies that make it possible to compare the magnitude of the impact assessed, it is to be assumed that only 1 extra teacher in the benefited schools is little. On the one hand, the data from the School Census indicate that the lack of trained educators working in Early Childhood Education and Early Years of Elementary9 9 In 2015, just over 1,197,000 teachers working at these school levels were accounted for, of which only about 48% had degrees in Pedagogia (INEP, 2015). Education has not yet been overcome. On the other hand, the improvement in the teacher qualification profile promoted by PARFOR was not accompanied by an increase in the quality of the benefited schools.

According to data in Table 4, there is no evidence that PARFOR improves the quality of education. There was even a negative impact of -0.05 points on the IDEB score of the participating schools, but with a very high probability of error, greater than 40%. Therefore, without validity.

Table 4
Effect of PARFOR on IDEB 2009 vs. 2013

The data indicate that, apparently, PARFOR was not able to significantly interfere in the performance trend of schools participating in IDEB, an index that measures the quality of Educação Básica in Brazil. However, it is necessary to consider that, in addition to teacher training, other elements may contribute to the results achieved by schools in this index.

As pointed out by Ell et al. (2019)ELL, Fiona et al. Conceptualising the Impact of Initial Teacher Education. The Australian Educational Researcher, v. 46, n. 1, p. 177-200, mar. 2019., initial teacher education is related to student learning, but this relationship is affected by the intervention of other elements that make up the educational system. For the authors, student learning is the result of several overlapping complex systems, which may occur due to the knowledge and practices of the teachers, but also due to the educational policy that supports the teaching-learning process, as well as the students’ own biosocial characteristics.

In addition, the authors point out that, in the context of initial training, there is a need for adequate structures - such as generating knowledge through research, consultancy, etc. - to ensure that the appropriate theoretical elements and updated technical instruments are available to teachers and reverberate in student learning.

It is also necessary to consider that the Pedagogia courses offered by PARFOR are inserted in the context of deficiencies in the Brazilian Pedagogia courses. As pointed out by Gatti (2010)GATTI, Bernardete. Formação de Professores no Brasil: características e problemas. Educação & Sociedade, Campinas, v. 31, n. 113, p. 1355-1379, out./dez. 2010., there is a lack of training content directly related to teaching (how to teach), as well as those associated with the subjects to be taught in Educação Básica (what and why to teach). Mascarenhas and Franco (2017)MASCARENHAS, Aline Daiane Nunes; FRANCO, Maria Amélia Santoro. De Pedagogos a Professores: balanço de uma década das Diretrizes Curriculares dos Cursos de Pedagogia no Brasil. Revista Internacional de Formação de Professores, Itapetininga, v. 2, n. 1, p. 41-55, jan./mar. 2017. suggest that the expected workload for the Pedagogia course is insufficient to train with quality profiles of different professionals at the same time. In addition, it points to the need to restructure the curricular guidelines envisaged for the course with regard to the organization of two training paths: one focused on undergraduate courses, with a view to a dense training of teachers in the initial grades and early childhood education; and another, in the bachelor’s degree, for school administration, coordination and management.

Apparently, considering the contributions of Souza (2017)SOUZA, Valdinei Costa. Qualidade da formação de pedagogos na perspectiva da oferta do Parfor Presencial. Educação e Pesquisa, São Paulo, v. 43, n. 1, p. 82-96, jan./mar. 2017., PARFOR did not offer theoretical and technical instrumental support in order to guarantee a similar quality standard between the courses. In addition, PARFOR did not provide support for the structuring conditions of the educational system that have the potential to interfere with the final result of the course, such as: support for the in-service teacher training process, support for needy teachers in training, among others.

Thus, graphically, the difference in difference model for the impact of the training of pedagogos by PARFOR in IDEB - of the benefited elementary schools can be seen in Figure 6. It shows that the IDEB of schools that had teachers licensed in Pedagogia by PARFOR is, on average, lower than those in the control group and that both grew between 2009 and 2015. However, the survey data does not allow us to specify to what extent PARFOR contributed to the growth achieved. The only evidence that the data shows is that PARFOR, in the case of Pedagogia courses, acted more in schools with weaker IDEB, when compared to control.

Figure 6
Graphical Representation - Effect of PARFOR on the Average Calculated by IDEB - 2009 vs. 2015

Finally, it is worth noting that, from the point of view of the data available in this research, it is necessary to consider that, as reported in the methodological description of this analysis, most schools belonging to the treatment group had, in 2015, only one educator trained between 2009 and 2014 by PARFOR. This data draws attention to the possibility that the impact of PARFOR on the quality of schools has also been affected by the low frequency of trained and active teachers in each school. The available data did not allow controlling the influence of the dispersion of beneficiaries on the impact of PARFOR on IDEB (more than 80% of the schools in the treatment group had only one teacher participating in PARFOR and this imbalance affected the estimates). However, it is possible to assume that such a small number of teachers trained by the Plan in the framework of the participating schools, despite the interventions pointed out by Ell et al. (2019)ELL, Fiona et al. Conceptualising the Impact of Initial Teacher Education. The Australian Educational Researcher, v. 46, n. 1, p. 177-200, mar. 2019., was, in itself, insufficient to modify, in fact and according to the Plan, the quality of education measured by IDEB.

Final Considerations

The idea that the professional development of teachers is an essential mechanism to improve teaching and student performance is widely shared in the literature. However, there is no consensus on the best way to promote this development, nor on the best way to measure training experiences, when searching for evidence on which one’s work or not.

Inserted in this theoretical context, the present analysis sought to assess the existence of causality between the actions of PARFOR, the improvement of the profile of the teaching staff and the quality of public elementary schools. In addition, this study sought to contribute to the formulation of methodological research designs, using the DD statistical technique as a viable alternative to meet the recommendations of validity and reliability of the analyzes discussed in the literature of the area.

The results obtained revealed that the actions promoted by PARFOR guarantee, at least one more pedagogos working in public schools in the early years of elementary school, benefiting, contributing to the improvement of the qualification profile of teachers in action. However, when observing the impact of the Plan on the quality of these schools, the results presented by the DD were not conclusive, despite the expectation that public policies aimed at teacher training have a positive impact on student learning (Delors, 1997DELORS, Jacques (Org.). Educação, Tesouro a Descobrir. Relatório para a Unesco da Comissão Internacional sobre Educação para o século XXI. Paris; Brasília: UNESCO/Edições ASA/Cortez, 1997. (Edição brasileira).; OECD, 2011OCDE. Building a High-Quality Teaching Profession: lessons from around the world. Paris: OECD Publishing, 2011. 65 p., 2013OCDE. Learning standards, teaching standards, and standards for school principals: a comparative study. OECD Education Working Papers, Chile, Centre of Study for Policies and Practices in Education (CEPPE), n. 99, 19 dez. 2013., 2018SOUZA, Valdinei Costa. Qualidade da formação de pedagogos na perspectiva da oferta do Parfor Presencial. Educação e Pesquisa, São Paulo, v. 43, n. 1, p. 82-96, jan./mar. 2017.).

Therefore, it is important to highlight two information underlying these results. The first of them has to do with the fashion of the distribution of teachers trained by PARFOR among the benefited schools. The frequency that is most repeated in the treatment group is that of one teacher per school, allowing us to assume, as a hypothesis, that the reduced frequency of graduates per school has influenced the absence of a significant impact of PARFOR on the quality of schools. The data did not allow for the contrast between the group of schools that benefited with only one teacher, in relation to those that benefited with more than one teacher with a degree in PARFOR Pedagogia. Thus, it is recommended that the referred hypothesis can be tested in future research, previously designed to control the results of the training model according to the number of graduates. The data available in this study did not allow to carry out the suggested analysis.

The second information that the results bring and that deserves to be highlighted, refers to the numbers presented previously in Figure 3. According to these data, about 50% of PARFOR students trained between 2009 and 2014, supposedly teachers of basic education, do not appeared in the 2015 School Census. In addition to a worrying indicator for the effectiveness of a public policy that aims to train teachers in service, but whose students mostly do not figure in educational statistics as teachers, this information is critical for assessing the impact of PARFOR. According to the data collected, most of the teachers trained by PARFOR are outside the Educação Básica system, negatively affecting not only the results of the Plan in meeting the demand for training, but also the expectation of impact of teachers trained in the IDEB of schools, as already noted in the previous paragraph.

It is also worth noting, in the interpretation of these data, the perspective brought by Ell et al. (2019)ELL, Fiona et al. Conceptualising the Impact of Initial Teacher Education. The Australian Educational Researcher, v. 46, n. 1, p. 177-200, mar. 2019., that initial teacher training is related to student learning, but this relationship is affected by the intervening of other elements that make up the educational system. For the authors, student learning is the result of several overlapping complex systems, which may occur due to the knowledge and practices of the teachers, but also due to the educational policy that supports the teaching-learning process, as well as the biosocial characteristics of the students. students themselves. In addition, they emphasize that, in the context of initial training, there is a need for adequate structures - such as generating knowledge through research, consultancy, etc. - to ensure that the appropriate theoretical elements and updated technical instruments contribute to students’ learning.

It should also be noted that the issue of formative models needs to be better resolved in the development of PARFOR. Throughout the study, it was not possible to identify the specific characteristics of the Pedagogia courses offered by PARFOR that identified them in clusters. The diversity of training models, in the terms described by Souza (2017)SOUZA, Valdinei Costa. Qualidade da formação de pedagogos na perspectiva da oferta do Parfor Presencial. Educação e Pesquisa, São Paulo, v. 43, n. 1, p. 82-96, jan./mar. 2017., combined with the absence of information that would allow the courses to be classified according to their characteristics, made the intention of observing differences between PARFOR’s training models the impact of the Program on the quality of schools.

Thus, considering the data and information analyzed in this impact assessment, it can be considered, despite PARFOR’s contribution to improving the qualification profile of teachers working in public schools, that there is a need for adjustments in the implementation of the Plan. The objective of meeting the demand for training of teachers in public education networks, expressed in the Ordinance of creation of PARFOR and represented in this work by the proxy indicator number of qualified teachers, has been fulfilled. However, the improvement of the professional qualification profile of teachers was not accompanied in the terms found in this research, the improvement in the quality of education measured by IDEB.

  • 1
    Translation note: In Brazil, in comparison with the American model, Educação Básica refers to: Educação Infantil (Kindergarten), to anos iniciais do Ensino Fundamental (Elementary School), to anos finais do Ensino Fundamental (Middle School) and Ensino Médio (High School). Therefore, we translate Ensino Fundamental as Elementary and Middle Schools and we do not translate Educação Básica, keeping the use of the term in Portuguese.
  • 2
    Translation note: We do not translate Pedagogia as Pedagogy, keeping the term in Portuguese, considering that, in Brazil, the Pedagogia course allows the exercise of teaching in Educação Infantil and anos iniciais do Ensino Fundamental - in American system, the Bachelors or Master’s Degree in Early Childhood and Elementary Education is a training requirement for acting in Kindergarten and Elementary School. Following the same reasoning, we do not translate pedagogos as educators with diploma in teaching, also keeping the term in Portuguese.
  • 3
    Official quality indicator of Brazilian education by the National Education Plan (PNE), Law No. 13,005, of June 24, 2014 (Brazil, 2014BRASIL. Lei n° 13.005, de 24 de junho de 2014 (BRASIL, 2014). Plano Nacional de Educação. Diário Oficial [da] República Federativa do Brasil, Brasília, 25 jun. 2014. Disponível em: <http://www.in.gov.br>. Acesso em: 30 jun. 2019.
    http://www.in.gov.br...
    ).
  • 4
    Translation note: We prefer not to translate terms like PARFOR and IDEB and others that refer to Brazilian public policies. As the text explains what they are, we therefore chose to allow the reader to understand their meaning in the Brazilian context, and not by inference from English words that do not have the same meaning. Therefore, we do not translate, but we emphasize in italics: Plano Nacional de Formação de Professores da Educação Básica (PARFOR), Índice de Desenvolvimento da Educação Básica (IDEB) e Política Nacional de Formação de Profissionais do Magistério da Educação Básica.
  • 5
    Decree No. 6,755, of January 29, 2009 (Brazil, 2009aBRASIL. Ministério da Educação. Conselho Nacional de Educação. Resolução CNE/CP nº 1, de 18 de fevereiro de 2002, que institui Diretrizes Curriculares Nacionais para a Formação de Professores da Educação Básica, em nível superior, curso de licenciatura, de graduação plena. Diário Oficial [da] República Federativa do Brasil, Brasília, DF, 19 fev. 2002. Disponível em: <http://portal.mec.gov.br/conselho-nacional-de-educacao/atos-normativos--sumulas-pareceres-e-resolucoes?id=12816>. Acesso em 8 out. 2019.
    http://portal.mec.gov.br/conselho-nacion...
    ).
  • 6
    Normative Ordinance No. 9, of June 30, 2009 (Brazil, 2009bBRASIL. Ministério da Educação. Conselho Nacional de Educação. Resolução CNE/CP nº 1, de 15 de maio de 2006, que institui Diretrizes Curriculares Nacionais para o Curso de Graduação em Pedagogia, licenciatura. Brasília, DF, 16 maio. 2006. Disponível em: <http://meclegis.mec.gov.br>. Acesso em 8 out. 2019.
    http://meclegis.mec.gov.br...
    ).
  • 7
    The proposal of the teaching base emerged from the movement of educators organized in the National Association for the Education Professionals Formation (Anfope), from the conception of teaching as a central element in the education of the pedagogue, with a view to overcoming the dichotomy between undergraduate and bachelor’s degrees , then identified. This proposal was incorporated into the Curricular Guidelines for the Pedagogy course by Resolution CNE / CP No. 1, of May 15, 2006 (Brazil, 2006BRASIL. Lei n° 9.394, de 20 de dezembro de 1996. Institui as diretrizes e bases da educação nacional. Diário Oficial [da] República Federativa do Brasil, Brasília, DF, 20 dez. 1996. Disponível em <http://www.planalto.gov.br>. Acesso em: 30 jun. 2019.
    http://www.planalto.gov.br...
    ).
  • 8
    Arts, Performing Arts, Arts and Media, Plastic Arts, Visual Arts, Biology, Educational Science, Sciences, Biological Sciences, Religious Sciences, Physical Sciences, Natural Sciences, Social Sciences, Education ..., Religious Studies, Studies Social, Philosophy, Physics, Teacher Education ..., Geography, Education Management, History, History of Culture, Informatics, Educational Informatics, Language ..., Sign Language, Linguistics ..., Literature, Mathematics ..., Music, Pedagogy, Psychology, Psychopedagogy, Chemistry, Public Health, Sociology, Theater, Education Technology and Theology.
  • 9
    In 2015, just over 1,197,000 teachers working at these school levels were accounted for, of which only about 48% had degrees in Pedagogia (INEP, 2015INEP. Microdados do Censo Escolar 2015. Brasília, 2015. Disponível em: <http://www.inep.gov.br>. Acesso em: 20 jun. 2020
    http://www.inep.gov.br...
    ).
  • Translated by Sabrina Mendonça Ferreira

Referências

  • AERA. Statement on Use of Value-Added Models (VAM) for the Evaluation of Educators and Educator Preparation Programs. Educational Researcher, Washington, v. 44, n. 8, p. 448-452, nov. 2015.
  • ARAÚJO, Viviane Patricia Colloca. A Multiculturalidade nas Políticas Educacionais e a Formação de Professores: Brasil e Portugal. 2009. 436 f. Tese (Doutorado em Educação) -. Universidade Federal de São Carlos, São Carlos, 2009.
  • BOYD, Donald et al. Complex by Design: investigating pathways into teaching in New York city schools. Journal of Teacher Education, v. 57, n. 2, p. 155-166, mar./abr. 2006.
  • BRASIL. Lei n° 9.394, de 20 de dezembro de 1996. Institui as diretrizes e bases da educação nacional. Diário Oficial [da] República Federativa do Brasil, Brasília, DF, 20 dez. 1996. Disponível em <http://www.planalto.gov.br>. Acesso em: 30 jun. 2019.
    » http://www.planalto.gov.br
  • BRASIL. Ministério da Educação. Conselho Nacional de Educação. Resolução CNE/CP nº 1, de 18 de fevereiro de 2002, que institui Diretrizes Curriculares Nacionais para a Formação de Professores da Educação Básica, em nível superior, curso de licenciatura, de graduação plena. Diário Oficial [da] República Federativa do Brasil, Brasília, DF, 19 fev. 2002. Disponível em: <http://portal.mec.gov.br/conselho-nacional-de-educacao/atos-normativos--sumulas-pareceres-e-resolucoes?id=12816>. Acesso em 8 out. 2019.
    » http://portal.mec.gov.br/conselho-nacional-de-educacao/atos-normativos--sumulas-pareceres-e-resolucoes?id=12816
  • BRASIL. Ministério da Educação. Conselho Nacional de Educação. Resolução CNE/CP nº 1, de 15 de maio de 2006, que institui Diretrizes Curriculares Nacionais para o Curso de Graduação em Pedagogia, licenciatura. Brasília, DF, 16 maio. 2006. Disponível em: <http://meclegis.mec.gov.br>. Acesso em 8 out. 2019.
    » http://meclegis.mec.gov.br
  • BRASIL. Decreto n° 6.755, de 29 de janeiro de 2009. Institui a Política Nacional de Formação de Profissionais do Magistério da Educação Básica, disciplina a atuação da Capes no fomento a programas de formação inicial e continuada e dá outras providências. Diário Oficial [da] República Federativa do Brasil, Brasília, DF, 30 jan. 2009a. Disponível em: <http://www.planalto.gov.br>. Acesso em: 30 jun. 2019.
    » http://www.planalto.gov.br
  • BRASIL. Ministério da Educação. Portaria Normativa nº 9, de 30 de junho de 2009. Institui o Plano Nacional de Formação dos Professores da Educação Básica no âmbito do Ministério da Educação. Diário Oficial [da] República Federativa do Brasil, Brasília, DF, 01 jul. 2009b. Disponível em: <http://meclegis.mec.gov.br>. Acesso em: 30 jun. 2011.
    » http://meclegis.mec.gov.br
  • BRASIL. Lei n° 13.005, de 24 de junho de 2014 (BRASIL, 2014). Plano Nacional de Educação. Diário Oficial [da] República Federativa do Brasil, Brasília, 25 jun. 2014. Disponível em: <http://www.in.gov.br>. Acesso em: 30 jun. 2019.
    » http://www.in.gov.br
  • BRASIL. Ministério da Educação. Conselho Nacional de Educação. Resolução CNE/CP nº 2, de 1º de julho de 2015, que define as Diretrizes Curriculares Nacionais para a formação inicial em nível superior (cursos de licenciatura, cursos de formação pedagógica para graduados e cursos de segunda licenciatura) e para a formação continuada. Diário Oficial [da] República Federativa do Brasil, Brasília, DF, 02 jul. 2015. Disponível em: <http://portal.mec.gov.br/conselho-nacional-de-educacao/atos-normativos--sumulas-pareceres-e-resolucoes?id=12816>. Acesso em 8 out. 2019.
    » http://portal.mec.gov.br/conselho-nacional-de-educacao/atos-normativos--sumulas-pareceres-e-resolucoes?id=12816
  • BRASIL. Decreto nº 8.752, de 9 de maio de 2016. Revoga o Decreto 6.755/2009 e dispõe sobre a Política Nacional de Formação dos Profissionais da Educação Básica. Diário Oficial [da] República Federativa do Brasil, Brasília, DF, 10 mai. 2016. Disponível em: <http://www.planalto.gov.br >. Acesso em: 30 jun. 2019.
    » http://www.planalto.gov.br
  • BRASIL. Ministério da Educação. Conselho Nacional de Educação. Resolução CNE/CP nº 2, de 20 de dezembro de 2019 - Define as Diretrizes Curriculares Nacionais para a Formação Inicial de Professores para a Educação Básica e institui a Base Nacional Comum para a Formação Inicial de Professores da Educação Básica (BNC-Formação). Diário Oficial [da] República Federativa do Brasil, Brasília, DF, 20 dez. 2019. Disponível em: <http://portal.mec.gov.br/conselho-nacional-de-educacao/atos-normativos--sumulas-pareceres-e-resolucoes?id=12816>. Acesso em: 2 fev. 2020.
    » http://portal.mec.gov.br/conselho-nacional-de-educacao/atos-normativos--sumulas-pareceres-e-resolucoes?id=12816
  • CARD, David; KRUEGER, Alan. Minimum Wages and Employment: a case study of the fast food industry in New Jersey and Pennsylvania. The American Economic Review, Pittsburgh, v. 84, n. 4, p.772-793, set. 1994.
  • COCHRAN-SMITH, Marilyn; ZEICHNER, Kenneth (Org.). Studying Teacher Education: the report of the AERA panel on research and teacher education. Mahwah: Lawrence Erlbaum Associates, 2005.
  • COCHRAN-SMITH, Marilyn et al. Holding Teacher Preparation Accountable: a review of claims and evidence. Boulder: National Education Policy Center, 2016. Disponível em: <https://nepc.colorado.edu/sites/default/files/pb_cochran-smith_teacher_prep_0.pdf>. Acesso em: 19 nov. 2019.
    » https://nepc.colorado.edu/sites/default/files/pb_cochran-smith_teacher_prep_0.pdf
  • DARLING-HAMMOND, Linda; YOUNGS, Peter. Defining ‘highly qualified teachers’: what does ‘scientifically-based research’ actually tell us. Educational Researcher, Washington, v. 31, n. 9, p. 13-25, dez. 2002.
  • DELORS, Jacques (Org.). Educação, Tesouro a Descobrir Relatório para a Unesco da Comissão Internacional sobre Educação para o século XXI. Paris; Brasília: UNESCO/Edições ASA/Cortez, 1997. (Edição brasileira).
  • DIMICK, Justin; RYAN, Andrew. Methods for Evaluating Changes in Health Care Policy: the difference-in-differences Approach. Journal of the American Medical Association, Chicago, v. 312, n. 22, p. 2401-2402, dez. 2014.
  • ELL, Fiona et al. Conceptualising the Impact of Initial Teacher Education. The Australian Educational Researcher, v. 46, n. 1, p. 177-200, mar. 2019.
  • FERNANDES, Reynaldo. Índice de Desenvolvimento da Educação Básica (Ideb) Brasília: Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira, 2007. (Série Documental. Textos para Discussão 26).
  • FIRPO, Sérgio; PIERI, Renan. Avaliando os Efeitos da Introdução de Computadores em Escolas Públicas Brasileiras. Revista Brasileira de Inovação, Campinas, v. 11, n. especial, p. 153-190, jul. 2012.
  • FRANCO, Maria Amélia Santoro; LIBÂNEO, José Carlos; PIMENTA, Selma Garrido. Elementos para a Formulação de Diretrizes Curriculares para Cursos de Pedagogia. Cadernos de Pesquisa, São Paulo, v. 37, n. 130, p. 63-97, jan./abr. 2007.
  • GATTI, Bernardete. Formação de Professores no Brasil: características e problemas. Educação & Sociedade, Campinas, v. 31, n. 113, p. 1355-1379, out./dez. 2010.
  • GOLDHABER, Dan. What Do Value-Added Measures of Teacher Preparation Programs Tell Us? carnegie foundation for the advancement of teaching, Stanford, CA: Carnegie Knowledge Network. What We Know Series: Value-Added Methods and Applications, p. 14, nov. 2013. Disponnível em: <http://www.carnegieknowledgenetwork.org/wp-content/uploads/2013/11/CKN-Goldhaber-TeacherPrep_Final_11.7.pdf>. Acesso em: 01 jun. 2020.
    » http://www.carnegieknowledgenetwork.org/wp-content/uploads/2013/11/CKN-Goldhaber-TeacherPrep_Final_11.7.pdf
  • HANUSHEK, Eric; WÖßMANN, Ludger. Does Educational Tracking Affect Performance and Inequality? Differences‐ in‐Differences Evidence Across Countries. The Economic Journal, Oxford, v. 116, n. 510, p. C63-C76, mar. 2006.
  • HECKMAN, James; ICHIMURA, Hidehiko; TODD, Petra. Maching as an Econometric Evaluation Estimator: evaluating a job training programme. The Review of Economic Studies, London, v. 64, n. 4, p. 605-654, out. 1997.
  • INEP. Sinopse Estatística do Professor da Educação Básica 2009 Brasília, 2009. Disponível em: <http://www.inep.gov.br>. Acesso em: 20 jun. 2020.
    » http://www.inep.gov.br
  • INEP. Microdados do Censo Escolar 2015 Brasília, 2015. Disponível em: <http://www.inep.gov.br>. Acesso em: 20 jun. 2020
    » http://www.inep.gov.br
  • LIBÂNEO, José Carlos. Reflexividade e Formação de Professores: outras oscilações do pensamento pedagógico brasileiro? In: PIMENTA, Selma Garrido; GHEDIN, Evandro (Org.). Professor Reflexivo no Brasil: gênese e crítica de um conceito. São Paulo: Cortez, 2008. P. 53-79.
  • MASCARENHAS, Aline Daiane Nunes; FRANCO, Maria Amélia Santoro. De Pedagogos a Professores: balanço de uma década das Diretrizes Curriculares dos Cursos de Pedagogia no Brasil. Revista Internacional de Formação de Professores, Itapetininga, v. 2, n. 1, p. 41-55, jan./mar. 2017.
  • MIHALY, Kata et al. Where You Come from or Where You Go? Distinguishing Between School Quality and the Effectiveness of Teacher Preparation Program Graduates. Center for Analysis of Longitudinal Data in Education Research, Washington, D.C. Working Paper 63, 41 p, jan. 2012 Disponível em: <https://files.eric.ed.gov/fulltext/ED529178.pdf>. Acesso em: 01 jun. 2020.
    » https://files.eric.ed.gov/fulltext/ED529178.pdf
  • OCDE. Building a High-Quality Teaching Profession: lessons from around the world. Paris: OECD Publishing, 2011. 65 p.
  • OCDE. Learning standards, teaching standards, and standards for school principals: a comparative study. OECD Education Working Papers, Chile, Centre of Study for Policies and Practices in Education (CEPPE), n. 99, 19 dez. 2013.
  • OCDE. Effective Teacher Policies: insights from Pisa. Paris: OECD Publishing, 2018, 163 p.
  • OLIVEIRA, Lilian Ribeiro de; PASSADOR, Claudia Souza. Ensaio Teórico Sobre as Avaliações de Políticas Públicas. Cadernos EBAPE.BR, Rio de Janeiro, v. 17, n. 2, p. 324-337, jun. 2019.
  • RIOS, Terezinha. Compreender e Ensinar: por uma docência da melhor qualidade. 8. ed. São Paulo: Cortez, 2010.
  • ROWAN, Leonie et al. Investigating the Effectiveness of Teacher Education for Early Career Teachers in Diverse Settings: the longitudinal research we have to have. The Australian Educational Researcher, v. 42, n. 3, p. 273-298, jul. 2015.
  • SCHEIBE, Leda. Diretrizes Curriculares para o Curso de Pedagogia: trajetória longa e inconclusa. Cadernos de Pesquisa, v. 37, n. 130, p. 43-62, 2007.
  • SCHNEIDER, Barbara et al. Estimating Causal Effects Using Experimental and Observational Designs: a think tank white paper. Washington: American Educational Research Association, 2007.
  • SILVA JÚNIOR, Luiz Honorato da; PEDROSA, Beatriz Mesquita Jardim; SILVA, Márcio Francisco da. Avaliação dos Impactos do ICMS Socioambiental na Criação de Unidades de Conservação e Unidades de Tratamento de Resíduos Sólidos em Pernambuco: uma análise a partir do método de diferenças-em-diferenças. Revista Econômica do Nordeste, Fortaleza, v. 44, n. 2, p. 559-574, abr./jun. 2013.
  • SLEETER, Christine. Toward Teacher Education Research that Informs Policy. Educational Researcher, Washington, v. 43 n. 3, p. 146-153, abr. 2014.
  • SOLÉ-OLLÉ, Albert; SORRIBAS-NAVARRO, Pilar. The Effects of Partisan Alignment on the Allocation of Intergovernmental Transfers: differences-in-differences estimates for Spain. Journal of Public Economics, Amsterdam, v. 92, n. 12, p. 2302-2319, dez. 2008.
  • SOUZA, Valdinei Costa. Qualidade da formação de pedagogos na perspectiva da oferta do Parfor Presencial. Educação e Pesquisa, São Paulo, v. 43, n. 1, p. 82-96, jan./mar. 2017.
  • YOON, Kwang Suk et al. Reviewing the Evidence on How Teacher Professional Development Affects Student Achievement Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southwest. Issues & Answers, n. 33, 2007. 62 p. Disponível em: <https://files.eric.ed.gov/fulltext/ED498548.pdf>. Acesso em: 19 nov. 2019.
    » https://files.eric.ed.gov/fulltext/ED498548.pdf

Edited by

Editor-in-charge: Beatriz Vargas Dorneles

Publication Dates

  • Publication in this collection
    09 June 2021
  • Date of issue
    2021

History

  • Received
    12 Aug 2020
  • Accepted
    05 Feb 2021
Universidade Federal do Rio Grande do Sul - Faculdade de Educação Avenida Paulo Gama, s/n, Faculdade de Educação - Prédio 12201 - Sala 914, 90046-900 Porto Alegre/RS – Brasil, Tel.: (55 51) 3308-3268, Fax: (55 51) 3308-3985 - Porto Alegre - RS - Brazil
E-mail: educreal@ufrgs.br