Education Finance and Policy - Volume 16, Issue 2, Spring 2021
Testing, Stress, and Performance: How Students Respond Physiologically to High-Stakes Testing. Jennifer A. Heissel, Emma K. Adam, Jennifer L. Doleac, David N. Figlio, Jonathan Meer. Education Finance and Policy (2021) 16 (2): 183–208.
We examine how students’ physiological stress differs between a regular school week and a high-stakes testing week, and we raise questions about how to interpret high-stakes test scores. A potential contributor to socioeconomic disparities in academic performance is the difference in the level of stress experienced by students outside of school. Chronic stress—due to neighborhood violence, poverty, or family instability—can affect how individuals’ bodies respond to stressors in general, including the stress of standardized testing. This, in turn, can affect whether performance on standardized tests is a valid measure of students’ actual ability. We collect data on students’ stress responses using cortisol samples provided by low-income students in New Orleans. We measure how their cortisol patterns change during high-stakes testing weeks relative to baseline weeks. We find that high-stakes testing is related to cortisol responses, and those responses are related to test performance. Those who responded most strongly, with either increases or decreases in cortisol, scored 0.40 standard deviations lower than expected on the high-stakes exam.
Too Little or Too Much? Actionable Advice in an Early-Childhood Text Messaging Experiment. Kalena E. Cortes, Hans Fricke, Susanna Loeb, David S. Song, Benjamin N. York. Education Finance and Policy (2021) 16 (2): 209–232.
Text-message-based parenting programs have proven successful in improving parent engagement and preschoolers’ literacy development. This study seeks to identify mechanisms of the overall effect of such programs. It investigates whether actionable advice alone drives previous studies’ results and whether additional texts of actionable advice improve program effectiveness. The findings provide evidence that text messaging programs can supply too little or too much information. A single text per week is not as effective at improving parenting practices as a set of three texts that also include information and encouragement, but a set of five texts with additional actionable advice is also not as effective as the three-text approach. The results on children's literacy development depend on the child's pre-intervention literacy skills. For children in the lowest quarter of the pretreatment literacy assessments, providing one example of an activity improves literacy scores by 0.19 standard deviations less than providing three texts. Literacy scores of children in higher quarters are marginally higher with only one tip per week than with three tips per week. We find no positive effects of increasing to five texts per week.
How College Credit in High School Impacts Postsecondary Course-Taking: The Role of Advanced Placement Exams. Oded Gurantz. Education Finance and Policy (2021) 16 (2): 233–255.
This paper uses Advanced Placement (AP) exams to examine how receiving college credit in high school alters students’ subsequent human capital investment. Using data from one large state, I link high school students to postsecondary transcripts from in-state, public institutions. I estimate causal impacts using a regression discontinuity that compares students with essentially identical AP performance but who receive different offers of college credit. I find that female students who earn credit from science, technology, engineering, and mathematics (STEM) exams take higher level STEM courses, significantly increasing their depth of study, with no observed impacts for male students. As a result, the male–female gap in STEM courses taken shrinks by roughly one third to two thirds, depending on the outcome studied. Earning non-STEM AP credit increases overall coursework in non-STEM courses and increases the breadth of study across departments. Early credit policies help assist colleges to produce graduates whose skills aligns with commonly cited social or economic priorities, such as developing STEM graduates with stronger skills, particularly among traditionally underrepresented groups.
High Bars or Behind Bars? The Effect of Graduation Requirements on Arrest Rates. Matthew F. Larsen. Education Finance and Policy (2021) 16 (2): 256–282.
This paper investigates the effect of high school graduation requirements on arrest rates with a specific focus on the number of required courses and the use of exit exams. Identifying variation comes from state-by-cohort changes in the laws governing high school graduation requirements from 1980 to 2010. Combining these law changes with arrest rates of young adults from the Federal Bureau of Investigation's Uniform Crime Reports, I find that the use of exit exams can reduce arrest rates by approximately 7 percent. Although it is difficult to parse out the exact mechanisms, additional exploration into heterogeneity by age and offense, as well as examination of labor market outcomes, suggest that these policies may have increased learning. Given the current debate around the use of exit exams, this paper provides evidence of beneficial effects on nonacademic outcomes. This paper also provides further evidence of the influence of education policy on crime.
Enacting the Rubric: Teacher Improvements in Windows of High-Stakes Observation. Aaron R. Phipps, Emily A. Wiseman. Education Finance and Policy (2021) 16 (2): 283–312.
Teacher evaluation systems that use in-class observations, particularly in high-stakes settings, are frequently understood as accountability systems intended as nonintrusive measures of teacher quality. Presumably, the evaluation system motivates teachers to improve their practice—an accountability mechanism—and provides actionable feedback for improvement—an information mechanism. No evidence exists, however, establishing the causal link between an evaluation program and daily teacher practices. Importantly, it is unknown how teachers may modify their practice in the time leading up to an unannounced in-class observation, or how they integrate feedback into their practice post-evaluation, a question that fundamentally changes the design and philosophy of teacher evaluation programs. We disentangle these two effects with a unique empirical strategy that exploits random variation in the timing of in-class observations in the Washington, DC, teacher evaluation program IMPACT. Our key finding is that teachers work to improve during periods in which they are more likely to be observed, and they improve with subsequent evaluations. We interpret this as evidence that both mechanisms are at work, and as a result, policy makers should seriously consider both when designing teacher evaluation systems.
Is Effective Teacher Evaluation Sustainable? Evidence from District of Columbia Public Schools. Thomas S. Dee, Jessalynn James, Jim Wyckoff. Education Finance and Policy (2021) 16 (2): 313–346.
Ten years ago, many policy makers viewed the reform of teacher evaluation as a highly promising mechanism to improve teacher effectiveness and student achievement. Recently, that enthusiasm has dimmed as the available evidence suggests the subsequent reforms had a mixed record of implementation and efficacy. Even in districts where there was evidence of efficacy, the early promise of teacher evaluation may not be sustainable as these systems mature and change. This study examines the evolving design of IMPACT, the teacher evaluation system in the District of Columbia Public Schools. We describe the recent changes to IMPACT, which include higher performance standards for lower-performing teachers and a reduced emphasis on value-added test scores. Descriptive evidence on the dynamics of teacher retention and performance under this redesigned system indicates that lower-performing teachers are particularly likely to either leave or improve. Corresponding causal evidence similarly indicates that imminent dismissal threats for persistently low-performing teachers increased both teacher attrition and the performance of returning teachers. These findings suggest that teacher evaluation can provide a sustained mechanism for improving the quality of teaching.
Principal Evaluation under the Elementary and Secondary Every Student Succeeds Act: A Comprehensive Policy Review. Morgaen Donaldson, Madeline Mavrogordato, Shaun M. Dougherty, Reem Al Ghanem, Peter Youngs. Education Finance and Policy (2021) 16 (2): 347–361.
A growing body of research recognizes the critical role of the school principal, demonstrating that school principals’ effects on student outcomes are second only to those of teachers. Yet policy makers have often paid little attention to principals, choosing instead to focus policy reform on teachers. In the last decade, this pattern has shifted somewhat. Federal policies such as Race to the Top (RTTT) and Elementary and Secondary Education Act waivers emphasized principal quality and prompted many states to overhaul principal evaluation as a means to develop principals’ leadership practices and hold them accountable for the performance of their schools. The development and dissemination of principal evaluation policies has proceeded rapidly, however, it is unclear whether focusing on principal evaluation has targeted the most impactful policy lever. In this policy brief, we describe where policy makers have placed their bets in post-RTTT principal evaluation systems and comment on the wisdom of these wagers. We describe the degree to which principal evaluation components, processes, and consequences vary across the fifty states and the District of Columbia, and review evidence on which aspects of principal evaluation policies are most likely to improve principals’ practice and hold them accountable.