AEFP 45th Annual Conference

Toward a Meaningful Impact through Research, Policy & Practice

March 19-21, 2020

Do resources matter? Generating a production function to estimate district-level performance on national assessments

Presenter: 
Samuel Correa, American Institutes for Research, scorrea@air.org

There is a rich literature interested in evaluating whether resource expenditure in education is connected to the performance of students; despite the wealth of research, there is far from a consensus on the answer to this question (see Baker, 2017; Hanushek, 1989; Greenwald et al., 1996). Since the 1966 report Equality of Educational Opportunity (also known as the Coleman Report) provided an innovative input-output approach to evaluating schools, concluding that socioeconomic factors are strongly related to student achievement (Coleman et al., 1966), many research approaches have explored the extent to which student performance can be explained in terms of education resource availability. Furthermore, as schools evolve to meet modern day needs, so too do their funding structures and financial needs. As such, this field of research continually needs to be refreshed and reassessed to respond to the changing education finance landscape. This study intends to contribute to the literature by looking at recent national-level performance data for an association between education spending and student achievement.
The current study takes a national approach to evaluating this question at the K-12 level, using districts (also referred to as local education agencies (LEA)) as the unit of measurement. Employing multiple-regression analysis and a production-function oriented theoretical model, this study attempts to provide a nationally representative perspective; adding to the literature by evaluating whether the amount of resources spent on education are indicative of district-level performance.
This study uses data from the 2015 National Assessment of Education Progress (NAEP) as the basis for evaluating performance. The NAEP dataset is fitting for a national analysis perspective because it is the largest nationally representative assessment of public and private schools in the U.S. Instead of assessing individual student level data for achievement, NAEP is designed to provide a common measure of student achievement for all participating schools. These common benchmarks provided by NAEP data then allow for a country-wide perspective of the effects of spending on performance in schools.
To understand funding in districts, this study uses the 2015 School District Finance Survey (also known as the F-33). This universe survey is a component of the Common Core of Data, the primary National Center for Education Statistics database for public elementary and secondary education in the U.S. The F-33 consists of district-level finance data that is submitted annually to NCES, containing detailed breakdowns of a districts’ revenues and expenditures (Cornman, 2017). This study utilizes this financial data by standardizing the dollar amounts in two ways: first, by constructing a per-pupil measure of expenditures and revenues in the district; second by evaluating expenditures on selected aspects of the district (such as instruction or support services) as a proportion of the district’s total expenditures. Additionally, dollar amounts are standardized for geographic wage variations using a Comparable Wage Index for Teachers (Cornman, 2017).
In order to control for aspects of the model outside of funding that could influence performance, factors known to have impacts on student achievement are implemented into the model. These factors include a principal component measure of socio-economic status factors (Fahle et al., 2019); a district size control to account for the economies of scale within a district; and a measure of district additional-need factors using the percentage of students who are enrolled in Special Education or English Language Learner programs.
Preliminary results indicate that after controlling for socio-economic status factors, the absolute level of per-pupil spending was not significantly related to district performance, however there is evidence that the proportion of a district’s spending on instruction has a positive effect on district-level student achievement. Further analysis will be conducted to discern the effects of funding on district performance within subsets of the K-12 population (e.g., different levels of socio-economic status, population density levels, and levels of district performance). In addition to this exploratory analysis, the current study explores potential options for future research using this linked dataset via advanced multi-level models and clustering techniques.

Comments

I think you identify a good set of limitations and next steps. A model that accounts for cross-state variance would definitely be an improvement--either with multi-level modeling or with state fixed effects. Are the wage variance controls included in the regressions presented? I see a lot of the other control variables you mention with coefficients presented, but not that one. Or are those embedded in the expenditures somehow (i.e., used to scale them differently across regions)? Thanks for sharing this work!

Have you thought about directly controlling for district staffing levels? Doing that seems more in line with the production function framework than does using expenditures as a catch-all for inputs? And whether you control for staffing levels or expenditures, I'm concerned that those inputs are statistically endogenous. For example, there can be unobserved attributes of districts that reduce scores and increase spending. Multi-level modeling and state fixed effects will help, but thinking about potential instruments is also useful. Finally, since you are observing mutltiple students within districts, be sure to cluster your standard errors.

This is an interesting question! I echo the other comments here--it's a great first step at looking at an important question. I wonder, given the new ESSA requirements that per pupil expenditures be reported at the school level, there's a way to account for variation in resources within a district. Resources are not distributed uniformly across schools, and that might affect your findings and/or conclusion as well. Thanks for sharing your work! - Dara Shaw (dara.shaw@maryland.gov)

Add new comment