AEFP 45th Annual Conference

Toward a Meaningful Impact through Research, Policy & Practice

March 19-21, 2020

Which Teachers Supplement Their Core Curriculum and Why?

Presenter: 
Daniel Silver, USC, dsilver@usc.edu

The vast majority of core subject area teachers in U.S. schools are given access to curricular materials such as textbooks to use in their instruction. The content of these materials affects how teachers select and refine activities for their students (Remillard, 2005), and experimental evidence has shown that variation in textbook content can affect students’ academic achievement outcomes (Agodini, et al., 2010). (This finding has been confirmed in several quasi-experimental studies (Bhatt & Koedel, 2012; Bhatt, Koedel, & Lehmann, 2013), though the largest and most recent study (Blazar, et al., 2019) found no differences in effectiveness among common mathematics textbooks). Despite this, recent education survey data show that teachers frequently supplement their district-adopted textbooks with unofficial curriculum materials, typically either self-created or downloaded from the Internet (Blazar, et al., 2019; Opfer, Kaufman, and Thompson, 2016). While we know that supplementation is widespread, we do not know much about who supplements and why.
If teachers supplement evenly across the board, perhaps they simply do not feel prepared to use their officially-adopted curricula, suggesting a need for better curriculum training in general. If, however, teachers in some contexts supplement more than those in other contexts, this could signal systematic variation in curriculum quality that would merit further attention or support. In this paper, we analyze the National Evaluation of Curriculum Effectiveness (NECE) from the Harvard Center for Education Policy Research, a representative six-state 2017 survey of 4th and 5th grade math teachers, to answer the following research questions:
1. In what ways do 4th and 5th grade math teachers vary in their supplementation of officially-provided curriculum materials?
2. What school-level, teacher-level, and student-level characteristics predict greater levels of supplementation?
The NECE survey sheds light on supplementation (e.g., “In what percentage of your lessons did you use materials you created yourself or with colleagues at your school?”) and provides school-level, teacher-level, and student-level characteristics (including, for example, hours of math instruction per week, hours of subject- and textbook-specific professional development in the last year, years of experience with this textbook, teacher attitudes toward state standards, teacher’s pathway to certification, student access to technology inside and outside school, etc.). Each survey response is linked to a school ID, so we are also able to connect teachers’ responses to their school contexts using the NCES Common Core of Data.
To address our first research question, we analyze the data on the various supplementation behaviors available in the survey by first creating survey scales to capture each of the distinct forms of supplementation. By measuring supplementation along multiple distinct dimensions, we assert that researchers must be mindful of how supplementation is measured, since a single measure of supplementation yields an incomplete picture of how teachers might supplement their officially-adopted curriculum, in practice. Next, we calculate descriptive statistics for each dimension of supplementation, reporting the results overall and by state, as well as by teachers’ officially-adopted textbooks. Finally, we partition the variance in supplementation within and between districts using intraclass correlation coefficients. Our analyses test the degree to which supplementation behaviors vary across teachers both within and between school districts, to ascertain whether there is a district-level component to these behaviors.
To address our second research question, we then use contextual factors from the survey and the NCES Common Core of Data to build multilevel models (teachers nested within districts) that predict the extent to which teachers supplement and highlight which factors have the greatest predictive value for which forms of supplementation. Factors in the model include, but are not limited to: which textbook the district provides, number of other ELA teachers in the grade, teacher perception of the quality of their official curriculum materials, and proportion of students of color, who are English learners, who have IEPs, and who qualify for free and reduced lunch.
This study contributes to a still-developing literature on supplementation; in fact, we know of no other studies that use school- and student-level factors to predict supplementation. Our findings may hold implications for policymakers and practitioners, since evidence of systematic variation in supplementation implies systematic variation in curriculum quality, at least in teachers’ eyes. Depending on the exact patterns of this variation, such findings could carry equity or efficiency implications around, for example, district criteria for curriculum adoption, school lesson-planning expectations, or textbook-specific professional development.

Poster: 

Comments

Very clear description of the data sources and methodological approach. The results (graphic/table + text box) makes it very easy to follow. I'm not sure what data is available to you, but think it would be helpful to have a sense of the extent to which teachers are expected to adhere to pacing guides, which might constrain their use of supplemental materials. Given your results for RQ3, I'm wondering about the relationship between students' achievement and supplementation. Is it possible that teachers of classes with higher average achievement supplement more (for enrichment purposes)? Does the amount of heterogeneity in achievement within classes play a role as well? Teachers might feel more need to supplement the curriculum for both enrichment and remediation activities if they are teaching students with a wide range of achievement. Thank you for sharing your work! If you have any questions about my comments, you can reach me at carajed@gmail.com.

Thank you for presenting this succinct exploration of this data set. I work primarily in the school improvement domain and this is a topic that greatly interests me. I am especially interested in the findings related to enrichment versus remediation. Do you have any thoughts about these findings? From my vantage point, it makes sense to me that teachers are adding enrichment activities at a statistically significant rate as opposed to remediation. It seems to me that many many curriculum products are designed with remediation in mind but fewer focus on enrichment activities. Do you have any existing literature that examines this phenomenon? Matthew Courtney matthew.courtney@education.ky.gov

I was interested to see the data on teachers' reasons for taking different jobs. I'm unsurprised broadly about the importance of geographic location--I wonder how cost of living plays into decisions to teach in certain districts within relatively close proximity to each other, and whether those considerations might be subsumed in the category of geographic preferences. For context, I'm in Davis, CA where we have pressures on recruiting teachers due to a substantially higher cost of living than in relatively close-by towns. I wonder how differences in cost of living in towns that are relatively close together plays into the geographic preferences that teachers hold.

I had two tabs open and left this comment on the wrong poster.

Add new comment