Sector Differences During the Academic Year and Summer in Elementary School

Very little research on differences in student achievement focuses on elementary school students, and there is minimal focus on gains in academic achievement for elementary students during summer break.   Given this, Carbonaro tests for sector effects in elementary school, and investigates how those effects vary by summer learning, with particular interest in determining whether or not gains made by students over the summers during early school years are being incorrectly attributed to school.  That is, if sector differences in summer learning are observed, how do sector differences in summer learning affect our understanding of conventional estimates of sector effects?  In other words, are we overestimating the effect of Catholic, sectarian, or other religious schools by including summer learning in school effects research? 

Data Notes
To answer these questions, Carbonaro analyzes the first four waves of the Early Childhood Longitudinal Study (ECLS)--a nationally representative panel study of 21,000 students, their parents, and teachers conducted by the Department of Education's National Center for Educational Statistics.  Because ECLS administered achievement tests in both the fall and spring of kindergarten and first grades, this data set is particularly well suited to address questions about sector effects early in the educational pipeline, and whether summer learning differs by school sector. 

To measure academic improvement using a series of test scores, Carbonaro calculates "growth scores" by subtracting students' scores on a previous test from their scores on subsequent tests.  In these analyses, there are five distinct growth scores, each corresponding to a period of learning.  The first period is during kindergarten. Growth for this period is calculated by subtracting the fall test scores, measured near the time children enrolled in kindergarten, from scores measured in the spring, toward the end of the school year.  The second period measures growth over the summer between kindergarten and first grade.  The first grade school year is the third period, the difference between the end of kindergarten and the end of first grade is the fourth period, and the fifth period spans the beginning of kindergarten to the end of first grade. 

Students were tested in three areas: reading, math and general knowledge. For the reading test, early assessments measured basic letter recognition and vocabulary, while later tests evaluated students’ reading comprehension.  Math assessments started with number recognition and properties and moved into more advanced numeric functions in the upper grades, while the general knowledge test measured students’ knowledge in science and social science.  

Learning in Kindergarten
Sector effects on both school year and summer learning are inconsistent and mixed.  After controlling for student background, which should allow for a comparison of students’ preparation for school on fairly equal footing, the reading and math skills of students who attend Catholic or "other religious" kindergartens do not improve as much as reading or math skills of students who attend public schools.  Students at private, secular schools, though, outpace peers at public schools in their acquisition of math skills in kindergarten and lag behind in the development of their general knowledge.  

Summer Learning
Socioeconomic factors are often associated with summer learning. High SES families engage in activities and have norms that contribute to increased summer learning relative to students in families in lower strata of the socioeconomic distribution.  Even after accounting for differences in family SES, Carbonaro finds meaningful differences between school sectors in student learning.  Again, these differences would be attributable to learning outside school.  The reading skills of students who attend private secular schools improve considerably more over the summer than the skills of students who attend public schools.  Summer learning is more stagnant, though, for students who attend Catholic and other religious schools – these students lag a great deal behind public school students in both math skills and general knowledge.

First Grade Learning
In first grade, students at "other religious" (non-Catholic religious) schools fall short of their peers in learning growth in all three areas – reading, math, and general knowledge.  At the same time, students in both Catholic and private secular schools gain more reading skills relative to their public school peers.

Learning Across Periods and Years
Catholic school students outperform public school students in reading skill development between kindergarten enrollment and the spring of first grade, but fall behind in math skills between both the fall and spring of kindergarten and the spring of first grade.  Students at private secular schools also gain more reading skills, but they do so between both kindergarten tests and the spring of first grade.  The picture is much more bleak for students at other religious schools, where students gain less than public school peers between both kindergarten tests and the spring of first grade in all three subject areas. 

As Carbonaro points out, though, the analysis of learning across periods and years indicates that "conventional estimates of sector effects are likely to overgeneralize sector effects across grade levels, and hence distort our understanding of when sector differences are present in students' academic careers" (p. 237).  This overgeneralization leads us to rely on estimates of sector effect that are often downwardly biased.  In other words, sector differences in learning might actually be larger than we previously thought. By not accounting for learning that happens at home over the summer, estimates that ignore summer learning underestimate the effect of attending a Catholic school in primary grades. 

Keywords: Catholic Schools, Private Secular Schools, Other Religious Schools, Math Skills, Reading Skills, General Knowledge, Sector Effects, Summer Learning

Sector: Catholic,  Evangelical Protestant,  Jewish,  Muslim,  Other Protestant,  Private,  Public

Outcomes: Academic

Date Posted: 2015-11-10