When Schools Shut Down: How Education Interruptions Can Hurt Student Achievement

By Denise-Marie Ordway

As children across the U.S. head back to school for the fall semester, educators will be busy helping them transition from online classes back to in-person instruction. One of teachers’ priorities will be helping students who fell behind after the coronavirus forced campuses to close catch up on their lessons.

Academic studies indicate education interruptions, whether planned or the result of an emergency, can have short- and long-term consequences for students in kindergarten through twelfth grade. Studies also show impacts can vary depending on a range of factors, including student age, the time of year schools close and children’s socioeconomic backgrounds.

Scholarly journals have published numerous articles investigating disruptions to face-to-face instruction — for example, when schools dismiss students for summer vacation or in cases of extreme weather such as a hurricane or snowstorm. However, academic researchers are still trying to measure the pandemic’s impact on student achievement. So far, few peer-reviewed studies focus on the U.S.

While high-income students generally tend to do better in school than low-income students, many education researchers predict the pandemic has widened learning gaps among elementary school students, according to survey results published earlier this year in the journal Educational Researcher.

“In recent months, private testing companies used by some school districts have reported inconsistent findings based on district-administered diagnostic tests,” Vladimir Kogan and Stéphane Lavertu, professors at The Ohio State University, write in a report prepared in early 2021 for the Ohio Department of Education. “Using national samples, some have found that student achievement at the beginning of the current academic year fell well below previous years, while others have suggested fairly minimal impacts — especially in reading and in higher grades.”

Kogan and Lavertu’s own analysis indicates Ohio third graders, as a whole, did worse on the state’s language arts assessment in fall 2020 than in fall 2019. But school districts there varied considerably in terms of how much test scores changed. In more than 60 districts, language arts scores did not decline.

Journalists can look to academic research to better understand how school closings, parent unemployment and other pandemic-related disruptions have impacted student learning over the last year and a half. We’ve updated this piece, originally published in May 2020, with new research and resources. We hope this will help newsrooms strengthen their back-to-school coverage and ask more probing questions about how educators will assist students who fell behind because they lacked the technology, adult guidance and other things needed to keep up with lessons and assignments.

Below, you’ll find a sampling of academic studies on student learning amid education disruptions. We also share reporting tips from two experts at Teachers College, Columbia University — Alex Bowers, an associate professor of education leadership, and Jeffrey M. Young, a former school district superintendent who’s now a professor of practice in education leadership. While we interviewed Bowers and Young in 2020, their insights remain relevant.

————–

Learning Loss Due to School Closures During the COVID-19 Pandemic
Per Engzella, Arun Freya and Mark D. Verhagen. Proceedings of the National Academy of Sciences of the United States of America, April 2021.

In this study, researchers at the University of Oxford examine the effect of school closures on student progress in the Netherlands. They find that children aged 8 to 11 years learned less while learning from home during an eight-week lockdown in early 2020. Learning losses were particularly pronounced among kids from disadvantaged homes.

“The effect is equivalent to one-fifth of a school year, the same period that schools remained closed,” the researchers write. “Losses are up to 60% larger among students from less-educated homes, confirming worries about the uneven toll of the pandemic on children and families.”

Primary school students in the Netherlands take national exams twice a year in math, spelling and reading — at the beginning of the calendar year and again in June. That allowed researchers to compare student test performance before and after the lockdown, which began in March 2020. They compared student performance for 2020 to student performance during the previous three years.

The analysis focuses on 15% of Dutch primary schools and a total of 350,000 students.

The researchers call the results a best-case scenario, considering the short lockdown and the fact that the Netherlands has equitable school funding and high rates of broadband access. “Despite favorable conditions, we find that students made little or no progress while learning from home,” they write.

The Revealed Preferences for School Reopening: Evidence from Public-School Disenrollment
Thomas Dee, Elizabeth Huffaker, Cheryl Phillips and Eric Sagara. National Bureau of Economic Research Working Paper 29156, August 2021.

U.S. public school enrollment fell 2% during the first full school year after the onset of COVID-19, and even more at schools offering remote-only instruction, finds a new working paper from researchers at Stanford University.

The researchers estimate that remote-only instruction reduced kindergarten enrollment by 3% to 4% and elementary school enrollment, as a whole, by 1%. In states where kindergarten is optional, enrollment fell 4.6%. Losses were “smaller and statistically imprecise” at middle schools and high schools.

The authors note reduced enrollment “is an important leading indicator of the educational impact of the pandemic.” Schools providing remote-only instruction lost a total of about 300,000 students, they find.

“Disenrollment from public schools implies that students have switched schools (e.g., to private schools or home-schooling), intend to skip or delay kindergarten, and dropped out or become truant,” the researchers write.

In places where students skipped kindergarten in 2020-21, schools in 2021-22 “will face the challenge of educating rising first graders who are in their first experience of formal schooling.” Delaying kindergarten for a year, known as redshirting, also will have consequences for schools in 2021-22.

“To the extent that a substantial number of students instead ‘redshirts’ into kindergarten classes this fall, schools may face the staffing and pedagogical challenge of managing an unusually large, mixed-age cohort,” they write. “In the absence of responsive staffing, these COVID-19 redshirters and their peers could experience unusually large class sizes throughout their schooling.”

Projecting the Potential Impact of COVID-19 School Closures on Academic Achievement
Kuhfeld, Megan; et al. Working Paper No. 20-226 from Brown University’s Annenberg Institute for School Reform, May 2020.

This working paper predicted U.S. public school students would likely start the 2020-21 school year having learned 37% to 50% of what they ordinarily would have learned in math had schools remained open. “In some grades, students may come back close to a full year behind in math,” write the authors, from the University of Virginia, Brown University and the nonprofit education research organization NWEA

The authors estimated that, on average, kids likely would begin the fall 2020 semester with 63% to 68% of the language arts knowledge they would have gained in a typical school year. Not all students would lose ground, however, according to preliminary estimates. The authors predicted the top one-third of students would continue to make gains in reading despite education interruptions.

If that happened, teaching might be more challenging in the fall, they note. “Students will likely return not only with lower achievement (on average), but with a wider range of academic skills that may require teachers to further differentiate instruction,” writes the research team, led by NWEA research scientist Megan Kuhfeld.

To generate preliminary estimates for how student learning could be impacted by school closures in the wake of the coronavirus pandemic, researchers analyzed data from prior studies on student absenteeism. They also examined data on learning patterns among a national sample of 5 million U.S. students in grades 3 to 8 across two school years — 2017-18 and 2018-19 — and the summer break in between.

“In this study, we produced a set of possible scenarios for learning loss rates during the extended period when schools are physically closed and students are not receiving normal face-to-face instruction,” Kuhfeld and her colleagues write. “These projections can help prepare educators and parents for the degree of variability in student achievement to expect when school resumes, including over the course of the upcoming school year.”

The researchers write that their estimates don’t take into account many factors that likely were influencing learning during the school shutdowns — changes in parents’ employment status and access to food, for example. “Families with financial resources, stable employment, and flexible work-from-home and childcare arrangements will likely weather this storm more easily than families who are renting their housing, working in low-paying fields that are hardest hit by the economic impacts, and experiencing higher rates of food insecurity, family instability, and other shocks from this disruption,” they write.

Flaking Out: Student Absences and Snow Days as Disruptions of Instructional Time
Goodman, Joshua. National Bureau of Economic Research working paper, June 2014.

In this study, researcher Joshua Goodman investigated how school closings for snow and student absenteeism in Massachusetts affect student achievement. His analysis reveals that, overall, there is “a strong relationship between student absences and achievement but no relationship between school closures and achievement.”

However, when Goodman examined the data more closely, he found evidence of a link between school closures and lower test scores at public schools where a high percentage of students live in low-income households. “Closures generally have little effect on achievement except in poor schools,” writes Goodman, now an associate professor of economics at Brandeis University.

Each day of closure, the study finds, is associated with a drop in scores on the math and English language arts sections of the Massachusetts state exam.

The study sample comprises all students enrolled in Massachusetts public schools between the 2003 and 2010 academic years who had no more than 60 absences.

Unscheduled School Closings and Student Performance
Marcotte, Dave E.; Hemelt, Steven W. Education Finance and Policy, Summer 2008.

Unscheduled school closings hurt student scores on statewide tests in Maryland, but the impact was largest when schools closed during the spring semester, this study finds. It also finds that these education interruptions posed greater harm to the academic performance of children in the early grades.

Two University of Maryland researchers examined the relationship between school closures for snow and student test scores in reading and math between 1994 and 2005. The researchers looked specifically at the scores of children in grades 3, 5 and 8.

“Snow early in the season has a negative effect only in the case of performance on third-grade reading exams,” the authors write. “Snow in December, but not in January, has a consistently negative effect on pass rates across subjects and grades. … It is disruption in February that appears to have the most consistent, negative impact on test performance.”

When there were five unscheduled school closures during the winter, the number of third graders who received satisfactory scores in reading and math was 3% lower than during winters with no closures. In winters with 10 unscheduled closings, more than 5% fewer third graders received passing reading and math scores. In both scenarios, these education interruptions had less impact on students in grades 5 and 8.

Social Impacts of Hurricane Katrina on Displaced K-12 Students and Educational Institutions in Coastal Alabama Counties: Some Preliminary Observations
Picou, J. Steven; Marshall, Brent. Sociological Spectrum, 2007.

This study looks at how student learning and behaviors changed in the wake of Hurricane Katrina, which damaged or destroyed dozens of public schools in Louisiana and Mississippi in August 2005. The study, based largely on interviews, focuses on displaced students who had been evacuated to Mobile and Baldwin counties in Alabama.

“Our discussions with school district social work counselors revealed that some displaced students seemed to have lost their ability to concentrate on assignments and manifested symptoms of clinical depression,” write the authors, J. Steven Picou, a sociology professor at the University of South Alabama, and Brent K. Marshall, a sociology professor at the University of Central Florida who died in 2008.

In the paper, researchers provide bulleted lists of the challenges and unforeseen consequences described in interviews they had with school personnel, parents and students. One major problem identified: A shortage of adequate and affordable housing forced displaced families into a pattern of ‘‘serial relocation,’’ moving from school to school within a school district.

Picou and Marshall indicate that, at the time of the study, many displaced students continued to experience stress, anxiety and uncertainty about their futures. “This fact has perpetuated poor academic performance, discipline problems, and irregular school attendance,” they write. “In general, these negative social, psychological, and educational impacts may continue for some students well into the future.”

The Effects of Summer Vacation on Achievement Test Scores: A Narrative and Meta-Analytic Review
Cooper, Harris; et al. Review of Educational Research, Autumn 1996.

This paper examines almost a century of research on the effects of summer vacation on student achievement. A key takeaway: Student test scores fall over the summer months — a loss equivalent to “about one month on a grade-level equivalent scale or one tenth of a standard deviation relative to spring test scores,” writes the research team, led by Harris Cooper, now a professor of psychology and neuroscience at Duke University.

The team’s analysis of the 39 studies spanning from 1906 to 1994 also shows that taking a summer break from school impacts academic subjects differently. Losses in learning are more dramatic for math-related subjects than for reading and language arts. Math computation and spelling skills deteriorate most, researchers explain.

“Cognitive psychologists suggest that factual and procedural learning requires extensive practice,” they write. “Thus, the relative lack of opportunity to practice computation and spelling over summer vacation may mean that these facts and procedural skills are most susceptible to decay.”

The researchers also find that kids from lower-income families backslide more than middle-income students do. “In fact, middle-class students appeared to gain on grade-equivalent reading recognition tests over summer, while lower-class students lost on them,” they write.

Beyond the research: reporting tips and additional insights

While research offers important insights, Columbia University professors Bowers and Young note there are substantial differences between education interruptions that happened before COVID-19 and campus closings during the pandemic.

An important distinction: Even though campuses closed during the last half of the 2019-20 academic year and for all or part of the 2020-21 academic year, educators provided instruction to children at home, although with varying approaches and success.

“It’s difficult to extrapolate from [research about] other types of school closures — it’s an unprecedented time for this many schools to close for this long,” Bowers says.

Another difference: When school officials have ordered campuses closed because of severe weather, they typically made up for lost class time by extending the academic year or shortening student breaks, Bowers notes.

During the 2021-22 academic year, schools will introduce a variety of changes to try to make up for instructional time lost because of COVID-19. The federal American Rescue Plan, announced in early 2021, includes $130 billion to help K-12 schools address learning gaps.

“Districts must ensure that funds are used to not only reopen schools, but also to meet students’ academic, mental health and social, and emotional needs in response to COVID-19, (e.g. through extended learning time, tutoring, and counselors), wherever they are learning,” according to a press release from the White House.

Young, who was superintendent of schools in Cambridge, Massachusetts from 2009 through 2016, recommends journalists build on what’s known from research on education interruptions. His advice: Ask school custodians, transportation managers, teaching assistants and other school district employees how they are experiencing the pandemic and what concerns they have about opening campuses.

Custodians, for example, can talk about the challenges of keeping classrooms, locker rooms, bathrooms, hallways and other areas sanitized — a monumental task, especially at high schools, which can house thousands of students and faculty at any one time, Young explains. Teachers, on the other hand, can speak to the ways COVID-19 has affected educators and students socially, emotionally and physically.

“Research is powerful,” Young says. “But drill down to the stuff the superintendents are worrying about every day, down to what it would actually look like in practice.”

Looking for more education reporting resources? Check out our collections of research on COVID-19’s impact on student mental health and how child abuse reports declined after schools closed. We also created a tip sheet to help journalists cover college finances during the pandemic.

—

Previously Published on journalistsresource with Creative Commons Licenses

***

You Might Also Like These From The Good Men Project

Compliments Men Want to Hear More Often

Relationships Aren’t Easy, But They’re Worth It

The One Thing Men Want More Than Sex

..A Man’s Kiss Tells You Everything

—

Photo credit: iStock

The post When Schools Shut Down: How Education Interruptions Can Hurt Student Achievement appeared first on The Good Men Project.