Can kids really learn as much from “Sesame Street” as from preschool?
The most authoritative study ever done on the impact of “Sesame Street” … finds that the famous show on public TV has delivered lasting educational benefits to millions of American children — benefits as powerful as the ones children get from going to preschool.
Really?
The paper — written by two economists, Melissa S. Kearney of the University of Maryland and Phillip B. Levine of Wellesley College — says “Sesame Street,” first broadcast in 1969, was effectively the first MOOC, or Massive Open Online Courses (available since 2008).
The authors examine differences in access to “Sesame Street” when and after it was first launched in 1969 in areas of the United States that had VHF, and other areas that had the weaker UHF, which did not reliably carry the station that broadcast the show. This comparison in broadcast strength was then matched with what the research said were student outcomes in an effort to show that kids in those areas where the show was broadcast had better academic outcomes that were statistically significant than in those areas where the broadcast signal was weak and where it was likely the kids didn’t see as much of “Sesame Street.”
The authors said they did not actually know whether kids in either group watched “Sesame Street”; just that it was more available, and that they were able to factor out other causes for the difference in outcomes for students. The paper says:
The results indicate that Sesame Street accomplished its goal of improving school readiness; preschool-aged children in areas with better reception when it was introduced were more likely to advance through school as appropriate for their age. This effect is particularly pronounced for boys and non-Hispanic, black children, as well as children living in economically disadvantaged areas. The evidence regarding the impact on ultimate educational attainment and labor market outcomes is inconclusive.
[W]e find that children who were preschool age in 1969 and who lived in areas with greater predicted Sesame Street coverage were statistically significantly more likely to be at the grade level appropriate for their age…
The authors don’t say conclusively that “Sesame Street” proved as academically beneficial as preschool, but they think highly enough of it as an educational method to conclude the body of the working paper with this: “As research and policy discussions continue to focus on early childhood education, we believe that the impact of Sesame Street deserves to be included along with Perry Preschool, Head Start and other programs.”
They also say that a preschool “blended learning environment incorporating both electronic communication of educational content and the human element to affect the ‘soft skills’ may be preferable, and cost-effective.”
In fact, there is at best scant evidence that blended learning is a successful model. Besides that, Steve Barnett, director of the National Institute for Early Education Research at Rutgers University, wrote in an e-mail:
To believe their results you have to believe that TV teaching through Sesame Street has a much deeper and more profound effect on the child than a teacher. What is the theory that would explain this? They do not have a theory or explain how their results are consistent with the larger body of knowledge about learning and teaching. This is the most disturbing aspect of the paper.
James J. Heckman is a Nobel laureate in economics who is the Henry Schultz Distinguished Service Professor of Economics at the University of Chicago and director of the university’s Center for the Economics of Human Development. I asked him what he thought of the paper. Here’s what he wrote (and this also appeared on his blog):
A recent study, Early Childhood Education by MOOC: Lessons from Sesame Street by Melissa Kearney and Phillip Levine, has been generating interest and, unfortunately, generalized comparisons with other early childhood programs. As noted by the authors, the study looks at the effects of access to Sesame Street, not at the value of Sesame Street versus Head Start or any other high-quality preschool program. The study measured a small set of relatively short-term outcomes that mirrored some — but by no means all — outcomes from Head Start and other preschool programs.If you’re looking for a silver bullet, it’s not television, but rather a scaffolding of support for skills development from birth to age five. Skill begets skill over the lifecycle of human development and learning. A large body of solid research shows that better outcomes for disadvantaged children come from a combination of health, nutrition, parental education and early learning resources from birth to age five that nurture cognitive and social and emotional development, and provide the foundation for success in school and life.The Carolina Abecedarian Study clearly made this point: it started from birth, incorporated parent education, health, nutrition, early learning and preschool. As such, Abecedarian is the only program that has produced lasting gains in IQ in addition to the better education, social and economic outcomes we see from other programs. More importantly, Abecedarian’s lasting IQ boost played a significant role in reducing chronic disease in males and obesity in females.The solution for promoting school readiness and fostering productive skills isn’t simply planting children in front of the television or tablet. High-quality educational programming can serve as a complement to quality early childhood education, not as a replacement.
Asked whether he believes “Sesame Street” is as “effective as preschool,” Levine, one of the authors, said in an e-mail:
I think the “as effective as preschool” (not our quote) comes from the finding that the impact of the show on test scores and on the likelihood of remaining at grade level are comparable to estimates obtained from Head Start. We do not endorse what may be an obvious extension … then we don’t need preschool.
In a separate e-mail Kearney, the other author, addressed the fact that the researchers did not know if students actually watched “Sesame Street” but looked at areas with greater vs. less access to the show. She wrote:
The fact that we don’t know which kids watched the show means we say something about the generation of kids who had access to the show. This is a real methodological strength, not weakness. We find that kids who lived in places where the show could be readily viewed saw improvements in educational outcomes as compared to kids in places where the show could not be readily viewed. This allows us to say that was because of the show.Our analysis controls for differences between the places by using a comparison group of older kids who were already in school when the show came on the air — we use those older kids to difference out any pre-existing differences in educational outcomes for kids in different cities. Then we see that after the show is aired, the kids in cities with broadcast access to it do relatively better. What we have to control for are other policies that might have come into effect at exactly 1969 and affected preschool age children differently from older children. We control for the roll-out of the food stamp program and Head Start funding levels just to be very cautious, though there were not discrete breaks in those programs at exactly the same time in a way that was correlated with the strength of PBS broadcast signals.In contrast, previous studies that look at kids who watched the show versus kids who didn’t watch the show have the problem that the kids who watch the show might have very different families or parents — for example, perhaps kids who watch the show have parents who are very committed to their learning, and that is why they watch the show. In that case, researchers are unable to attribute differences to outcomes to the viewership of the show. Our approach surmounts that challenge.
Barnett doesn’t agree. Asked about the authors’ methodology, Barnett said it “just pushes the selection bias problem up from the individual level to the community level where they must be correct that there are no other differences around the same time that would affect the outcomes or the access. It seems reasonable, but the answers do not seem reasonable.”
(Why economists think they can factor out every possible influence except for the one thing they are studying to draw a conclusive cause and effect is an issue for another day. Some economists in fact say they can eliminate every single factor that affects a child’s performance on a single standardized test to determine a teacher’s “value” to that child, including how children react to violence in their neighborhood, post traumatic stress disorder, depression, hunger and sickness — and, presumably, a lousy test design. Sure they can.)
There is another issue about the paper worth mentioning — the title, “Early Childhood Education by MOOC: Lessons From Sesame Street.” The working paper says:
An analysis of the effectiveness of Sesame Street can potentially also inform current discussions regarding the ability of Massive Open Online Courses (MOOCs) to deliver educational improvements. In essence, Sesame Street was the first MOOC. Although MOOCs differ in what they entail, Sesame Street satisfies the basic feature of electronic transmission of online educational material. Both Sesame Street and MOOCs provide educational interventions at a fraction of the cost of more traditional classroom settings. Most (but not all) MOOCs exist at the level of higher education, which clearly differs from a preschool intervention. Our knowledge of the ability of MOOCs to improve outcomes for its participants is so limited, though, that any proper evaluation of the impact of electronic transmission of educational content is beneficial.
Beneficial it may be, but the idea of suggesting that the discussion of MOOCS for adults can be informed by how preschoolers take in information is, well, a stretch. Just like thinking a television show is as good an educational model as a good preschool.