By William T. Gormley, Jr.
Over the past decade, my colleagues and I have produced a series of peer-reviewed articles evaluating the effectiveness of Oklahoma’s universal pre-K program, which President Obama praised in his State of the Union Address. Oklahoma’s program, established in 1998, now reaches approximately three-fourths of the state’s four-year-olds. The “Sooner State” has decided that sooner is better than later when it comes to early childhood education.
Our research has focused on Tulsa, the largest school district in the state. The Tulsa school district was an attractive research site because it serves children from diverse racial and ethnic groups.
When we first visited Tulsa, we had reason to believe that the Tulsa pre-K program would be successful. Unlike many other preschool programs, Oklahoma mandates high quality, at least in terms of educational inputs: every lead teacher must have a B.A. degree, must be early childhood certified, and must be paid a public school wage. A child-staff ratio of 10/1 ensures that each child receives personal attention.
As it turns out, our hunch was correct. In fact, the Tulsa pre-K program has generated phenomenal improvements in school readiness. If we look at students as a whole, kindergarten students who attended the Tulsa pre-K program are 9 months ahead of their peers in reading, 7 months ahead in writing, and 5 months ahead in math.
Both disadvantaged and middle-class students experience statistically significant and substantively significant gains in test scores. Disadvantaged students benefit more, but middle-class students also benefit considerably.
Gains are especially striking for English language learners – specifically, Hispanic students whose parents were born in Mexico. For Hispanic children whose parents primarily speak Spanish at home, pre-K participants scored 12 months better than their control group in reading, 4 months better in writing, and 10 months better in math. We also see statistically significant and substantively significant gains in test scores for whites, blacks, and Native Americans.
Thanks to an in-depth investigation of preschool classrooms, we know that what is happening in Tulsa is not magic – it is good, solid teaching. Using Robert Pianta’s instrument for observing classroom dynamics, we found that Tulsa’s pre-K teachers provide higher levels of “instructional support” than their counterparts in school-based pre-K programs in 11 other states. Using another instrument, we found that Tulsa’s pre-K teachers devote more classroom time to reading and math than their counterparts elsewhere.
We also know that the Tulsa pre-K program is producing some socio-emotional impacts. In concrete terms, we found that kindergarten students who participated in pre-K were more attentive and less timid than kindergarten students who did not participate in pre-K. Unlike some national child care studies, we found that participation in the Tulsa pre-K program produced no negative socio-emotional effects.
Since our studies were published, we have received many questions, some praise, and some criticism.
One persistent criticism is this: if Oklahoma’s pre-K program is so fantastic, then why haven’t Oklahoma’s NAEP scores increased sharply as a result? Shikha Dalmia and Lisa Snell of the Reason Foundation and Lindsey Burke of the Heritage Foundation have raised this question, noting that 4th grade reading scores have actually declined in Oklahoma since universal pre-K was introduced.
The first problem with this critique is that NAEP scores are more useful for describing state and national trends than for evaluating state policy interventions. Sweeping conclusions about preschool based on NAEP data ignore other public policies that profoundly affect the performance of Oklahoma’s school students. One of these is the level of state funding for K-12 education. Whereas Oklahoma ranks extremely high in both pre-K access and pre-K quality, it ranks extremely low (in the bottom five nationally) in financial support for K-12 public schools. Without a substantial infusion of state funds, it would be difficult, though not impossible, for Oklahoma to improve its 4th and 8th grade test scores.
A second problem with the critique is more subtle. It assumes that an intervention, like a pre-K program, will produce 4th grade effects as soon as the first cohort of participants reaches 4th grade. Unfortunately, 4th grade test scores depend not just on exposure to pre-K but also on any changes in pedagogy that take place in K-3 classrooms. In practice, it probably takes a while for K-3 teachers to adjust their habits and expectations to reflect the new surge in pre-K enrollments. Our latest evidence from Tulsa is consistent with this hypothesis. When we investigated the persistence of positive pre-K program effects on 3rd grade test scores for the earliest cohort in our data base (students who participated in pre-k in 2000-01), we found no difference between pre-K participants and non-participants. However, when we investigated this for our latest cohort (students who participated in pre-K in 2005-06), we did find persistent differences between pre-K participants and non-participants through 3rd grade. In short, it takes a few years for a school system to adapt to the potentially powerful changes wrought by a high-quality pre-K program.
A different criticism of our Tulsa findings is that our research design falls short of the high standards established by randomized experiments, such as the Perry Preschool and Carolina Abecedarian Projects. The rigorous research design we employed, considered novel at the time, was a “regression discontinuity” strategy for estimating the effects of early childhood education, which took advantage of a strict September 1 birthday cutoff for determining preschool eligibility. By comparing children whose parents decided to enroll them in the school-based pre-K program but who differed in their date of birth, we were able to focus exclusively on students who had just completed pre-K and on students who were about to begin pre-K. This is an excellent way to reduce “selection bias” because the treatment group and the control group both participated in the same pre-K program, albeit at different times.
Of course, we also controlled for each student’s precise date of birth, because older students generally do better on standardized tests than younger students, and for many other variables as well.
To some critics, like Grover Whitehurst of the Brookings Institution, our regression discontinuity design is problematic. Whitehurst cites the hypothetical example of two adopted children who are roughly similar in age but one qualifies for pre-K a year earlier than the other. Won’t parents try to give the slightly older child cognitive and play experiences that help him or her to keep up with older peers at school? If so, it might not be fair to compare the two children.
This is a reasonable hypothesis, but it is only that. The opposite hypothesis, suggested by others, would be that parents focus more assiduously on the child who just missed the pre-K cutoff, in an effort to compensate for delayed entry into preschool.
The bottom line is that our treatment group children and our control group children are strikingly similar in their observable characteristics, even before applying statistical controls. This suggests that our regression discontinuity design is working and working well.
Although we strongly prefer our regression discontinuity approach when studying short-term effects, we have also utilized other techniques, like OLS regression and propensity score matching, to estimate our model. Whichever technique we use, the positive effects of the Tulsa pre-K program are clear and unmistakable.
Because students from recent pre-K cohorts in Tulsa have not yet become adults, it is impossible to say for sure whether the changes wrought by Oklahoma’s universal pre-K program will eventually yield higher earnings and other favorable outcomes. However, by integrating Tulsa kindergarten test scores with Tennessee data linking kindergarten test scores to adult earnings, we have projected that the adult earnings benefits of the Tulsa pre-K program are likely to exceed the costs by 3 to 1 or 4 to 1. Were we to include criminal justice effects, the benefit-cost ratio could be even higher.
There is much more to be learned from Oklahoma, Georgia, Chicago, Boston, and other research sites where high-quality pre-K programs have been made available to large numbers of students. Scholars and activists should continue to scrutinize the empirical research on early childhood education. At the same time, we should not allow skeptics to blind us from the central message of many well-designed empirical studies: a high-quality preschool education benefits children in the short run, especially disadvantaged children, and benefits society as a whole in the long run. That is the key insight animating President Obama’s universal pre-K initiative, and it has been validated by plenty of credible scientific research.
William T. Gormley, Jr. is professor of public policy at Georgetown University and has served as the Co-Director of the Center for Research on Children in the U.S. since 2001.