Reading First Impact Study
Analysis of the May 2008 Interim Report on Reading First
The recently publish Reading First Impact Study has attracted a lot of attention and been used to show that Reading First is ineffective. What Does the report really say?
When USA Today and the New York Times both published articles on the recently published Reading First Impact Study: Interim Report they used the word "ineffective" in their headlines. Along with the Associated Press, the Washington Post, and most other major news sources, the media in general has left readers with the opinion that the recent report basically says that Reading First is ineffective. But even a quick read of the report makes clear that that conclusion is not accurate.
What is Reading First?
Reading First is a federal initiative whose goal is improve reading instruction in public schools - especially in grades K-3. The program gives grant funds to states and states distribute the money to qualifying schools based on an application process. The schools then use the award money to implement Reading First curriculum and instruction requirements.
The Scope of the Impact Study
The Impact Study, published a couple of weeks ago, looked at about 240 schools in 18 "sites" - 17 school districts across the nation and in one state-wide program. In all, the study looked at reading instruction in schools that had gotten reading first awards in 12 states. As of April of 2007 Reading First awards had been given to 5,880 schools in 1,809 school districts.
The Impact Study measured teacher behavior by counting how many times teachers referred to the one of the five components of reading as defined by Reading First. Those five components are phonemic awareness, phonics, fluency, vocabulary, and comprehension. The impact study also measured reading comprehension in students at schools in the study.
News Coverage
News sources generally seized on a single statement that occurs early in the Impact Study's Executive Summary: "On average, across the 18 participating sites, estimated impacts on student reading comprehension test scores were not statistically significant." It was also difficult to distinguish news coverage of the report from news coverage of reaction to the report by politicians. A number of leaders in Congress took the opportunity to voice concerns about Reading First.
The Study's Findings
The Impact Study makes a distinction between schools that received award money early in the program's history (in 2003) and those that received award money later (in 2004). Study schools that received their awards in 2003 didn't show significant gains in reading comprehension when compared to schools without grant money. But schools that received their awards in 2004 did show significant improvement.
The "late award" schools probably benefited from the practical experiences of the early award schools. Federal programs do, after all, have a learning curve. The late award schools also receive considerably more money than early award schools - $574 per student, compared to only $432 per student for the early award schools. While it is not possible, according to the Impact Study, to say why, the late award schools show a significant, positive difference in reading comprehension when compared to schools without reading First grant money.
A Final Note
The study's findings are open to interpretation. But this Impact Study is an interim report. The hope is that many of the questions about the study will be resolved when a final report is published late in 2008. The study's conclusion may not matter for a couple of reasons, though. First, there are questions about the methods of the study - questions that make the study's scientific validity doubtful. Second, Congress has begun cutting funding to Reading First for political reasons; good or bad, the program may not exist in another year or two...
