Comparing the Average Scores of Students in One Cohort to the Average in a Later Cohort

When a new superintendent came to Mountain View School District he mandated the use of “balanced literacy” and “whole language” programs throughout the elementary schools. After two years of this new approach, some school board members became concerned about rumors of lowered achievement. They asked the Superintendent for data on reading achievement of first graders over the last five years, and he provided information on the average and standard deviation of first graders on a curriculum-based measurement (e.g. DIBELS or AIMSWeb). As the school board members suspected, scores had declined after the change in reading programs. The Superintendent insisted that the change was simply due to chance and wasn’t significant. The school board members used the EIC to test that assertion.

In the current year, two years after the curriculum change, the average reading composite score of district first graders was 58, with a standard deviation of 16. In contrast, two years earlier and before the change the average score was 65, with a standard deviation of 12. Three hundred students were tested in each year. The data entered into the EIC and the results are shown below. They indicate that the school board members had good reason to be concerned. The effect size of -.50 indicates that the decline in first graders’ reading skills since the curriculum was instituted was educationally significant. The probability level shows that this result was very unlikely to have occurred by chance (a probability of less than one in 1,000). The average first grader in the current year had scores that were 19 percentile ranks lower than the average first grader two years earlier using the previous curriculum.