Comparing Changes in the Percentage of Students at Benchmark in One Group with Changes in a Larger Group

As described in another example,  Oak Elementary began using Reading Mastery in grades K-3 in the fall of 2012. In Spring 2012, before starting the new curriculum 40 percent of Oak Elementary third graders scored at the proficient or advanced level on the state assessment. In Spring 2016 65 percent of Oak Elementary third graders scored at that level. Seventy students were tested in 2012 and 85 students were tested in 2016. But, over that time period, there was also a change in the percentage of students in the state as a whole who scored at the proficient level – from 55 percent in 2012 to 60 percent in 2016. Was the change at Oak Elementary educationally significant when compared to the changes in the state as a whole? Could it have just appeared by chance?

To answer these questions, data could be entered into the EIC. Note that, as with other examples, the EIC specifies the lines in which data for each year should be entered. The data for the more recent year are entered first, followed by the data for the later year. Also note that, to obtain tests of significance, the user only needs to know the number of cases at the organization of interest (in this case Oak Elementary), not the larger group.

The effect size of .40 is beyond the level generally used to denote educational significance, the improvement index is large, and there is only a remote possibility the results could appear by chance. Thus, the Oak Elementary teachers could conclude that the improved achievement of their students was greater than that of students in the state as a whole.