Effect Sizes

Researchers often use effect sizes to describe the magnitude of a result. Technically, an effect size describes the magnitude of a difference between two groups in standard deviation terms. A value of zero indicates no difference, while larger absolute values (i.e., either positive or negative) indicate more of a difference. A value of positive one (+1.00) indicates that a target group had scores that were one standard deviation larger than the comparison group; while a value of negative one (-1.0) indicates that a target group’s scores were one standard deviation lower than the comparison. An effect size of .50 indicates a difference of one-half of a standard deviation, etc. Within the field of education, effect sizes of .25 or larger have traditionally been considered educationally significant (Tallmadge, 1977). It should be noted, however, that the effect sizes associated with a strong curriculum are generally substantially larger.

The average effect size associated with implementations of Direct Instruction is estimated to be to be well over twice the .25 level. Hattie analyzed the results of four meta-analyses that included Direct Instruction (DI), incorporating 304 studies, 597 effects and over 42,000 students and found an average effect size of .59. Stockard (2013) used methods like those used in the EIC to examine assessment data from 18 different sites using the DI curriculum. She found an average effect size of .56, slightly smaller than the value reported by Hattie, but still more than twice the level used by Tallmadge.

In addition, the criterion of .25 should be seen as a touchstone or helpful guide to interpreting results. There is no magic associated with this particular number. It just provides a useful signpost, and examples of interpretations of results are included in the pages of examples.

References:

Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. London and New York: Routledge.

Stockard, J. (2013). “Merging the Accountability and Scientific Research Requirements of the No Child Left Behind Act: Using Cohort Control Groups” Quality and Quantity: International Journal of Methodology, 47, 2225-2257.

Tallmadge, G. (1977). The Joint Dissemination Review Panel Idea Book. Washington, D.C.: NIE, U.S. Government Printing Office.