Possible Uses and Limitations of the EIC

The No Child Left Behind Act requires schools to publish information on their students’ performance on standardized assessments. State departments of education make this information publicly available on the web and often disseminate it through the media. The EIC lets educational consumers use these data in ways typically employed by educational researchers and compute effect sizes that can indicate the extent to which comparisons would be deemed educationally significant.

Thus, the Educational Impact Calculator is designed to empower educational consumers and counter the impression that only “specialists” can understand achievement data. The hope is that it will give parents and policy makers the tools to independently assess the extent to which achievement in their schools differs from that in others and the extent to which changes in achievement in their schools over time would be deemed educationally important and statistically significant. The procedures described are simple and use publicly available data. Because they do not require high priced educational consultants, they are also inexpensive. Most important, they provide accurate and valid results to the questions that educational consumers typically ask and provide data for the most effective advocacy for their students.

While the EIC can be useful to educational consumers, users need to remember its limitations. Most important, they should realize that the findings that are produced are only as good as the data that are available. The assessment information that is typically released to the public is, at best, a snapshot in time. Thus, it provides only a partial view of the learning that may be occurring in a school. Wise users will want to consult as many sources of information and data as possible, looking at a variety of assessments, data from a range of time periods, and information for different grade levels.  In addition, users need to remember all the many factors that can influence student learning. Changes in administration, staffing levels, supports for teachers, curricular materials, time for instruction, and behavioral climate within a school can all be important factors. Wise consumers will want to ensure that they have considered as many relevant factors in their analyses as possible. In short, while the results certainly provide useful information, wise consumers continue to gather and assess data – knowing that more information is always better than less information.