Measure for measures: What do standardized tests really tell us about students and schools?
HGSE Professor Daniel Koretz
Education leaders need to think deeply about the role of standardized testing in public schools. While many people agree that some system of accountability is essential to maintain school quality, there is disagreement about the role of high-stakes testing in that system. In this Usable Knowledge video interview, HGSE Professor Daniel Koretz shares insights about the strengths and limitations of standardized tests from his new book, Measuring up: What educational testing really tells us.
Daniel Koretz aims to “make the guts of testing accessible” to the education community. As a professor at HGSE, Koretz teaches students how to interpret test results and how to use them to make decisions. His new book reaches out to a broader audience of policy makers, principals, teachers, and parents, with two goals: to explain the basic principles of testing to the general public, and to address such controversial issues as the “high stakes” increasingly riding on tests.
“What I would like is for people to understand what tests can and cannot do,” Koretz says, “and then put tests in perspective to get a more balanced perspective of how you judge schools, teachers, and kids. I’m not arguing that we should stop testing – I think it’s valuable – but we need to temper our expectations of what tests can do.”
What tests can tell us about student performance
Standardized tests are designed to enable us to compare the performance of students in a relatively efficient way. But how much can tests tell us about what students actually know? In this video clip, Koretz describes the kind of conclusions we can draw from standardized tests – first about American schools, and then from international comparisons.
Video Clip 1: <view video>
Limitations of standardized and high-stakes testing
There are limits to the meaning we can derive from test scores. What education leaders want is a fair, straight-forward measure of school performance, to be able to monitor schools and hold them accountable. The problem, in Koretz’s view, is that we tend to overestimate what tests can do. Tests are not designed to summarize all that students and schools can do.
Video Clip 2: <view video>
Current testing practices and NCLB
In his interactions with policy makers, Koretz notes that there is a political stalemate in Washington about where to go next with the No Child Left Behind (NCLB) act. While there is a slowly growing awareness that accountability systems based only on test scores are too simple, we need more information about alternatives to determine which might be improvements. In this clip, Koretz shares some of his concerns about current testing practices, and his thoughts about what should be done to tackle these problems.
Video Clip 3: <view video>
Testing in preschool and the early grades
Extra care needs to be taken when we consider testing very young children. Koretz wants education leaders to weigh thoughtfully the instructional purpose of the test, and the effect of the testing experience on the child.
Video Clip 4: <view video>
Testing students with special needs
Making accommodations for some students seems to violate the notion of standardization – what’s the point of using a standardized test, if it is not administered in a standardized way for all students? However, when students have certain disabilities, their test scores may in fact be misleadingly low.
Consider students with a range of visual impairments. Few people would withhold reading glasses from a student who would otherwise strain to bring text into focus. With the use of glasses, the student’s actual proficiency is more accurately reflected in the test score. For students with more severe vision problems, larger print, modified room lighting, or a specialized computer screen can often serve as a corrective lens that makes the test-taking process more manageable. The purpose of an accommodation is not to make the students’ scores higher, but to help them score as well as their actual proficiency warrants.
Providing accommodations to offset vision problems is straightforward enough, and as long as resources are available, there may be little debate about offering them. The issues are stickier, however, when the impediments caused by the student’s disability are directly relevant to the knowledge and skills the test is designed to measure. A sharp example is the challenge of testing reading comprehension skills of a student with dyslexia.
One potential accommodation is to read the test to the student, or present it via pre-recorded audio, so that lower-level word-reading problems are bypassed. After all, interpreting the meaning of text is a different skill from that of decoding words, which is a barrier for students with dyslexia. It is clear, however, that an accommodation like oral presentation of material would fundamentally change the test. In this case, the reading comprehension task would turn into an oral language comprehension task.
Teachers and students often feel helpless when students make genuine gains in their understanding, but are not able to demonstrate those gains on a test. Recognizing the limitations of tests can help zero in on the real challenge of accurately estimating the abilities of special needs students.
For more information, see Daniel Koretz (2008), Measuring up: What educational testing really tells us (Harvard University Press).