How accurate are reading assessments?

Jun 17, 2024

“Data-driven” instruction makes sense only if the data that’s driving it makes sense, but experts say much of the data used to guide reading instruction doesn’t, as Forbes reported.

Classrooms across the country use standardized tests to determine students’ reading levels, identify where they need help, and predict their performance on end-of-year state reading exams. According to a Rand survey, 93% of reading teachers administered some kind of “benchmark” or “interim” assessment during the 2021-22 school year as part of a $1 billion-plus market.

Schools and parents rely on this data more than ever in the wake of the pandemic despite scant evidence the assessments are accurate.

In 2016, EdReports, an organization that reviews curricula, launched a project to conduct such evaluations. But the company recently suspended the effort because it couldn’t get enough publishers to participate. Perhaps some were wary of what the evaluation would reveal about test reliability.

In one study, roughly a thousand students were given four different tests to determine their reading comprehension ability. On average, only 43% of the children identified by one test as poor readers were also identified as poor readers by another test — and the same thing happened when the tests tried to identify the good readers. In other words, the chances of any two tests putting a particular student in the same category were less than half.

That’s because, many experts say, it is impossible to assess comprehension ability in the abstract, as interim assessments — and end-of-year state reading tests—aim to do, as Forbes reported. Comprehension is so intertwined with prior knowledge of a subject that research has shown that the more you know about the topic you’re reading about, the better your comprehension gets. This partly explains why a child may get a high score on one reading assessment and a low score on another.