Your K-12 district runs a lot of assessments. Are they valid and reliable?
by
When it comes to assessments, the first question K-12 leaders must confront is: Do we have confidence in the results?
Schools conduct a range of assessments — including summative, formative, progress-monitoring and interim/benchmarking assessments. But as a new national survey from Pearson Assessments and K-12 Dive’s studioID shows, few K-12 leaders and educators put a lot of stock in the results.
Only 38% of the survey respondents strongly agree that their assessment tools allow them to pinpoint student needs. Just 37% report that assessments enable them to personalize instruction. And less than half (44%) say they can measure students’ progress toward specific learning targets.
These findings indicate challenges for students and educators alike. For example, formative assessments should guide instruction and shape student learning. But to do that, educators need to have confidence in their results.
The importance of valid and reliable results
To serve the teacher and the student, assessment results must be valid (accurate) and reliable (consistent). Otherwise, educators can’t make instructional decisions based on sound evidence.
Some educators — and assessment providers — discount the validity and reliability of classroom formative assessments. Such assessments “are often considered ‘low stakes,’” a peer-reviewed study noted, cautioning that “they need not be dismissed as being of ‘low quality.’”
The truth is, educators should have the most confidence in formative assessments. Why? Because they inform instruction and affect the trajectory of student learning. Summative assessments, by contrast, measure learning after it has happened and deliver scores according to the state’s grade-level expectations.
That’s why it’s so troubling that educators in our survey said they feel they can’t pinpoint student needs, gauge progress toward learning targets or personalize instruction based on the assessments available to them.
In short, they don’t trust that their assessments produce valid and reliable data at the granular level needed to effectively influence instruction.
The questions to ask when vetting assessments
How can K-12 leaders better support teachers and students with assessments? The process starts by doing your homework on assessments to find out which measures are valid and reliable, and not just at a high level.
Amy Reilly is an assessment expert who serves as Vice President at Pearson School Assessments, which has been a leader in the field for more than 80 years. Reilly advises K-12 leaders to ask pointed questions of assessment providers. For example:
- How valid and reliable are your assessment results down to the level of the state standards and the skills reflected in those standards? Reilly explains: “Educators might look at a percentage correct on a formative assessment and think, ‘Great, that score shows proficiency.’ But a ranking doesn’t tell you what standards the student learned. It doesn’t point to specific areas where the student may need support.”
- Do these assessment tools align with our state’s standards? “Some tools align just to Common Core or a general set of skills,” Reilly says. “Then you end up with a disconnect between what you’re teaching and what you’re testing students on.”
- Does each assessment tool use a principled approach to design and development? And is it built on data as well as a sound measurement model that supports the intended use? Reilly notes, “Assessments vary based on their purpose, and often their results can be used in ways they weren’t intended.” Make sure each tool is purpose-built to achieve its particular goal.
- Can the teacher administering an assessment gauge its rigor? A common problem with assessments, Reilly explains, is that they don’t give the teacher enough context. Which questions were easy and which were difficult? Knowing that information helps them better understand and take action based on the results.
- Do the classroom assessments include enough items to provide useful learning evidence? Some assessments are too short to yield valid and reliable data, and educators may need to reconsider whether and how they are deployed to students.
In addition to staying in communication with assessment providers to help better understand what their tools are designed to measure and how, K-12 leaders should consult a wide variety of other resources when selecting assessments. Other steps to take include:
- Talk to districts that have seen proven results with assessments.
- Look at third-party research into assessments.
- Consult external experts who can point out pitfalls to avoid.
Support educators by making the most of your assessments
Assessments are expected to become a bigger part of classroom learning going forward. In fact, states now require a broad range of assessments. That means now is the time to ensure your educators can leverage assessments in the way that matters most: to improve student learning.
Learn more about Pearson’s assessment solutions by visiting the Pearson Assessment for Learning Suite.