Taking an educated approach to evaluating interim assessments
by
In 2022, the International Society for Technology in Education (ISTE) launched the Benchmark project, which aimed to explore how edtech buyers select digital assessment products and develop resources to support that process. One of the project's goals was to explore if the ISTE Seal, a product certification that showcases edtech products aligned with ISTE Standards and research-based usability indicators, could be applied to interim or benchmark assessment products. The terms “interim” and “benchmark” are often used interchangeably to describe periodic assessments used to compare groups of students by measuring them on a common set of skills or content standards.
As part of the evaluation, ISTE partnered with Pearson assessment experts and selected the aimswebPlusTM product, which includes multiple benchmark administrations throughout the schoolyear and more frequent progress monitoring measures in between as a part of a holistic MTSS solution. After pressure testing the Seal framework with interim assessment experts and reviewing the test design and characteristics of the aimswebPlus interim/benchmark assessments, the article outlines several factors that make formal evaluation of these products challenging. Interim/benchmark assessments:
- Are typically adaptive, with complex mechanisms underlying their adaptability that are difficult for reviewers to fully understand.
- Have rigorous psychometric requirements, which would be difficult for most product reviewers to properly evaluate.
- Often include substantial "under the hood" proprietary functionality that is not easily visible or shareable.
Given these unique characteristics of these assessment products, the current ISTE Seal framework and evaluation process may not be the best fit, however, school and district leaders still need guidance on selecting appropriate interim assessment tools.
When evaluation interim assessments, it’s important to:
- Evaluate how interim assessments connect to curricular content, measurement standards, and student growth/outcomes. Helpful questions to ask include "How does the product guide instructional decisions?" and "What does the product measure and what should teachers do with those measures?"
- Understand that evaluating the usability of interim assessment products requires specialized expertise in areas like assessment design, item creation, curricular alignment, and data-informed instruction. Schools and districts should involve stakeholders with this expertise in the procurement process.
- Treat interim assessment products as a long-term, strategic investment rather than a quick fix. Implementing these tools effectively requires a multi-year effort to train staff, integrate with existing curricula, and collect meaningful longitudinal data on student learning outcomes.
Key takeaways
When evaluating interim assessment products, decision-makers must acknowledge the complexities involved, including adaptive testing, psychometric rigor, and alignment with curricular standards. By adopting a long-term, strategic approach and involving assessment experts, schools and districts can more effectively leverage interim assessment products to inform instruction and impact student learning outcomes.
There are a few resources to consider when evaluating EdTech products for your school or district that are intended to help buyers understand quality indicators and make more informed purchasing decisions.