What is good assessment, revisited

 Originally posted on April 17, 2019, on LindaSuskie.com

Another week, another critique of assessment, this one at the Academic Resource Conference of the WASC Senior College and University Commission.

The fundamental issue is that, more than a quarter century into the higher ed assessment movement, we still aren’t doing assessment very well. So this may be a good time to reconsider, “What is good assessment?”

A lot of people continue to point to the nine Principles of Good Practice for Assessing Student Learning developed by the old American Association for Higher Education back in 1992. In fact, NILOA once published a statement that averred that they are “aging nicely.” I’ve never liked them, however. One reason is that they combine principles of good assessment practice with principles of good assessment results without distinguishing the two. Another is that nine principles are, I think, too many—I’d rather everyone focus on just a few fundamental principles.

But most important, I think they don’t focus on the right things. They overemphasize some minor traits of good assessment (I’ve seen plenty of good assessments conducted without much student involvement, for example) and are silent on some important ones. They say nothing, for example, about the need for assessment to be cost-effective, and I think that omission is a big reason why assessment is under fire today. A year ago, for example, I did a content analysis of comments posted in response to two critiques of assessment published in the Chronicle of Higher Education and the New York Times. Almost 40% of the comments talked about what a waste of time and resources assessment work is.

When I was Director of AAHE’s Assessment Forum in 1999-2000, I argued that it was time to update them, to no avail. In the mid-2000s, I did a lit review of principles of good assessment practice. (You’d be amazed how many there are!) I created a new model of just five principles, which I presented at several conferences. Good assessment practices:

• Lead to results that are useful and used.
• Flow from and focus on clear and important goals.
• Are cost-effective, yielding results that are useful enough to be worth the time and resources invested.
• Yield reasonably accurate and truthful results.
• Are valued.

These are not discrete, of course, so since I’ve developed this model, I’ve played around with it. About five years ago I took it down to two principles. Under this model, good assessment practices:

• Yield results that are used in meaningful ways to improve teaching and learning. This can only happen if assessment practices focus on clear and important goals and yield reasonably accurate and truthful results. And using assessment results to inform meaningful decisions is the best way to show that assessment work is valued.
• Are sustained and pervasive. This can only happen if assessment practices are cost-effective and are valued.

While I like the simplicity of this model, it buries the idea that assessments should be cost-effective, which we really need to highlight. Today when I do presentations on good assessment, I present the following four traits, because these are the traits that we most need to focus on most today. Good assessment practices:

• Lead to results that are useful and used. This is what psychometricians call consequential validity. I continue to think that this is **THE** most important characteristic of effective assessment practices—all other traits of good assessment practice flow from this one. One corollary, for example, is that assessment results must be conveyed clearly, succinctly, and meaningfully, in ways that facilitate decision-making.
• Flow from and focus on clear and important goals. While this is a corollary of the useful-and-used principle, this is so important, and so frequently a shortcoming of current assessment practices, that I highlight it separately. Learning goals need to be not only clear but relevant to students, employers, and society. They represent not what we want to teach but what students most need to learn. And those goals are treated as promises to students, employers, and society; if you pass this course or graduate, you will be able to do these things, and we will use assessments to make sure.
• Are cost-effective, yielding results that are useful enough to be worth the time and resources invested. This is a major shortcoming of many current assessment practices. They suck up enormous amounts of time and dollars, and whatever is learned from them just isn’t worth the time and money invested.
• Are part of everyday life of the college community. In other words, the culture is one of collaboration and evidence-informed planning and decision making.