Originally posted on August 7, 2019, on LindaSuskie.com
In my July 9, 2019, blog post I encouraged using summertime to reflect on your assessment practices, starting with the question, “Why are we assessing?”
Here are the next questions on which I suggest you reflect:
Who are our audiences for the products we’re generating through our assessment processes?
What decisions are they making?
How can the products of our assessment work help them make better decisions?
Your answers to the questions I’ve listed will affect the length, format, and even the vocabulary you use in each product. Consider these products of assessment processes:
Assessment results. The key audience for assessment results should be obvious: the faculty and administrators who need them to make decisions, especially what and how to teach, how to help students learn and succeed, and how best to deploy scarce resources.
Faculty and administrators are always making these decisions. The problem is that many people make those decisions in a “data-free zone.” They make a decision simply because someone thinks they have a good idea or perhaps because a couple of students complained about something.
Today time and resources at virtually every institution are limited. We can no longer afford to make decisions simply because people think they have a good idea. Before plunging ahead with a decision, we first need some evidence that we’ve identified the problem correctly and that our solution has a good chance of solving the problem.
In his book How to Measure Anything, Douglas Hubbard points out we’re not aiming to make infallible decisions, just better decisions than we would without assessment results.
Many reports of assessment results say the results have been used only to make tweaks to assignments and course curricula (“We’ll emphasize this more in class.”) But what if, say, six programs all find that their seniors can’t analyze data well? That calls for another audience: your institution’s academic leadership team. Your institution needs a process to examine assessment results across programs holistically—and probably qualitatively—to identify any pervasive issues and bring them to the attention of academic leaders so they can provide professional development and other support to address those issues across your institution.
All this suggests that you need to involve your audiences in designing your assessments and your reports of assessment results, both to make sure you’re providing the information they need and that it’s in a format they can easily understand and use.)
Learning goals have several audiences. The most important audience is students, because research has shown that many students learn more effectively when they understand what they’re supposed to be learning. Prospective students—those who are considering enrolling in your institution, program, course, or co-curricular learning experience—are another important audience. Key learning goals might help convince them to enroll (“I’ve always wanted to learn that” or “I can see why it would be important to learn that.”). A third audience is potential employers (“These are the skills I’m looking for when I hire people, so I’m going to take a close look at graduate of this program.”). And a fourth audience is potential funding sources such as foundations, donors, and government policymakers (“This institution or program teaches important things, the kinds of skills people need today, so it’s good place to invest our funds.”).
All these audiences need learning goals stated in clear, simple terms that they will easily understand. Academic jargon and complex statements have no place in learning goals.
Strategic and unit goals. These often have two key audiences: the employees who will help accomplish the goals and potential funding sources such as donors. Both need goals stated in clear, simple terms that they will easily understand so they can figure out where the institution or unit is headed, what it will look like in a few years, and how they can help achieve the goals.
Curriculum maps. Curriculum maps are a tool to help faculty (1) analyze the effectiveness of their curricula and (2) identify the best places to assess student achievement of key learning goals. So they need to be designed in ways that help faculty accomplish these quickly and easily.
Student assignments. In many assignments we give students, the implicit audience for their work—be it a paper, presentation, or performance—is us: the faculty or staff member giving the assignment. That doesn’t prepare students well for creating work for other audiences. When I taught first-year writing, one assignment was to write solicitations for gifts to a charity to two different audiences (and a third statement comparing the two). When I taught statistics, I had students write a one-paragraph summary their statistical test addressed to the hypothetical individual who requested the analysis. When I taught a graduate course in educational research methods, I had students not only draft the first three chapters of their theses but deliver mock presentations to a foundation explaining, justifying, and seeking funding for their research.
Documentation of assessment processes (how each learning goal has been assessed). The key audience here is the faculty and staff responsible for the program, course, or other learning experience being assessed. They can use this documentation to avoid reinventing the wheel (“How did we assess this last time?”).
Another audience for documentation of assessment processes is whatever group is overseeing and supporting assessment efforts at your institution, such as an assessment committee. This group can use this documentation to (1) recognize and honor good practices, (2) share those good assessment practices with others at your college, (3) give each program or unit feedback on how well its assessment work meets the characteristics of good assessment practices, and (4) plan professional development to address any pervasive issues they see in how assessment is being done.
Documentation of uses of assessment results. Here again the key audience is the faculty and staff responsible for the program, course, or other learning experience being assessed. They can use this information to track the impact of improvements they’ve attempted (“We tried adding more homework problems but that didn’t help much. Maybe this time we could try incorporating these skills into two other required courses.”).
Why haven’t I mentioned your accreditor as an audience of your assessment products? Accreditors are a potential audience for everything I’ve mentioned here, but they’re a secondary audience. They are most interested in the impact of your assessment products on students, colleagues, and the other audiences I’ve mentioned here. They want to see what you’ve shared with your key audiences—and how those audiences have used what you’ve shared with them. Most of all, they want your summary and candid, forthright analysis of the overall effectiveness of your institution’s or program’s assessment products in helping those audiences make decisions.