Why are you assessing?

 Originally posted on July 9, 2019, on LindaSuskie.com

Summer is a great time to reflect on and possibly rethink your assessment practices. I’m a big believer in form following function, so I think the first question to reflect on should be, “Why are we doing this?” You can then reflect on how well your assessment practices achieve those purposes.

In Chapter 6 of my book Assessing Student Learning I present three purposes of assessment. Its fundamental purpose is, of course, giving students the best possible education. Assessment accomplishes this by giving faculty and staff feedback on what is and isn’t working and insight into changes that might help students learn and succeed even more effectively.

The second purpose of assessment is what I call stewardship. All colleges run on other people’s money, including tuition and fees paid by students and their families, government funds paid by taxpayers, and scholarships paid by donors. All these people deserve assurance that your college will be a wise steward of their resources, spending those resources prudently, effectively, and judiciously. Stewardship includes using good-quality evidence of student learning to help inform decisions on how those resources are spent, including how everyone spends their time. Does service learning really help develop students’ commitment to a life of service? Does the gen ed curriculum really help improve students’ critical thinking skills? Does the math requirement really help students analyze data? And are the improvements big enough to warrant the time and effort faculty and staff put into developing and delivering these learning experiences?

The third purpose of assessment is accountability: assuring your stakeholders of the effectiveness of your college, program, service, or initiative. Stakeholders include current and prospective students and their families, employers, government policy makers, alumni, taxpayers, governing board members…and, yes, accreditors. Accountability includes sharing both successes and steps being taken to make appropriate, evidence-based improvements.

So your answers to “Why are we doing this?” will probably be variations on the following themes, all of which require good-quality assessment evidence:

• We want to understand what is and isn’t working and what changes might help students learn and succeed even more effectively.

• We want to understand if what we’re doing has the desired impact on student learning and success and whether the impact is enough to justify the time and resources we’re investing.

• Our stakeholders deserve to see our successes in helping students learn and succeed and what we’re doing to improve student learning and success.

Why do I assess?

Originally posted on January 31, 2019, on LindaSuskie.com

Last year was not one of the best for higher ed assessment. A couple of very negative opinion pieces got a lot of traction among higher ed people who had been wanting to say, “See? Assessment is really as stupid and pointless as I’ve always thought it was.” At some American universities, this was a major setback on assessment progress.

The higher ed assessment community came together quickly with a response that I was proud to contribute to. But now that we’re in 2019, perhaps it would help if each of us in the assessment community reflects on why we’re here. Here’s my story, in three parts, about why assessment is my passion.

The first part is that I’m a data geek, so I find assessment fun. My first job out of grad school was in institutional research, and my favorite part of the job was getting a printout of student survey results and poring over it, trying to find the story in the numbers (to me it’s a treasure hunt), and sharing that story with others in ways that would get them excited about either feeling good about what’s going well or doing something about areas of concern.

The second part is that I love to teach. I’m not a great teacher, but I want to be the best teacher I can. I’ve always looked forward to seeing how my students do on tests and assignments. I can’t wait to tally up how they did on each test question or rubric criterion (that’s the data geek part of me). I cheer the parts they did well on and reflect on the parts where they didn’t. Why did so many miss Question 12? Can I do anything to help them do better, either during what’s left of this class or in the next one? If I can’t figure out what happened, I ask my students at the next class and, trust me, they’re happy to tell me how I screwed up!

The final reason that assessment is my passion is that I’m convinced that part of the answer to the world’s problems today is to help everyone get the best possible education. This dawned on me about 25 years ago, when I was on an accreditation team visiting a seminary. The seminary’s purpose was to educate church pastors (as opposed to, say, researchers or scholars). It was doing a thorough job educating students on doctrine, but there was very little in the curriculum on preparing students to help church members and others hear what Christians call the Good News. There was little attention to helping students develop skills to listen to and counsel church members, communicate with people of diverse backgrounds, and assess community needs, not to mention the practical skills of running a church such as budgeting and fundraising. While I’m not one to push my faith on others, I think the world might be a better place if people truly understood and truly followed the teachings of many faiths. If that’s the case, the world needs pastors well-prepared to do this. The seminary I visited had, I thought, a moral obligation to ensure—through assessment—that its graduates are prepared to be the best possible pastors, with all the skills that pastors need.

Since then, I’ve felt the same about many other colleges and many other disciplines. The world needs great teachers, nurses, lawyers, accountants, and artists. When I’ve visited U.S. service academies, I’m reminded that the U.S. needs great military officers.

Even more, the world needs people who can do all the things we promise in our gen ed curricula. The world needs people who can think critically, who recognize and avoid unethical behavior, who are open to new ideas, who can work with people from diverse backgrounds, who can evaluate the quality of evidence, arguments or claims, who are committed to serving their communities. Again I’m convinced that the world would be a far better place if everyone could do these things well.

None of us can change the world alone. But each of us can do our best with the students in our orbit, trying our best to make sure—through decent-quality assessments—that they’ve really learned what’s most important. Whenever anyone looks at the results of any assessment, be it a class quiz or a college-wide assessment, and uses those results to change what or how they teach, at least some students will get a better education as a result.

We need those better-educated students. This is what drives me. This is why I am devoting my life to helping others learn how to assess student learning. Assessment is one way each of us can help make the world a better place.

Is assessment worth it?

 Originally posted on August 14, 2018, on LindaSuskie.com

A while back, a faculty member teaching in a community college career program told me, “I don’t need to assess. I know what my students are having problems with—math.”

Well, maybe so, but I’ve found that my perceptions often don’t match reality, and systematic evidence gives me better insight. Let me give you a couple of examples.

Example #1: You may have noticed that my website blog page has an index of sorts on the right side. I created it a few months ago, and what I found really surprised me. I aim for practical advice on the kinds of assessment issues that people commonly face. Beforehand I’d been feeling pretty good about the range and relevance of assessment topics that I’d covered. The index showed that, yes, I’d done lots of posts on how to assess and specifically on rubrics, a pet interest of mine. I was pleasantly surprised by the number of posts I’d done on sharing and using results.

But what shocked me was how little I’d written on assessment culture: only four posts in five years! Compare that with seventeen posts on curriculum design and teaching. Assessment culture is an enormous issue for assessment practitioners. Now knowing the short shrift I’d been giving it, I’ve written several more blog posts related to assessment culture, bring the total to ten (including this post).

Example #2: Earlier this summer I noticed that some of the flowering plants in my backyard weren’t blooming much. I did a shade study: one sunny day when I was home all day, every hour I made notes on which plants were in sun and which were in shade. I’d done this about five years ago but, as with the blog index, the results shocked me; some trees and shrubs had grown a lot bigger in five years and consequently some spots in my yard were now almost entirely in shade. No wonder those flowers didn’t bloom! I’ll be moving around a lot of perennials this fall to get them into sunnier spots.

So, yes, I’m a big fan of using systematic evidence to inform decisions. I’ve seen too often that our perceptions may not match reality.

But let’s go back to that professor whose students were having problems with math and give him the benefit of the doubt—maybe he’s right. My question to him was, “What are you doing about it?” The response was a shoulder shrug. His was one of many institutions with an assessment office but no faculty teaching-learning center. In other words, they’re investing more in assessment than in teaching. He had nowhere to turn for help.

My point here is that assessment is worthwhile only if the results are used to make meaningful improvements to curricula and teaching methods. Furthermore, assessment work is worthwhile only if the impact is in proportion to the time and effort spent on the assessment. I recently worked with an institution that undertook an elaborate assessment of three general education learning outcomes, in which student artifacts were sampled from a variety of courses and scored by a committee of trained reviewers. The results were pretty dismal—on average only about two thirds of students were deemed “proficient” on the competencies’ traits. But the institutional community is apparently unwilling to engage with this evidence, so nothing will be done beyond repeating the assessment in a couple of years. Such an assessment is far from worthwhile; it’s a waste of everyone’s time.

This institution is hardly alone. When I was working on the new 3rd edition of my book Assessing Student Learning: A Common Sense Guide, I searched far and wide for examples of assessments whose results led to broad-based change and found only a handful. Overwhelmingly, the changes I see are what I call minor tweaks, such as rewriting an assignment or adding more homework. These changes can be good—collectively they can add up to a sizable impact. But the assessments leading to these kinds of changes are worthwhile only if they’re very simple, quick assessments in proportion to the minor tweaks they bring about.

So is assessment worth it? It’s a mixed bag. On one hand, the time and effort devoted to some assessments aren’t worth it—the findings don’t have much impact. On the other hand, however, I remain convinced of the value of using systematic evidence to inform decisions affecting student learning. Assessment has enormous potential to move us from providing a good education to providing a truly great education. The keys to achieving this are commitments to (1) making that good-to-great transformation, (2) using systematic evidence to inform decisions large and small, and (3) doing only assessments whose impact is likely to be in proportion to the time, effort, and resources spent on them.