The Chronicle has the report here, highlighting new research showing bias and irrelevance of the old evaluations and new efforts to replace them with meaningful feedback from students and colleagues, A snippet:
What sparks this kind of change? Growing concern about the inequity of student course evaluations has inspired some campuses to start there, either rewriting them in ways that make them more useful or reducing their weight in determining raises and promotions. That work frequently opens the door to deeper conversations in departments and across campus about how to create a culture of teaching excellence.
University of Oregon leaders took this approach, scrapping the traditional course evaluations in favor of a new instrument called the Student Experience Survey. They created new teaching-evaluation standards, grouping them into four categories — professional, inclusive, engaged, and research-informed — and made sure the questions on the student survey aligned with those categories. And they created new tools for peer review and self-reflection.
Lee Rumbarger, associate vice provost for teaching engagement, notes that this was a multi-year process starting with the Office of the Provost and the University Senate, then moving out into colleges and departments.
For the record, the process started with two of my Economics honors students, Emily Wu and Ken Ancell. A previous Chronicle report notes:
“Having a female instructor is correlated with higher student achievement,” Wu said, but female instructors received systematically lower course evaluations. In looking at prerequisite courses, the two researchers found a negative correlation between students’ evaluations and learning. “If you took the prerequisite class from a professor with high student teaching evaluations,” Harbaugh said, “you were likely, everything else equal, to do worse in the second class.”
The team found numerous studies with similar findings. “It replicates what many, many other people found,” said Harbaugh. “But to see it at my own university, I sort of felt like I had to do something about it.”
He did. In the spring of 2017, Harbaugh assembled a task force on the issue and invited Sierra Dawson, now associate vice provost for academic affairs, to join. The last time that course evaluations had been reviewed was a decade earlier, when the university moved from a paper system to online.
This academic research result would have gone nowhere, however, without the enthusiastic support and hard work of Dawson and Rumbarger from the Provost’s office who worked with the UO Senate to come up with a new system – which asks students about the use of specific teaching methods, not about how they like the instructor – or without the somewhat less eager but essential support of then UO President Michael Schill.
So how’s it going? From The Chronicle:
Samantha Hopkins, head of the department of earth sciences at Oregon, sees those barriers on her campus.
Faculty members are largely happy with the reforms to the course evaluations and other changes that have made evaluations more substantive, she says. But administrators accustomed to the numbers-driven systems are finding the new process challenging. “I’ve heard a lot of people expressing a feeling that they miss the student evaluations, and they don’t like the student-experience surveys as much because it’s so much harder to pull an assessment of someone’s teaching out of it,” she says.
Hopkins doesn’t miss the old system but understands the feeling: It is so much easier to compare numbers to numbers. “It’s something I’m struggling with right now,” during annual evaluation time, she says. “It’s the challenge of looking at what people are doing and saying: Is this good enough? What is good enough?”
And then there are the old, hard-to-budge hierarchies. “You hear a lot of lip service given to the importance of teaching,” she says. “But really when it comes down to it, so much of university culture is really centered around the importance of research.”
She recalls a conversation she had with a senior administrator who objected to the idea that a senior instructional faculty member should make as much as assistant professors. It’s a view widely held across campus, she says. “This idea that someone who does only teaching and not research can’t make as much as someone who does research, even the most junior member of the research faculty, tells you where they are actually putting their money.”