[Previous entry: "Blog Conversations"] [Main Index] [Next entry: "Dangerous"]

02/17/2006 Archived Entry: "Evaluations"

The End of Evaluations
After reading Collin's insightful critique of his university's Facebook fiasco, I have realized that we have reached the end of evaluations' efficacy.
By saying that, I mean the online variants which have materialized: Facebook comments, RateMyProfessor, etc. But also the institutional evaluations we cling to for purposes of pay raises, checks and balances, grad student performance, and so on.
Because course/instructor evaluations do nothing. They stroke our egos when they are good - "They like me, they really like me!" - and break our hearts when they are bad - "How could this person say that! I tried so hard!" They very seldom, if ever, teach us about our work, help us get better, or even recognize what teaching means and why professors assign and produce for courses what they do. Evaluations have become revenge tools for imaginary crimes: "I'll teach you for making me do homework." They become signposts of apathy: Many come back in the envelope unfilled in. They reconfirm our sense of self-esteem: Aw. This person liked the assignment where we run around and jump up and down.

When I was WPA, I saw evaluations used as pretext to fire adjuncts. In grad school, bad evaluations became a detriment for continued funding; good evaluations were slapped on to CVs and applications (sometimes like movie blurbs in a newspaper advertisement: "Best course ever!"). Never is there context. Never do we consider the abilities of students to understand the nuances of a profession they don't know anything about. Never do we think about student immaturity. Never do we think about the day to day activities, the forgotten moments of a semester, an instructor's effort and care, misunderstandings, cultural difference, and so on and so on and so on. If anything, evaluations are the moment of no-critical thinking. They are just accepted. As is. Signed, sealed, and delivered.

So listen up, universities. Give up evaluations. Let the online world continue to be stoopid. But we need to move on and realize that this is another level of pedagogy and administration that has run its course and died.

Posted by Nels @ 02/17/2006 03:26 PM EST

Even for wider differences in eval scores, however, I don't understand why anyone would think that these scores are effective measures. I mean, even for the social science types, this strikes me as bad methodology: no control group, no baseline measurments, etc. It's just a big free for all.

It's only after the fact that some schools mark trends in lower/higher scores (for required classes, for classes in one's major, etc.). And I guess we'll just ignore the UT study that came out a few years ago that concluded attractive people get higher scores on average.

If we're going to be all "scientistic" about it, then do it right!

Posted by jenny @ 02/17/2006 01:15 PM EST

Some folks in my department are under pressure because of their low teaching evaluation scores. While the numbers are not the sole means by which we're evaluated, they are important enough that not meeting the minimum standards means frequent discussions with the chair, mentoring, etc.

And I remember Ulmer complaining that he was denied a merit raise because of teaching evaluation scores. (I can't remember if this was the only reason or a contributor.)

No way evaluations are going away, so we're better off either hacking them (free pizza!) or finding better ways to do it.

Posted by cbd @ 02/17/2006 12:53 PM EST

Decisions like that (tenth of a point, etc.) are objectively innumerate, and I've heard of similar situations at Florida, though not in the English Dept.

Posted by Jonathan @ 02/17/2006 12:20 PM EST

At Texas, teaching evaluations were used to decide who got the (badly needed) summer teaching gigs. Teaching evals are commonly used as a deciding factor for funding like that.

Not that anyone would get funding cut off for bad evals, of course. But I had some moments of frustration where I (and others) were not given a requested class because our evals were slightly (tenths of points) lower. Did that mean that the better teachers got the gigs?

Ha. It meant that the "buddy" teachers usually stand out: the ones who make class like a bull session. I know of one instructor who consistently received all 5.0 scores. This same instructor also went to student parties, did personal things w/ students outside of class, etc. Naturally, students rated this person as a "Great" teacher. But it was so sadly far from the truth.

Of course, the computer scores don't know that.

Posted by jenny @ 02/17/2006 12:11 PM EST

It's news to me, and I made a point of studying this issue with the aid of perl scripts. For instance, the average when we began graduate school for "Instructor Overall" in 1101 classes was about 4.0. By 2002, it was nearly 4.5. Was that attributable to better teaching? Grade inflation?

Furthermore, it sounds to me uncharacteristic of the relevant administration during this period.

Posted by Jonathan @ 02/17/2006 11:12 AM EST

"Um...example?"

Not on the blog. Privately, maybe, maybe not. I don't want to name names here. But I knew of folks who were told as such.

Posted by jeff @ 02/17/2006 10:35 AM EST

"In grad school, bad evaluations became a detriment for continued funding"

Um...example?

Posted by Jonathan @ 02/17/2006 10:31 AM EST

Very well said. The first comment in this thread at New Kid's contains a good point: "evaluations are just a measure of how much the student actually likes the professor (as in thinks their nice, smart, etc.) and how happy they are with their own performances in the class." I've also heard that whether or not a student gets along well with *the other students* in the class influences the evaluations. Evaluations offer an impression of teaching persona and rapport; that's about it.

On a forum thread at The Chronicle about a year ago, someone made a comment that really stuck with me. He/she said that all you have to do to get good student evaluations is: Assign little to no reading, show lots of videos in class, give everyone A's, and bring snacks to class. The formula produces the desired result, and the teacher is rewarded -- with tenure, contract renewal, teaching awards, etc.

Another poster in that thread at New Kid's said:

Most of the people I know with good teaching evaluations are either: 1. very attractive and flirty (and the students swoon) or 2. sensationalist and pop culture-y. Often they are 1 and 2.

I once taught for a professor that made jokes about Britney Spears and J. Lo, knowing that if she kept her enrollment high enough, she would be excused from teaching a lecture course the following year (yes, that's how we reward 'good' teaching at my institution). She was basically a glorified babysitter, and she knew it, so she cashed it in. She was silly and fun and approachable, which can all be great things to be as a professor... but she taught very little useful content and I was frustrated as a TA with the low quality of the course, her reductionist and simple analysis of very subtle and interesting material, and the fact that it was obvious to us as TAs that she had no real investment in teaching.

But she's won all sorts of awards.

And that's how it works.

Posted by Clancy @ 02/17/2006 09:56 AM EST