love that!

smartchica47 — Mon, 12 May 2008 23:54:21 -0500

I love the idea of the mid-quarter (or in my university's case, mid-semester) review! I love even more the idea of teaching in a place where faculty actually use such a tool! On my campus, I imagine that relatively few faculty would take advantage of such an option, except perhaps junior faculty trying to make a good impression for tenure. But I will be asking our Center for Teaching and Learning about this nonetheless...

My University just moved to doing online evaluations - students can't access their grades through the online system until they complete the course evaluation (I assume that's supposed to increase the response rate). I believe we can learn a lot from evaluations but as with many things that might improve teaching, there is a lot of self-selection - that is, teachers who care about being good teachers will use whatever information they can to improve their teaching while those who don't really care about being good teachers will just come up with reasons why the information isn't worth anything...

Jenn

http://economicsforteachers.blogspot.com

Hoo boy--it's course evaluation season

Leslie Madsen Brooks — Wed, 07 May 2008 23:34:46 -0500

It's the end of the academic year in the U.S., and that means it's time for course evaluations. Course evaluations are controversial little beasts, and both student and faculty bloggers have plenty to say about them. Herein I share some of their laments and offer some tips on conducting a course (or program) evaluation that's really meaningful.

Lauren expresses a concern that professors won't look at course evaluations, particularly for large lecture courses where they don't interact much with students. She also worries about the timing of course evaluations:

The timing of the evaluations is not chosen wisely. At the end of a semester students are just finalizing their last projects, tests, and papers. Grades for the most part have already been determined. Why would a student care about shedding light on a professor’s inadequacies when nothing they say will better the experience they will take from the class. The class is over; any complaints will just be a waste of time and energy. Evaluations should be a dynamic process that is available to students throughout the suration of the class. Feedback should be always provided to the instructor if a class if to be effective.

It is possible for professors to ignore course evaluations, which is too bad, because in my experience there's usually something there--a thoughtful critique or an expression of gratitude for a job well done--that makes them worth a read.

I agree with Lauren that the timing of course evaluations is bad: at the end of the term, everyone is tired and it can be hard to motivate students to take the time to write something meaningful on the evaluations. Worse, students' grades in the class often influence their opinion of a professor's skills. So if a student learns that she is getting a C in a course, it's likely--unless the professor is truly exceptional--that she will perceive the professor as below average. It's entirely possible that the professor in this example is below average--perhaps she couldn't teach the students well enough for them to earn As, or she wrote tests that didn't accurately assess what students were supposed to be learning in the course, or her grading methods were unfair. It's also possible that this professor is a pretty good instructor--but we won't necessarily know that by looking at end-of-course evaluations because these assessments tend to be colored by (a) a student's grade and (b) whether or not a student likes the professor's personality.

Craig Monk, a faculty member, dislikes the anonymity of student evaluations of faculty:

If you have a problem with a course and the way it is taught, you should always talk to the instructor about it while the course is ongoing. I have never understood why students fear reprisals in such a situation. How is it that we dignify such suspicions by structuring a whole system to protect people from something we only assume might happen? If a student with a complaint was ever somehow punished for complaining, mechanisms already exist to investigate and reverse any manner of asymmetrical treatment of individuals in the classroom. If the problem cannot be resolved, ultimately, if it is the kind of thing that should be reflected in a course evaluation, it is the principled thing to do to sign that evaluation, even though the instructor will not (and should not) see that signature.

It is my personal opinion, and only my personal opinion, that we should always require signatures. If we are going to say things about people, we should have to put our names to those things. We should send our students out into the world respecting the power of their opinions and better respecting those people on whom those opinions are brought to bear.

I'm not sure I need to have students sign their course evaluations, but it would be nice to see, in the aggregate, whether positive evaluations in my course correlated to high grades and more reserved evaluations associated with lower grades. I think anonymity is especially important in the case of students majoring in a discipline who may have the same professor again. Monk suggests student signatures should be removed from the evaluations faculty see, but there's a danger that evaluations to "leak" into the hands of the professor or her colleagues, thus jeopardizing students' right to comment freely without fear of reprisal in future courses.

The departments where I have worked have allowed us to see the original handwritten evaluations. In this digital age, the chances of me identifying a student's handwriting are slim, so I don't feel students' anonymity is being compromised. It's fun to imagine, however, which students heaped praise upon me, and I have to admit I even enjoy thoughtful critiques because ZOMG! at those moments students are thinking about learning. But every batch of course evaluations also includes maddening "evaluations," off-handed comments that make you furrow your brow or gasp in horror. For example: "the professor is biased towards those who did the readings" and "Professor Kibitzer makes me feel like I am stupit."

That said, I never have wanted to sue my students after reading their course evaluations. (The professor, Priya Venkatesan, has since dropped the lawsuit. That case, in which it appears professor and students really distrusted and mistreated one another, deserves its own blog post--and the professor thinks her experience is book-worthy. Oy.)

Of course, formal course evaluations aren't the only way students rate courses--there's also the infamous Rate My Professors, where students can post whatever they'd like to say about faculty, even if they never even attended the courses in question. Most faculty don't take the site seriously, but Lucky Jane thinks it has its merits:

we wandered onto the subject of Rate My Professors.

I’ll never forget their looks of horror when I defended the site, but I’d do it again. I know, I know it has no credibility, that it exists for disgruntled, anonymous students to indulge in their disgruntlement and anonymity as they trash their professors publicly, by name and institutional affiliation. Just as there’s no safeguard against one disgruntled student posting dozens of libelous reviews, so there’s no need even to have completed a course with a particular faculty member to post a review. Take one of my favorite mentors, a scholar-teacher who taught me everything I know. On RMP she is represented by a blue frown and someone who boasts of having dropped her class after the first day, because she said “um” so often as to indicate that she lacked the eloquence necessary to profess English. (I can’t help picturing that poster’s righteous head toss.) But then no one’s checking to make sure raters attended the class for even one day, or attends that institution at all, for that matter. That “ease” and “hottness” are on equal footing with “quality” speaks volumes about who uses the site: generally, stupid people rely on these criteria, and I don’t want stupid people in my classes.

Still, on a campus as vast as ours, students need—they deserve—a way to share opinions about us. College is expensive, and in terms of time more than money.

Click through the read all the reasons she feels the site is worth defending.

More suspect, I think, than Rate My Professors is Pick-A-Prof, which posts the grade distributions in courses taught by a given professor. Nice, eh? Now if you want to go to medical school, you can practically assure your 4.0 GPA by taking courses that don't challenge you or in which professors inflate grades relative to their colleagues. Pick-A-Prof came on the scene just as I was ending my last stint in the undergraduate classroom, so . What can you learn from Pick-A-Prof? Not much. Depending on your POV, the stats at that link tell you that a) I'm generous with grades (lots of As and Bs), b) it's hard to get an A in the survey course I teach, c) my TAs in the survey course grade too harshly, d) I instruct my TAs to give a lot of Bs, and/or e) my students in my upper-division course were How many of these are true? You'll never know. And honestly, I don't want students in my course who use Pick-A-Prof. If you're the kind of student who shops around for As and who sees Bs as a "bad" grade, then you're not likely to enjoy my efforts to educate you.

So if sites like Rate My Professors are so popular, why don't more universities offer evaluations online? (Hint: low response rates.) There's a lively discussion at FemaleScienceProfessor about the ins and outs of online and in-person course evaluations administered, in the case of FSP, by a "hostile zombie."

One partial solution to the ennui engendered by end-of-course evaluations is administering what my institution calls a "mid-quarter interview" or MQI. I didn't come up with the MQI system, but it's pretty damn brilliant. Here's how it works: A professor requests that a teaching consultant (moi) come to the classroom to chat with her students for 15-20 minutes. I come to the class with brief evaluation forms and pass them out to students. The students answer the first two questions on their own: "What do you most appreciate about this course or instructor?" and "What do you least appreciate about this course or instructor?"

Students then form small groups to brainstorm responses to the final question: "What can the instructor do to increase your ability or motivation to learn in this course?" I list nine or ten of these suggestions on the board, and then individual students indicate whether they agree, disagree, or have no opinion about each suggestion. I take these results back to my office and calculate the results, noting where significant numbers of students agreed or disagreed with a statement.

I then sit down with the professor and we chat for half an hour about how the course is going and whether the students' suggestions are worthy of implementing, based on what we know about students at our university and sound pedagogical practices. Students often provide excellent suggestions, but some of them, while well-meaning, are not so good. Students are not always savvy about their own learning styles.

Regardless of whether they make the changes recommended by students (and I think many faculty do respond positively to student suggestions), faculty tend to receive better end-of-course evaluations when they request an MQI than they do when they go without an MQI. In short, students perceive faculty who invite teaching consultants into the classroom to be better teachers than those who don't.

What are your thoughts on course evaluations? Should course evaluation data--or grades--be made public? Why or why not?

Leslie Madsen-Brooks helps university faculty improve their teaching. She blogs at The Clutter Museum, Museum Blogging, and The Multicultural Toy Box.

BlogHer - Hoo boy--it's course evaluation season - Comments

love that!

Hoo boy--it's course evaluation season