Provost's Corner: Teaching is a Risky Business
|Scott G. McNall|
I initially enrolled in their classes because they were offered at convenient times and because I had to get my humanities and social science requirements out of the way. I ended up taking as many electives as I could from these two.
I enjoyed the class in modern literature because the selection of texts resonated with my interests at the time and because the professor welcomed my participation in class. He knew my name, talked to me after class, and invited me to his apartment to talk about what he loved. My psychology professor was one of the few to escape from Auschwitz, and he told us how he did it. He ran the large introductory class as an experiment, sending half of the class away to read the book and lecturing to the other halfto see who would do best on the tests. He said enigmatic things in classfirst in German and then in Englishlike, "When the sun rises on the dung heap in the morning, either the cock crows or it doesn't." I also took an experimental psychology class from him. Class time was spent proofreading his articles or talking about things that had nothing to do with experimental psychology. I loved that professor and I learned a lot of psychology.
I have often wondered just how my two favorite teachers would fare on Chico's SEF. I suspect both would have scored rather poorly because one was excruciatingly shy, throwing out questions during the class hour with the desperate hope that somebody would respond so he wouldn't have to talk; the other professor talked but mostly to himself and didn't seem to care too much whether you listened or not. In an educational age in which many students expect to be captivated, entertained, and lured into learning, and are also provided an opportunity to comment anonymously on the performance of teachers, we need to consider carefully how teaching can be evaluated.
This last spring the Academic Senate passed a resolution that requires the entire campus to use a new Student Instructional Report (called SIR II) on an experimental basis for a period of two years. We will begin in spring 1999. SIR II replaces our old SEF.
There were compelling reasons to cast aside the SEF. The SEF had not been validated, i.e., there was no necessary relationship between student learning and evaluation of the professor's performance. A campuswide committee, under the able leadership of Professor Edward Vela, reviewed many alternative ways of measuring student perception of faculty effectiveness to see if one of them would suit our purposes. They finally recommended SIR II because it has been widely used, has been validated, and provides mean comparative figures for institutions like ours.
Thus, when you receive a score on one of the standard areas of assessment, you will have data on our own institution, as well as data from others. The areas to be evaluated are standard: course organization and planning; communication; degree and type of student-faculty interaction; nature and clarity of assignments; use of supplementary instructional methods; course outcomes; student effort; and a single overall evaluation score. You will also have the option of adding ten questions of your own.
The SIR II comes with a caveat, as do most instruments of this nature. It says that it should not be used as the sole basis for evaluating teaching effectiveness. Our own academic senate has also noted that student evaluations of faculty should be weighted along with other contributions to teaching, having deleted from the FPPP the previous reference that said that student evaluations should count for no more than 25 percent in assessing teaching effectiveness. However, the reality is that at Chico we use SEFs as if they counted for almost 100 percent. We do so because we have no other systematic way of evaluating teaching effectiveness.
I want to raise two questions. The first has to do with whether or not student evaluations of faculty should be used at all, and the second has to do with what we might offer as either a supplement or a substitute.
The use of student evaluations of faculty is widespread, with over 88 percent of all colleges and universities in the country using them. SEFs are used because they are simple to administer, are less costly than other forms of evaluation, and, in my opinion, provide an illusion of objectivity in an era when everyone is being held accountable. Promotion, tenure, and retention decisions can hinge on the scores students give professors.
Teaching is hard work and it can be risky work. Let us consider the new colleagues who have joined us over the course of the last two or three years. For some, this has been their first introduction to handling four courses at once and to teaching in a university that takes seriously the injunction that students come first. Teaching requires experimentation; it requires using different techniques to respond to the diversity of intellectual talent that students bring to the classroom; and it requires time to determine how to create the kind of environments that will maximize student learning.
In short, teaching requires us knowingly to take risks. Imagine, then, a first set of student evaluations that are strongly critical of a professor's performance and further imagine that it is pointed out to the new professor that the scores fall below the department or college "norm." The lesson many learn from such an experience is that they had better improve their scores and they had better do what everybody else is doing. This is a misuse of the SEF, as it discourages innovation. We must guard against defining teaching as that which is measured solely by an evaluation form; otherwise, teaching will shrink to fit the definition offered by the form.
Does this mean we should never use student evaluations of faculty? No. It does mean they need to be used with caution. There is a rich literature on student evaluation of faculty. In general, most researchers believe student evaluations are reliable, valid, and free from bias. The latter means students would be able to judge the merit and worth of an instructor's effort regardless of the grade they received or expected. Other findings are less encouraging. Will high grades produce higher student evaluations? The general answer is "yes." There are other "positive" factors which are likely to increase student ratings. Generally speaking, you would want to teach classes with fifteen or fewer students; classes in the major as opposed to beginning classes, which everyone needs to take; classes in which the course expectations are very clear; and classes about which you are very "enthusiastic," if you want high evaluations.
In fact, one recent study (Williams and Ceti, 1997) suggests that how students perceive you is more important than what they learn, at least in terms of filling out an evaluation form. Williams and Ceti noted the work of colleagues who had found that students form their opinion of a professor in the first few minutes. Building on this idea, they wanted to know if, by changing students' perceptions of the professor, they could change students' perceptions of how much they learned and whether or not they would give the course a higher rating. They worked with teachers to improve the teachers' presentation style, while at the same time making sure the teachers used exactly the same textual material and provided the same lectures. The result was that even though nothing had changed, except how teachers acted, a new group of students thought they had learned more. This may call into question the ability of students to evaluate accurately how much they have learned.
Some have argued that students should not be allowed to evaluate faculty at all, finding student evaluations to be pernicious, striking at the very heart of education. Whether the writer is pointing to what some have called "Generation X," or just students in general, it is said that students come wanting to be entertained, wanting it easy, and wanting an A. And if they do not get these things they will strike back by giving a professor a negative evaluation. The message: Dumb things down, give higher grades, get higher evaluations, get promoted and tenured.
Once again, things are not that simple. As, William McKeatchie, dean of evaluation researchers, has pointed out, there is a strong relationship between the scores students give professors and the scores that their peers or neutral observers give. We absolutely must not reify evaluation scores, says McKeatchie, any more than we would reify an I.Q. score. What we should do is use evaluations to help people improve their teaching. If 60 percent of a group of students says that the instructor didn't make the course requirements clear, was late to class, was disorganized, was rude to the students, and was boring, there is probably a pattern that needs to be corrected. It also means that if one's peers sat in the classroom they would probably come to the same conclusions. Teaching evaluations and scores need to be contextualized;we need to take into account the nature of the course, its size, and the diversity of the class. We can have, as I want to argue, rigor along with teaching evaluations, provided we use the evaluations carefully.
What else might we do to measure teaching so that SEFs are not the only evidence offered? Faculty who are coming up for tenure and/or promotion could submit a teaching portfolio. Seldin (1997:2) has defined a teaching portfolio as "a factual description of a professor's teaching strengths and accomplishments." It includes documents and materials which collectively suggest the scope and quality of a professor's teaching performance. It is to teaching what lists of publications, grants, and honors are to research and scholarship." It is within the context of a teaching portfolio that faculty can explain, precisely, how they manage their classes, what their philosophy of teaching is, and how they work to create high-quality learning environments. They could provide concrete examples of student achievement. It would be in a teaching portfolio that they could respond directly to the "Faculty Inventory of Seven Principles for Good Practice in Undergraduate Education" (available from the Center for Excellence in Learning and Teaching.) Their colleagues' would then verify the material in the portfolio, offering comment on their course syllabi and other accompanying material.
As I said at the beginning, my two professors would not have done well on a standardized evaluation of their teaching, but they were both professors who loved their subject matter, showed it, and were kind and helpful. They would have done quite well under a process that took all of their accomplishments as teachers into account. I believe we owe it to our colleagues and ourselves to fairly and accurately assess all aspects of teaching.
Scott G. McNall