Yet, the research on student evaluations is troubling. It confirms not some connection between a professor's style and student evaluations, but an overwhelming link between those two factors. Nonverbal behaviors appear to matter much more than anything else in student ratings. Enthusiastic gestures and vocal tones can mask gobbledygook, smiles count more than sample exam questions, and impressions formed in thirty seconds accurately foretell end-of-semester evaluations. The strong connection between mere nonverbal behaviors and student evaluations creates a very narrow definition of good teaching. By relying on the current student evaluation system, law schools implicitly endorse an inflexible, largely stylistic, and homogeneous description of good teaching. Rather than encouraging faculty to use nonverbal behaviors to complement excellent classroom content, organization, and explanations, the present evaluation system largely eliminates the "dog" of substance, leaving only the "tail" of style to designate good teaching. Neither law students nor faculty benefit from such a narrow definition of good teaching." (notes deleted).This article is chuck full of summaries of experiments relating to teaching evals. My favorites are those that show a few seconds of a teacher on tape with the sound off. A group of students is then asked to evaluate the teacher on a number of measures. These evaluations -- again based on sound off seconds -- turn out to be remarkable close to the evaluations the same teachers receive at the end of a semester from their regular classes. In short, looks, movement, expressions, etc, may trump everything else. Later on in the article Professor Merritt reports on a study that seem to indicate that whatever the students are responding to has virtually nothing to do with objective measures of learning.
This leads to two questions. How would a Moneylaw school evaluate teaching? If tenure, promotions and salaries are based on student evaluations would be fair to view the process as arbitrary and capricious?