As we reach the last weeks of the semester, we will once again be reminded to have students complete their evaluations of our classes. It is worth reminding ourselves of the problems with these forms of evaluation.
Q: Why do schools use Student Evaluations of Teaching?
A: Student Evaluations of Teaching emerged in the 1970s as a way to improve and shape the quality of teaching. They were used as part of a formative evaluation process, that is, evaluating teaching while it was occurring so as to improve it. Since then student evaluations of teaching have become forms of summative evaluation, evaluations after the process is completed wherein average scores are compared. In many places they are the sole determinant of faculty quality.
Q: Are student evaluations of teaching valid and/or biased?
A: Student Evaluations of Teaching have been the subject of research for decades. Their problems are well known.
They are not well correlated with other measures of teacher quality (1).
They demonstrate bias against women, GLBTQ individuals, and faculty of color (2).
They are biased against differential levels of assigned workload and level of course difficulty (3).
Scores of the same faculty member by different classes diverge widely (4).
Are correlated with factors like attractiveness, likability, accent and use of humor (5) .
Q: What do student evaluations of teaching really tell us? Why are they still used?
Student’s opinions of teaching capability--students can reflect on their experience in a course but they cannot be dispassionate observers of instructor performance as they are not trained in evaluation or pedagogy.
Cheap, easy and fast way to assess faculty
Provide a veneer of objectivity by offering meaningless numerical averages given that they are based on ordinal scales (e.g. Likert Scales)-- means have no meaning and the idea that all faculty should be “above the mean” is senseless.
Have inconsistent response rates.
Mostly measure faculty qualities beyond their control.
Means to encourage students to think of themselves as customers and universities as service providers.
Encourage faculty to reduce standards so as to please students.
Offer disgruntled students a way to punish faculty with disproportionate effects as they are extrapolated to an entire class.
Recall of events subject to the same types of distortions found in witness testimony--except without the opportunity for cross-examination
Q: Are there better approaches?
Most critics recommend that evaluations be professionalized and conducted by a well trained team of peers that represent the diversity of faculty (We would add that they be compensated). They should start with classroom observation and gather teacher portfolios. Standards should be transparent. Faculty should be evaluated over time. The goal of evaluations should primarily focus on improving teaching not promotion or job security. They should be formative rather than summative.
Q: What should I do if I get called in by the administration to discuss a negative evaluation?
A: Given the unreliability of student evaluations of teaching, it is not recommended that faculty try to defend themselves by explaining their scores or justifying their actions. The best course of action is simply to listen and indicate one’s openness to feedback and that one is always looking to improve their teaching. If pressed, faculty can indicate that their recollection of events differs. If you suspect that such a conversation might be used as a form of discipline rather than professional development always invoke your Weingarten rights, that is, your right to have a union representative join the meeting. Given the decades of evidence against student evaluations of teaching, our contract protects faculty against their inappropriate use.
In Solidarity
The Grievance Committee of UFCT 1460
Sources
(1) Justin Esarey & Natalie Valdes (2020) Unbiased, reliable, and valid student evaluations can still be unfair, Assessment & Evaluation in Higher Education, 45:8, 1106-1120, DOI: 10.1080/02602938.2020.1724875
Uttl, B., C. A. White, and D. W. Gonzalez, Meta-analysis of faculty's teaching effectiveness: Student evaluation of teaching ratings and student learning are not related, Studies in Educational Evaluation, 54, September 2017, 22-42.
(2) Anderson, Kristin J. and M. Kanner. Inventing a gay agenda: Students’ perceptions of lesbian and gay professors. Journal of Applied Psychology, (2011) 41-1538-1564
Basow, S. A. and N. T. Silberg, Student evaluation of college professors: Are female and male professors rated differently?, Journal of Educational Psychology 79 no. 3 308-314 (1987).
Kaschak, E. Sex bias in students' evaluations of professors' teaching methods. Psychology of Women Quarterly, 3 (3) (1978), l35-l43.
Drake, S., A. Auletto, and J. Cowen. Grading Teachers: Race and Gender Differences in Low Evaluation Ratings and Teacher Employment Outcomes. American Educational Research Journal (2019).
Miller, J. & M. Chamberlin, Women are teachers, men are professors: A study of student perceptions, Teaching Sociology, 28 (4), 283– 298 (2000).
(3) Greenwald, A. G. & G. M. Gillmore, No pain, no gain? The importance of measuring course workload in student ratings of instruction, J of Educational Psychology, vol. 89, no. 4, 743-751 (1997).
Uttl B, Smibert D. 2017. Student evaluations of teaching: teaching quantitative courses can be hazardous to one’s career. PeerJ 5:e3299
(4) Dennis E. Clayson (2018) Student evaluation of teaching and matters of reliability, Assessment & Evaluation in Higher Education, 43:4, 666-681, DOI: 10.1080/02602938.2017.1393495
(5) Duque, L. C. (2013). A framework for analyzing higher education performance: Students’ satisfaction, perceived learning outcomes, and dropout intention. Total Quality Management and Business Excellence , 25 (1–2), 1–21.