Equity/Bias in Teaching Evaluations Abstracts

Back to Bias in Teaching Evaluations

Equity/Bias in Teaching Evaluations Abstracts

M. H. Abel and A. L. Meltzer, , Students ratings of a male and female professors’ lecture on sex discrimination in the workforce

Male (n = 41) and female (n = 46) undergraduates evaluated an identical written lecture that they were told was by a male professor or a female professor. The lecture focused on pay disparities between men and women in the workforce and was intended to instill a perception of sex discrimination in the workforce. Both male and female students rated the male professor and his lecture more positively and less sexist than they rated the female professor and her lecture. J. Arbuckle and B. D. Williams, Students’ perceptions of expressiveness: Age and gender effects on teacher evaluations

This study investigated the relationship between students’ perceptions of the expressiveness of professors and implicit age and gender stereotypes. Male and female undergraduates (N = 352) watched slides of an age- and gender-neutral stick figure and listened to a neutral voice presenting a lecture. Then, they completed teacher evaluation forms that indicated 1 of 4 different age and gender conditions (male, female, “old,” and “young”). Students rated the “young” male professor higher than they rated the “young” female, “old” male, and “old” female professors on speaking enthusiastically and using a meaningful voice tone during the lecture. C. M. Bachen, M. M. McLoughlin, & S. S. Garcia, Assessing the role of gender in college students’ evaluations of faculty

In this study, researchers surveyed nearly 500 university students about their perceptions of male and female faculty, in order to investigate whether students’ assessments of male and female professors are influenced by traditional gender schema. Female students rated female faculty especially high across five teaching dimensions and male faculty comparatively lower. By contrast, male students did not evaluate male and female professors as significantly different. Faculty assessments were further influenced by the strength of students’ gender schema. S. A. Basow, Student evaluations of college professors: When gender matters

In this study, researchers analyzed student evaluations completed over a four-year period at a private liberal arts college for the effects of teacher gender, student gender, and divisional affiliation. Overall, the ratings of male professors appeared to be unaffected by student gender. By contrast, female professors tended to receive their highest ratings from female students and their lowest ratings from male students. The interaction between student gender and teacher gender generally remained when possible confounding factors (e.g., teacher rank) were removed. The mean ratings that female and male professors received also varied as a function of the divisional affiliation of the course. S. A. Basow and N. T. Silberg, Student evaluation of college professors: Are female and male professors rated differently?

More than 1,000 male and female college students of 16 male and female professors (matched for course division, years of teaching, and tenure status) evaluated their instructors on teaching effectiveness and sex-typed characteristics. Male students gave female professors significantly poorer ratings than male professors on the six teaching evaluation measures. Male students’ ratings of female professors were poorer than those of female students on four of the six measures. On three measures, female students evaluated female professors less favorably. Student perceptions of a professor's instrumental/active and expressive/nurturant traits accounted for only a few of the gender-related effects. L. Bates, Female academics face huge sexist bias – no wonder there are so few of them

A new online tool reveals the stark gender bias in how students evaluate their university lecturers. This is yet another hurdle for women in academia to overcome. Gender Biases in Student Evaluations of Teachers and their Impact on Teacher Incentives

This paper uses a unique database from a French university to analyze gender biases in student evaluations of teachers (SETs). The results of generalized ordered logit regressions and fixed-effects models suggest that male teachers tend to receive higher SET scores because of students’ gender biases. Male students in particular express a strong bias in their favor: male students are approximately 30% more likely to give an excellent overall satisfaction score to male teachers compared to female teachers. The different teaching dimensions that students value in men and women tend to correspond to gender stereotypes. The teaching dimensions for which students perceive a comparative advantage for women (such as course preparation and organization) tend to be more time-consuming for the teacher, compared to the teaching dimensions that students value more in men (such as class leadership skills). Men are perceived as being more knowledgeable (male gender stereotype) and obtain higher SET scores than women, but students appear to learn as much from women as from men, suggesting that female teachers are as knowledgeable as men. Finally, I find that if women increased students’ continuous assessment grades by 7.5% compared to the grades given by their male colleagues, they could obtain similar overall satisfaction scores as men. Yet, women do not act on this incentive (men and women give similar continuous assessment grades), suggesting that female teachers are unaware of students’ gender biases. These biases have strong negative consequences for female academics, who may spend more time on teaching to try to obtain high SET scores, reducing time available for research. The results suggest that better teaching is not necessarily measured by SETs. A. Boring, Gender biases in student evaluations of teaching

Gender biases partly explain student evaluation fo teaching (SET) scores.
Male students in particular discriminate in favor of male professors.
Students appear to rate professors according to gender stereotypes.
No evidence that male professors are better instructors.

A. Boring, K. Ottoboni, and P. B. Stark, Student evaluations of teaching are not only unreliable, they are significantly biased against female instructors

A series of studies across countries and disciplines in higher education confirm that student evaluations of teaching (SET) are significantly correlated with instructor gender, with students regularly rating female instructors lower than male peers. Anne Boring, Kellie Ottoboni and Philip B. Stark argue the findings warrant serious attention in light of increasing pressure on universities to measure teaching effectiveness. Given the unreliability of the metric and the harmful impact these evaluations can have, universities should think carefully on the role of such evaluations in decision-making. J. H. Bray and G. S. Howard, Interaction of Teacher and Student Sex and Sex Role Orientations and Student Evaluations of College Instruction

From the article: This study investigated the relationship between teacher and student sex and sex role orientations and student ratings of their progress in the course and satisfaction with the instructor. Thirty college instructors and four hundred and ninety-seven students participated in the study. Results indicate that androgynous teachers received somewhat higher student evaluations than their masculine or feminine counterparts. A teacher sex with student sex interaction was found, as well as a teacher sex with teacher sex role interaction. Finally, a hypothesis, that similarities in teacher and student sex role orientations would relate to higher teacher evaluation ratings, was not supported. These findings support previous studies which found that teacher personality characteristics are related to student satisfaction. S. Buck and D. Tiene, The Impact of Physical Attractiveness, Gender, and Teaching Philosophy on Teacher Evaluations

This study explored the impact of teachers’ physical appearance and teaching philosophy on other persons’ perceptions of their competence. Secondary-level student teachers were given photographs of attractive and unattractive teachers of both genders. Each photograph was attached to a written statement about teaching, describing the teachers’ instructional approach as either authoritarian or humanistic. We did not find any main effects of attractiveness or gender upon perceptions of competence. These findings contradicted the results of some previous studies in which the only basis for evaluation was a photograph. A significant interaction occurred between attractiveness, gender, and authoritarianism. The attractive female authoritarian teacher was rated significantly less negatively than the other three types of authoritarian teachers. We hypothesized that she may have been less credible as an authoritarian figure and was consequently spared some of the negativity directed toward the others. K. Doerer, Colleges are Getting Smarter About Student Evaluations. Here's How.

This article begins by discussing a project at the University of Oregon to review teaching evaluations. Other studies efforts to de-emphasize student evaluations are mentioned. There are links to articles on the same topic in the current article. M. Falkoff, Why We Must Stop Relying on Student Ratings of Teaching

This article discusses the finding of a study by the American Political Science Association, which found that the "language students use in evaluations regarding male professors is significantly different than language used in evaluating female professors." The study also showed that "a male instructor administering an identical online course as a female instructor receives higher ordinal scores in teaching evaluations, even when questions are not instructor-specific." There are links to articles on the same topic in the current article. Y. Fan, L. J. Shepherd, E. Slavich, D. Waters, M. Stone, R. Abel, E. L. Johnston, Gender and cultural bias in student evaluations: Why representation matters

From the article: Gendered and racial inequalities persist in even the most progressive of workplaces. There is increasing evidence to suggest that all aspects of employment, from hiring to performance evaluation to promotion, are affected by gender and cultural background. In higher education, bias in performance evaluation has been posited as one of the reasons why few women make it to the upper echelons of the academic hierarchy. With unprecedented access to institution-wide student survey data from a large public university in Australia, we investigated the role of conscious or unconscious bias in terms of gender and cultural background. We found potential bias against women and teachers with non-English speaking backgrounds. Our findings suggest that bias may decrease with better representation of minority groups in the university workforce. Our findings have implications for society beyond the academy, as over 40% of the Australian population now go to university, and graduates may carry these biases with them into the workforce. C. Flaherty, Zero Correlation Between Evaluations and Learning

The meta-analysis summarized in this article shows no correlation between good teaching evaluations and learning outcomes. The actual study was conducted by Uttl, White, and Gonzalez. See that reference below. M. E. Heilman and T. G. Okimoto, Why Are Women Penalized for Success at Make Tasks? The Implied Communality Deficit

From the article: In 3 experimental studies, the authors tested the idea that penalties women incur for success in traditionally male areas arise from a perceived deficit in nurturing and socially sensitive communal attributes that is implied by their success. The authors therefore expected that providing information of communality would prevent these penalties. Results indicated that the negativity directed at successful female managers-in ratings of likability, interpersonal hostility, and boss desirability-was mitigated when there was indication that they were communal. This ameliorative effect occurred only when the information was clearly indicative of communal attributes (Study 1) and when it could be unambiguously attributed to the female manager (Study 2); furthermore, these penalties were averted when communality was conveyed by role information (motherhood status) or by behavior (Study 3). These findings support the idea that penalties for women's success in male domains result from the perceived violation of gender-stereotypic prescriptions. Ellen M. Key and Phillip J. Ardoin, Students rate male instructors more highly than female instructors. We tried to counter that hidden bias

This is the second piece in a two-week series on the gender gap in political science. In the author's experiment, they divided 52 classes into two groups. The first group was told via email that course evaluations were about to begin. The second group received the same email with an added statement that encouraged them to keep research about biases, both explicit and implicit, in mind. The findings were that informing students about bias changed the tone of their comments on female faculty. J. M. Lang, We Don't Trust Course Evaluations, but Are Peer Observations Much Better?

James Lang, a professor of English and director of the D’Amour Center for Teaching Excellence at Assumption College in Worcester, MA, reflects on this experiences with teaching evaluations and tenure and promotion committee work. He links to 3 models for evaluating teaching. It is worth taking a look at these models. L. MacNell, A. Driscoll, & A.N. Hunt, What’s in a name: Exposing gender bias in student ratings of teaching

From the article: Student ratings of teaching play a significant role in career outcomes for higher education instructors. Although instructor gender has been shown to play an important role in influencing student ratings, the extent and nature of that role remains contested. While difficult to separate gender from teaching practices in person, it is possible to disguise an instructor’s gender identity online. In our experiment, assistant instructors in an online class each operated under two different gender identities. Students rated the male identity significantly higher than the female identity, regardless of the instructor’s actual gender, demonstrating gender bias. Given the vital role that student ratings play in academic career trajectories, this finding warrants considerable attention. F. Mengel, J. Sauermann, & Ulf Zölitz, Gender Bias in Teaching Evaluations

“This paper provides new evidence on gender bias in teaching evaluations. We exploit a quasi-experimental dataset of 19,952 student evaluations of university faculty in a context where students are randomly allocated to female or male instructors. Despite the fact that neither students’ grades nor self-study hours are affected by the instructor’s gender, we find that women receive systematically lower teaching evaluations than their male colleagues. This bias is driven by male students’ evaluations, is larger for mathematical courses and particularly pronounced for junior women. The gender bias in teaching evaluations we document may have direct as well as indirect effects on the career progression of women by affecting junior women’s confidence and through the reallocation of instructor resources away from research and towards teaching.” P. Miles and D. House, The Tail Wagging the Dog; An Overdue Examination of Student Teaching Evaluations

This study by Patti Miles and Deanna House analyzes 30,000 student evaluations of 255 professors spanning six semesters during a three-year period. The authors found that while “women seem to be competitive with their male colleagues on many levels…when it comes to teaching large classes their scores on STEs seem to drop substantially”. They urge educators and administrators not to rely solely on student evaluations to measure teacher effectiveness, and to utilize this information in conjunction with other methods. J. Miller & M. Chamberlin, Women are teachers, men are professors: A study of student perceptions

From the article: Sociology students’ perceptions of their instructors’ educational attainment levels are examined empirically. We find gender disparities: students misattribute in an upward direction the level of education actually attained by male graduate student instructors, while they misattribute in a downward direction the level of formal education attained by women, even when the female faculty member is a full professor. The misattributions are linked to the imputed statuses “teacher” for women, and “professor” for men, regardless of the actual positions held or the credential earned by faculty members and graduate student instructors. We suggest that a process of marginalization explains the empirical findings - a process that is attributed by others, but chosen by the self, regardless of the social and economic costs incurred. Consequences for students and sociology professors are discussed. K. Mitchell and J. Martin, Gender Bias in Student Evaluations

From the article: Many universities use student evaluations of teachers (SETs) as part of consideration for tenure, compensation, and other employment decisions. However, in doing so, they may be engaging in discriminatory practices against female academics. This study further explores the relationship between gender and SETs described by MacNell, Driscoll, and Hunt (2015) by using both content analysis in student-evaluation comments and quantitative analysis of students’ ordinal scoring of their instructors. The authors show that the language students use in evaluations regarding male professors is significantly different than language used in evaluating female professors. They also show that a male instructor administering an identical online course as a female instructor receives higher ordinal scores in teaching evaluations, even when questions are not instructor-specific. Findings suggest that the relationship between gender and teaching evaluations may indicate that the use of evaluations in employment decisions is discriminatory against women. K. Mitchell Student Evaluations Can't Be Used to Assess Professors

Gender and Bias

V. Ray, Is Gender Bias an Intended Feature of Teaching Evaluations

in spite

because

A. S. Rosen, Correlations, Trends, and Potential Biases among Publicly Accessible Web-Based Student Evaluations of Teaching:A large-scale study of RateMyProfessors.com data

From the article: Student evaluations of teaching are widely adopted across academic institutions, but there are many underlying trends and biases that can influence their interpretation. Publicly accessible web-based student evaluations of teaching are of particular relevance, due to their widespread use by students in the course selection process and the quantity of data available for analysis. In this study, data from the most popular of these websites, RateMyProfessors.com, is analysed for correlations between measures of instruction quality, easiness, physical attractiveness, discipline and gender. This study of 7,882,980 RateMyProfessors ratings (from 190,006 US professors with at least 20 student ratings) provides further insight into student perceptions of academic instruction and possible variables in student evaluations. Positive correlations were observed between ratings of instruction quality and easiness, as well as between instruction quality and attractiveness. On average, professors in science and engineering disciplines have lower ratings than in the humanities and arts. When looking at RateMyProfessors as a whole, the effect of a professor’s gender on rating criteria is small but statistically significant. When analysing the data as a function of discipline, however, the effects of gender are significantly more pronounced, albeit more complex. The potential implications are discussed. B. Uttl, C. A. White, and D. W. Gonzalez, Meta-analysis of faculty’s teaching effectiveness: Student evaluation of teaching ratings and student learning are not related

Students do not learn more from professors with higher students evaluation of teaching (SET) ratings.
Previous meta-analyses of SET/learning correlations in multisection studies are not interpretable.
Re-analyses of previous meta-analyses of multisection studies indicate that SET ratings explain at most 1% of variability in measures of student learning.
New meta-analyses of multisession studies show that the SET ratings are unrelated to student learning.

N. Wagner, M. Rieger, and K. Voorvelt, Gender, ethnicity and teaching evaluations: Evidence from mixed teaching teams

This paper studies the effect of teacher gender and ethnicity on student evaluations of teaching at university. Wagner, Rieger and Voorvelt analyze a unique data-set featuring mixed teaching teams and a diverse, multicultural, multi-ethnic group of students and teachers. Blended co-teaching allows them to study the link between student evaluations of teaching and teacher gender as well as ethnicity exploiting within course variation in a panel data model with course-year fixed effects. The authors document a negative effect of being a female teacher on student evaluations of teaching, which amounts to roughly one fourth of the sample standard deviation of teaching scores. Overall women are 11 percentage points less likely to attain the teaching evaluation cut-off for promotion to associate professor compared to men. The effect is robust to a host of co-variates such as course leadership, teacher experience and research quality, as well as an alternative teacher fixed effect specification. There is no evidence of a corresponding ethnicity effect. Our results are suggestive of a gender bias against female teachers and indicate that the use of teaching evaluations in hiring and promotion decisions may put female lectures at a disadvantage.