Don't Blame the Teacher? The Need to Account for Classroom Characteristics in Evaluations of Teaching Quality

Previous research provides evidence that ratings from students and external observers can deliver reliable and valid measures of teaching quality. However, there is still a lack of research on how stable these measures are across time and classes, which is particularly important when they are used for evaluations. We addressed this issue in 3 different studies from Germany. Study 1 examined stability over time using a longitudinal sample of 174 teachers and 106 classes. The results show that student ratings of teaching quality are relatively stable when the same class rates the same teacher in 2 subsequent school years. However, we also found teacher-independent stabilities, which occurred when the same students rated different teachers. Study 2 examined stabilities in student ratings across classes. In this study, each teacher (N = 95) was evaluated by 2 different classes (N = 190). The results of a 3-level model reveal rather low stabilities across classes within teachers. Study 3 drew on a unique dataset in which 1 single teacher taught the same content to 30 different classes of the same grade level. Teaching quality as rated by students and external observers varied considerably between these classes. Differences in teaching quality were related to student learning, and the motivation of students in a class was a significant predictor of differences in teaching quality. In summary, the results of these studies show that both researchers and policymakers should be cautious when interpreting measures of teaching quality as indicators of teacher quality.