'Tis the season to file your grades, and in that holiday spirit, I present to you my tried-n-true method for normalizing scores for different TA's in a large class. The problem is this: When you teach a 300 person class, you typically run it with a single lecturer and multiple TA's handling sections. The students do labs and problem sets, which are graded by the TA's. However, not all TA's are equally benevolent when it comes to grading, which can lead some sections to have lower scores than they should. On the other hand, not all sections are equally on the ball, so maybe their low scores are exactly what they deserve. So, how do you tell the difference between a TA who graded more harshly than average, and a TA that was stuck with somewhat dimwitted students? The key is to use the exams, which are taken by all the students. Presumably, a student that does well on the exam is probably sharp enough that they did well in their problem sets and labs. Thus, if that student has a lower than expected section score, then there is a chance that too many points were taken away by the TA, compared to the average TA. So, the trick is to make a plot of the student's section grade divided by their exam grade, ranked by exam grade. When you do so, you'll find a well-defined sequence which goes towards 1 at the high end (i.e. those top-notch students who ace everything). I tend to make the section work easier than the exams, so for me this ratio goes to larger values for lower exam scores, but another instructor who gives tough assignments but puffball exams might find the opposite. There is a scattering below the main sequence, due primarily to students who did not turn in all their assignments. TA's who readily accept late assignments tend not to have this tail. So, if you have a TA who takes off more points then average, you'll find that all their section points lie below the main trend, particularly at the low-scoring end, where lots of points were taken off. You can see the effect whether or not the TA's sections were dimmer than average or brighter than average. The plot on the left shows the section-to-exam ratio for the class as a whole (open circles) and one particular TA (solid circles), whose points clearly fall below the mean trend (click on the image for a larger, more readable version):
You can then easily scale the points taken off by some factor to correct them back to the typical TA. Note that it's critical to do this scaling on the points taken off, not the points earned, because you don't want to penalize the students who got everything correct; presumably they'd get everything right no matter how tough the TA was. The plot on the right shows what happens to that same TA's scores after rescaling the points taken off to reflect the kinder-and-gentler average TA. As you can see, it jumps right back onto the mean line. Pretty cool, huh? You may now go back to your regularly scheduled grading.
