Mechanical TA: Partially Automated High-Stakes Peer Grading
Full Text: 2676723.2677278.pdf
We describe Mechanical TA, an automated peer review system, and report on our experience using it over three years. Mechanical TA differs from many other peer review systems by involving human teaching assistants (TAs) as a way to assure review quality. Human TAs both evaluate the peer reviews of students who have not yet demonstrated reviewing proficiency and spot check the reviews of students who have. Mechanical TA also features" calibration" reviews, allowing students to quickly gain experience with the peer-review process. We used Mechanical TA for weekly essay assignments in a class of about 70 students, a course design that would have been impossible if every assignment had had to be graded by a TA. We show evidence that it helped to support student learning, leading us to believe that the system may also be useful to others.
Citation
J. Wright, C. Thornton, K. Leyton-Brown. "Mechanical TA: Partially Automated High-Stakes Peer Grading". ACM Technical Symposium on Computer Science Education (ACM-SIGCSE), Kansas City, USA, pp 96-101, March 2015.| Keywords: | Computer Science Education, Peer Grading, Peer Review, Calibration, Scalability |
| Category: | In Conference |
| Web Links: | ACM Digital Library |
BibTeX
@incollection{Wright+al:ACM-SIGCSE15,
author = {James R. Wright and Chris Thornton and Kevin Leyton-Brown},
title = {Mechanical TA: Partially Automated High-Stakes Peer Grading},
Pages = {96-101},
booktitle = {ACM Technical Symposium on Computer Science Education
(ACM-SIGCSE)},
year = 2015,
}Last Updated: March 03, 2020Submitted by Sabina P
