This tool is designed for the problem of peer-grading/peer-reviewing. Given a set of assignments that need to be graded, we aggregate the grades provided by the peer graders/reviewers. The peer-grading toolkit takes as input a set of orderings provided by the reviewers indicating their preferences over the different assignments. Given these orderings you can use the tool to produce an overall ranking of all assignments as well as an estimate of how reliable each of the different reviewers were.
Please find the papers detailing the machine learning methods implemented in this toolkit along with other tecnical details in the papers provided below.
To learn more about the methods used by the peer-grading tool please refer to the following papers: