This tool is designed for the problem of peer-grading/peer-reviewing. Given a set of assignments that need to be graded, we aggregate the grades provided by the peer graders/reviewers. The peer-grading toolkit takes as input a set of orderings provided by the reviewers indicating their preferences over the different assignments. For instance, in the example provided below reviewer 1 rates assignment 1 as being better than assignment 2 which in turn is better than assignment 3. Given these orderings you can use the tool to produce an overall ranking of all assignments as well as an estimate of how reliable each of the different reviewers were.
DATA USAGE POLICY: We do not store any of the data uploaded. The output rankings produced by the toolkit are deleted daily.
To learn more about the machine learning techniques we use please check out our papers. You can also download the code and run it offline.