Automated grading for advanced topics courses

Abstract

As Computer Science course enrollments have increased over recent years, instructors have turned to automated grading systems to help relieve the burden of processing student assignments. However, the available autograding solutions have generally lacked support for traditionally difficult-to-grade advanced topics courses. In this thesis, I explore the "autogradeability" of the assignments presented in advanced topics computer science courses. I assert that automated grading systems can both support such courses and can be designed such that they are easy to use for instructors and of great educational value to students. I introduce a set of six performance axes through which the viability of an autograding system can be assessed: Repeatability, Scalability, Security, Extensibility, Instructor Ease of Use, and Educational Value. I lay out an extensibility-based design philosophy, which promotes system modularity for the simpler integration of features to support new courses, and I champion the inclusion of features which increase educational value to students within an autograding system. I detail the conception, design, implementation, and success of automated grading subsystems to process networked assignments and interactive computer graphics assignments, as well as the advancements in automated grading infrastructure necessary to achieve this success. Finally, I look to assignments which cannot be automatically processed, and turn to peer grading as a method of alleviating instructor burden while providing a valuable learning experience to the student. To that end, I propose a novel algorithm for peer review matching. This algorithm uses stratified sampling informed by a student's likelihood to produce feedback of high quality to increase the probability that a peer assessee will receive actionable, educationally valuable feedback.