Cornell College, Computer Science, Mount Vernon, IA 52314
Many Course Management Systems (CMSes) can automatically grade simple homework assignments involving multiple choice, word matching and number matching questions. We extend one CMS (WeBWorK) to automatically grade the correctness of code fragments and programs written in the Java programming language, and provide a set of assignments appropriate for teaching CS1 courses. The WeBWorK Java Evaluator is designed with several goals in mind. The system gently introduces programming to beginning students in CS1 and CS2. Instructors can ask students to write single methods or even single lines of a Java program. The evaluator then automatically checks for syntactic and semantic correctness, and students receive immediate feedback on how well they did. Students submit their code in a web browser, and so do not have to install any new software or learn a new development environment. The system is also designed with helpful feedback in mind, so that students are better equipped to fix their code and get the most out of the trial and error process. Extending WeBWorK has a number of advantages over writing a custom tool from scratch. WeBWorK’s already implements standard CMS features, is open source and designed to be extended, and has a significant installed base in higher education. The existing framework is used to collect student data in web forms and return feedback and solutions. Problems consist of three pieces: a description, a template file written in Java, and a set of unit tests to confirm the correctness of the fragment. When students submit a code fragment, it is inserted into the template, compiled, and tested using the unit tests. The results are then displayed to the student explaining which tests failed and why. For example, a question that requires a student to write a method to sort an array can be unit tested to ensure the method runs correctly, and via Java’s built-in introspection, that it has the correct name, return type and parameter types. Because unit testing is a standard technique, problem authors don’t have to learn much in order to create new problems. Because systems that execute arbitrary code have many security issues, security was taken seriously. For example, the Java .policy framework was used to tightly limit what the fragments were allowed to do, among other things limiting i/o access. CPU and memory limits on execution were placed both at the level of the Java executable and operating system, and all the standard Perl security measures were implemented. Future research includes expanding the problem library, further improving and testing security, providing integration with other CMSes such as Moodle, and ultimately testing the systems effectiveness in the classroom.
[Abstract (DOC)]