AuDoscore is an extension of JUnit that is used to evaluate Java programming exercises.
We use AuDoscore in our algorithm and data structure course to automate evaluation of student homework submissions.
| Annotation | Required | Position | Arguments | Semantics |
|---|---|---|---|---|
@Exercises |
Yes | class level | Array of @Ex annotations |
contains a list of @Ex annotations; see @Ex |
@Ex |
Yes | element in @Exercises list |
exId: String, points: Double |
for every (sub)exercise, create an @Ex annotation with unique exID |
@Points |
Yes | test case level (method) | exId: String, bonus: Double, malus: Double, comment: String, defaults to method name |
Student earns bonus / sum(bonus) * Ex.points points for passing this test case. Student looses malus / sum(bonus) * Ex.points points for not passing this test case. |
@Forbidden |
No | class level | value: Array of Strings, type: Forbidden.Type.PREFIX (default), Forbidden.Type.FIXED, or Forbidden.Type.WILDCARD |
Arguments specify forbidden classes/methods/etc., depending on the type. PREFIX: Arguments are interpreted as prefix to forbidden elements. FIXED: Arguments are exactly the forbidden elements. WILDCARD: Like PREFIX but also supports wildcards. See also @NotForbidden |
@NotForbidden |
No | class level | value: Array of Strings, type: Forbidden.Type.PREFIX (default), Forbidden.Type.FIXED, or Forbidden.Type.WILDCARD |
Arguments specify allowed classes/methods/etc., these take precedence over @Forbidden; see also @Forbidden |
@Replace |
No | test case level (method) | Array of Strings | Strings refer to methods in the student's code. For this test case, all methods mentioned in the @Replace annotation will be replaced with their cleanroom counterparts. Note: @Replace can only be used in secret tests! |
@SecretClass |
No | class level | None | Marks a test class to be secret. Results will not be shown to students before the submission deadline. |
@CompareInterface |
No | class level | Array of String | Checks if methods and fields of students have the same signature as their cleanroom counterparts. Possible Strings: "Classname.Methodname, "Classname.Fieldname", "Classname". If only the Classname is given all public methods/fields are checked. |
@InitializeOnce |
No | static field | value: String |
At first execution, the method given as value is called and its result is stored in a temporary file. At subsequent executions, the annotated attribute is initialized using the precomputed result from the file. Note: @InitializeOnce can only be used in secret tests and the result of the given method must be serializable! |
@SafeCallers |
No | class level | value: Array of Strings |
Fully qualified class names of classes to be granted additional permissions through the security manager (otherwise blocked e.g. for student code in order to prevent malicious behavior, file/net access, access to grading infrastructure, ...). |
AuDoscore comes with a test script that can be used to locally grade a submission. The test script requires that the files belonging to an exercise are organized in a specific directory structure:
exercise_dir
↦ cleanroom
↦ Solver.java
↦ interfaces
↦ ISolver.java
↦ junit
↦ UnitTest.java
↦ SecretTest.java
↦ skeleton
↦ Solver.java
↦ student
↦ Solver.java
The cleanroom directory contains the cleanroom solution (i.e. the sample solution from the lecturer).
The annotated unit tests reside in the junit directory.
The interfaces directory is optional and contains the interfaces to be implemented by a student submission.
The skeleton directory is optional and contains skeleton code provided to the students
(thus it is ignored by the AuDoscore infrastructure).
A student submission can optionally be placed in the student directory
(if placed in another folder, this folder must be explicitly provided as argument to the test script).
To run AuDoscore on this example, cd to the exercise_dir and execute the test.sh script from the AuDoscore repository.
The test.sh script takes the name of the directory containing the submission as an optional parameter,
so you can also test submissions in other directories than student (e.g. the cleanroom itself).
The test script produces a nicely formatted output of the passed and failed tests and also prints the scored points.
There are currently several active branches in this repository.
Development usually happens in the master branch.
The release branch just contains the version of AuDoscore currently in use by us.
The project is licensed under the Apache License, Version 2.0.
Please report bugs via the issue tracker.
Feedback: [email protected]