CSCI 3310 uses a system for automatically checking your projects and ensuring their correctness. The autograder is designed to help you do the projects correctly and to provide feedback in a timely fashion. By design, you can submit each project to the autograder multiple times as you work (instead of just once at the very end).
When you submit a project to the autograder, your submission will be checked automatically using a series of test cases. In each test case, your program's output on the test is compared to the output of a known correct program; the test is passed if the outputs are identical. The results (i.e., tests that passed and failed) will be emailed back to you as soon as the autograder is finished checking your submission. Be warned, however, that passes and failures will be the only information you will get: the autograder results will not tell you what the test cases are doing, what your program's outputs were, or what the correct outputs are. This design is intentional: the autograder does not function as a debugger, but rather as a tool to let you when your project is working properly. This feedback will help you avoid erroneously thinking that your project is finished when there is still work to be done.
Rather than relying on the autograder to debug, the best way to debug your program is to write your own test cases, figure out the correct outputs (i.e., according to the project specification), and then compare your program's output to the correct answers. The only way you can be sure that your program follows a given part of the specification is if you have a test that checks that part. Designing test cases will also give you a better understanding of the concepts covered by the projects.
Later projects will require you to submit your test cases to the autograder along with your program, and your tests themselves will be checked to determine how well they exercise the program. The autograder checks submitted test cases by running them through known buggy programs; if the submitted test case output on the known buggy program differs from its output on a correct program, then the bug is considered "exposed" by the test case. If the outputs are the same, then the bug is not exposed. The autograder results will tell you which bugs were exposed by your test cases -- without, of course, telling you anything about what the bugs are.
The submission policy of the autograder is as follows:
As you might surmise, the autograder submission policy is designed to incentivize you to start the projects as early as possible, which will both give you more time and provide you with more feedback.
In addition to the autograder's evaluation of your program's correctness, I will manually evaluate your final submission for issues such as program design, efficiency, documentation, style, etc. Your final score on each project will be a combination of the autograder score and my hand-graded score.
The autograder can be a bit picky about the programs that you submit. If you submit a program that the autograder doesn't like, it will refuse to check your submission and tell you so via your submission feedback. This will make you sad for two reasons: (1) you will not get useful feedback, and (2) you will have wasted an autograder submission. In particular, any of the following will cause the autograder to reject your submission:
.cc
.-std=c++11
flag.-Wall
and -std=c++11
flags.Avoid these potential missteps by following a few simple rules:
.cc
.-std=c++11
and -Wall
(you should always be using the latter anyways). The former flag tells g++
to use the 2011 standard of C++ (aka C++11); don't use any language features introduced in later standards. Put your flags in a Makefile
to save you time and ensure you won't forget to add them. The starter files for most projects already include a Makefile
containing these flags.Submissions to the autograder are made using the submit3310
command on the class server, like so:
submit3310 <projectnum> <file1> <file2> ...
For example, to submit Project 1, you would execute the following:
submit3310 1 inverter.cc
The system will package up your project and send it to the autograder. Within an hour or so (a few minutes in most cases) you should receive an email telling you how your submission did. If you do not receive a response within an hour of submission, let me know (but check your spam filter first). The official time of submission for your project is the time that the project is received by the autograder, regardless of how long it takes to check. If you make any submission after the project due date, your project will be considered late (and will use up your flex days).
Past students have correctly noted that it would be nicer if the autograder sent Slack messages instead of emails. This effort is left to future work.
Many parts of the autograder were initially developed by Peter Chen, Brian Noble, Atul Prakash, Mark Corner, and others.