MarkUs Blog

MarkUs Developers Blog About Their Project

New database schema for storing test configuration and results

with one comment

I have taken Byron’s schema and thought about it a little more, and here is what I came up with. I am also trying to figure out how to save the test configuration in a way that makes it easy for us to send it to the test runner to start the test. Here is what I have so far:

  test_suite_id           // identify this test
  assignment_id           // the assignment this test belongs to
  name                    // test name
  description             // description of the test suite, if any
  halt_on_error           // flag whether to halt or continue on error
  pretest_scripts         // ordered space separated list of pretest scripts
  build_scripts           // ordered space separated list of build scripts
  test_scripts            // ordered space seperated list of test scripts

  run_id                  // sequential number identifying this run
  test_suite_id           // the test suite that was ran
  group_id                // group that ran the test
  assignment_id           // assignment being tested
  cumulative_mark         // sum of marks from all the test files
  date                    // date & time run was requested

  run_id                  // link to the Automated_test_run it belongs to
  test_suite_id           // the specific test that was run
  name                    // name of the specific test file that was ran
  kind                    // one of {pretest, build, test}
  status                  // one of {passed, failed, error}
  marks                   // marks awarded for this test file
  input                   // input passed to the student file
  expected_output         // expected output from the student file
  actual_output           // actual output from the student


The test suite contains information we get from the instructor when he/she first sets up the testing. The most interesting fields are pretest_scripts, build_scripts, and test_scripts. The three fields are simply space separated strings of the names of respective scripts the instructor wants to run for each test.

The test run table contains an record of an individual run of a test suite

The test_file_result table contains the results of running each individual test file in the test suite.

One thing to note. For this schema, I assume that the instructor will go through the trouble of creating pretest, build, and test scripts that don’t need any flags or input files to run, and hence all we need to run them is the name of the script, which the only thing this this schema stores about the test files the instructor uploads. Is that what we are aiming for? If not, then I think we will have to add another table dedicated to the test files, and information about them (their input files and the flags they run with).

Written by mina

October 7th, 2012 at 9:10 pm

One Response to 'New database schema for storing test configuration and results'

Subscribe to comments with RSS or TrackBack to 'New database schema for storing test configuration and results'.

  1. I think this is a great start, but it needs a bit more normalization. The ordered lists for the tests is the first code smell. They should be in a separate table so we can add other information associated with each test like which information to display, the max marks it’s worth, when to run it {submission, student request, final mark}, etc. [Ah, I see a note about this at the end.]

    This implies to me that one assignment might have many test suites. Is that the intention? It’s not how I was personally visualizing it.

    I don’t think Test_run needs to have both assignment_id and test_suite_id. THe assignment_id can be had by joining with test_suite.

    Test_file_result doesn’t need name and kind. That should be an attribute of the missing table about tests.

    Byron Weber Becker

    9 Oct 12 at 9:19 am

Leave a Reply