MarkUs Blog

MarkUs Developers Blog About Their Project

Test Framewok – State of Art

with one comment

Here is the State of Art of the Markus Test Framework. We will describe what is working for each view (Administrator, Grader and Student).

Administrator view

Create a new assignment with test

When we create a new assignment, we can enable the test framwork. If we do so, we can upload a build.xml and build.properties, and add some test files (public or private).

This part is now working well, since the routing bug are fixed.

Due date passed

Buttons “Release Marks” and “Unrelease Marks” lead to a 404 error.

“Collect All Submissions” links to RecordNotFound in SubmissionsController#collect_all_submissions.

For each submission, the “Collect Submission” lead to a 404 error (issue https://github.com/MarkUsProject/Markus/issues/441)

We can’t test if the Admin can run the test because of the errors posted in our last post on this blog.

Grader’s view

We suppose this configuration being applied by the administrator :

  • Tests have been uploaded
  • Tests are public
  • Tokens are available
  • The return date is passed

First, the student has to create a working group or click he wants to work alone.

The Administrator assign a Grader to the Student.

Then, the Grader is supposed to be able to use the testing framework :

  • Click on one of the current assignment.
  • “Can begin grading” is True
  • On clicking on a Group name : the Marking State changes from « Not Collected » (Blue square) to “In progress” (a yellow pencil) and a confirmation message “The submission has been given collection priority. It will be ready soon.” appears on screen.

Then there are problems with collecting all assignments, applying tests or getting results :

  • On clicking on « Collect all your assigned submissions » an error page appears ” ActiveRecord::RecordNotFound in submissionsController#collect_ta_submissions
  • No test can be run and no results loaded.
  • The button “see and comment the result details” is still unavailable.

Student’s view

We suppose :

  • Tests have been uploaded by the administrator on the administrator view
  • Tests are public
  • Tokens are availables
  • The return date is not passed

Steps to use the testing framework as a student:

First, the student has to create a working group or click he wants to work alone.

Then, he can post his work (1. Click on submissions, 2. Click on “add new” and find the file to upload, 3. Click on “submit”).

Then, he can use the testing framework:

  • Click on “assignments”
  • For the first time of use: the button “collect and prepare tests” is available so he can click on.
  • Click on the button “run the test”

What happens ?

At this point, we can launch the tests, they are running but no result is collected and so there is no result available.

Previously, there were some issues of displaying but now all is well displayed (11/30/2011 patch by Guillaume).

If we change some configurations:

  • We suppose that there is no token available or tests are private.

We have the same situation as previously. (Shouldn’t we have no button available to run the tests in those cases?)

  • If the due date is passed: No problem appears. Indeed, the student cannot launch the tests.

Written by gbugnet

December 1st, 2011 at 12:46 pm

Posted in Automated Testing

Tagged with ,

One Response to 'Test Framewok – State of Art'

Subscribe to comments with RSS or TrackBack to 'Test Framewok – State of Art'.

  1. Nice blog post! Thanks for sharing.

    Severin

    2 Dec 11 at 10:29 am

Leave a Reply