MarkUs Blog

MarkUs Developers Blog About Their Project

Automated Testing Framework

with 3 comments

Talks concerning MarkUs Automated Testing Framework

We defined some specifications for the Automated Testing Framework we wanted to implement. First, the framework must add as few dependencies as possible, and not disturb the core of MarkUs (MarkUs must not be overcrowded by other programs for testing student’s code). The Automated Testing Framework must also support as many languages as possible, allowing users to test any language regardless of the languages MarkUs has been implemented in.

Technical side

Today, C, Java, Python and Scheme are used in the Universities where MarkUs is deployed (Department of Computer Science, University of Toronto.School of Computer Science, University of Waterloo.Ecole Centrale de Nantes).
C++ should be easy to implement according to the work done with C.

Diane and IĀ are going to build our framework by using examples from these languages. When we speak of an Automated Testing Framework, the idea is not to build a new Unit Testing Framework. We aim to build an “abstraction layer” between MarkUs and most used Unit Testing Frameworks for most used languages in University.

A picture is far better than words :

Automated Testing

Automated Testing Framework - version 1

MarkUs will call an Ant script written by the TA. This script, and all necessary files for the test environment (student code, data and tests) will be sent to a “working environment”. Later, the working environment could be moved to a dedicated server.

MarkUs will ask the Ant script to do many tasks (see Ant tasks), in an asynchronous way. Next, the Ant script, customizable by each instructor, will compile, test and run student’s code. The Ant script will produce an xml or text output for tests and build. Ant’s output will be saved in MarkUs and will be filtered for instructors and students.

The student will be able to see the result of the compilation and some test results will be made available by the instructor.

Written by Benjamin Vialle

June 9th, 2010 at 10:29 am

Posted in Uncategorized

3 Responses to 'Automated Testing Framework'

Subscribe to comments with RSS or TrackBack to 'Automated Testing Framework'.

  1. Idea:

    MarkUs could provide two standard interfaces for exporting data: One SSH/SCP connector and one file system connector. See for doing SSH stuff from within Ruby. The file system connector is basically your SVN export.

    MarkUs users could choose between either one of the two and would have to configure it appropriately.


    What’s the Ruby script for in your picture?


    10 Jun 10 at 7:36 pm

  2. In fact, the Ruby Script should be in the blue box. It symbolize at first the libsvn connection (in lib/repo/) of MarkUs for exporting a svn repository.

    I will have a look on the ssh/scp connector. It is actually the second option to be implemented. We wanted to be able to scp files directly to a VM for example.

    Benjamin Vialle

    10 Jun 10 at 8:20 pm

  3. thks for info. i’m developing Automatic grading in my Final Project and i used marmoset in my analisys sistem.


    8 Jun 12 at 7:21 am

Leave a Reply