MarkUs Blog

MarkUs Developers Blog About Their Project

MarkUs Performance Analysis (2)

without comments

As mentioned earlier, I am currently working on analyzing MarkUs’ performance. After a few initial bumps I have a basic curl based script ready which can be used to simulate students submitting code through MarkUs. Here is the review where I introduce this new script. I should mention that this script is not very resilient to errors and is fairly sensitive with respect to the URLs you configure in that script. Have a look at this post as to why that’s the case. So here are the requirements for this to work:

  • Use MarkUs 0.10.x (MarkUs 0.11.x should work as well, but a Rails 3 based MarkUs will likely no quite work; at the very least it has been untested)
  • Prepare the server as described below.
  • Make sure that you have curl installed on the client machine.

The curl based script (lib/benchmarking/post_submissions.sh) comes with a buddy rake task called markus:benchmark:students_create. This rake task can be used to create many student users. It’ll destroy any existing student users and will hand out user names of the form student_1, student_2, student_3 and so on. It is important that you use this rake task on the server in order to create many student users, since post_submissions.sh attempts to log in as student_<#> (<#> will depend on the range you passed as a parameter to this script when you ran it). Ok, without further ado, here is the cookie-cutter-recipe:

Server Setup

First make sure that CSRF is turned on for your MarkUs instance. Make sure you have this line in your config/environments/production.rb:

config.action_controller.allow_forgery_protection    = false

 
Then to do the following:

$ bundle exec rake repos:drop RAILS_ENV=production
$ bundle exec rake db:reset RAILS_ENV=production
$ bundle exec rake db:populate RAILS_ENV=production
$ bundle exec rake markus:benchmark:students_create num=1000 \
  RAILS_ENV=production

 

The last command will create 1000 student users, student_1 to student_1000.

Client Script

After this it’s good to do some sanity checking. Make sure that logging in as student_1 and student_1000 works. Now you are ready to run post_submissions.sh. Here is one way to do it. It will use the MarkUs instance as specified by MARKUS_BASE_URL and submits all files in directory lib/benchmarking/submission_files for students student_1 to student_10.

$ cd path/to/copy/of/lib/benchmarking
$ ./post_submissions.sh 1 10

 
Note that the user who runs post_submissions.sh needs to have write permissions in the current working directory (the script needs to store/update the MarkUs cookie). After running the script you should see a directory created which holds one log file per student submission. At the moment this is the only way to figure out what happened when the script attempted to log in, visit the assignments page and submitted the files for assignment 1. Keep in mind that if you get any conflict while submitting, nothing will be submitted. This is how MarkUs works. Thus, make sure you are not submitting a file (same file name) twice.

If all went well, you should see the submitted files in a browser session as well.

So that’s the status for the time being. I’ll be likely writing another script which allows one to distribute and execute post_submissions.sh on many clients. Keep your eyes peeled for a follow up post 🙂 Questions, concerns? Let me know in the comments. Thanks!

Written by Severin

October 31st, 2011 at 12:05 pm

Posted in Uncategorized

Leave a Reply