MarkUs Blog

Mike Wu

Status
- Submitted a pull request #998(https://github.com/MarkUsProject/Markus/pull/998), followed by two additional commits: updating French translations and creating a migration.
- Started looking at #938(https://github.com/MarkUsProject/Markus/issues/938).

Roadblocks
- Learned how to create a migration for changing database schema in #915(https://github.com/MarkUsProject/Markus/issues/915).

Next week
- Continue working on #938(https://github.com/MarkUsProject/Markus/issues/938).
- Revise pull #998(https://github.com/MarkUsProject/Markus/pull/998) as needed.

Alysha

Status
- Continuing to work on issue 941(https://github.com/MarkUsProject/Markus/issues/941). Located and switched out all instances of submission.result/submission.remark_result for the appropriate method. Resolved majority of issues that occurred after changing the schema.

Roadblocks
- After changing the submission schema, a foreign key is no longer being set for the remark_result_id column in the Submissions table. I sent an email to the developers list to gain some insight. Working on determining the best way to approach this. Results are used in many aspects of the application. I need to thoroughly test the application to ensure all areas that were affected have been updated.

Next week
- Determine a fix to populate the remark_result_id in the database with the new schema. Test application for any errors that may have been overlooked. Write a blog post to follow-up on this post(http://blog.markusproject.org/?p=4614) which was about the original issue and the temporary fix that I am working on replacing.

Marc

Status
- Started work on converting Prototype to jQuery. Keeping track of my changes as I go along.

Roadblocks
- None so far.

Next week
- Keep working on the changes over to jQuery. Will post a blog post as well as documenting the changes I have done so far so other members on the MarkUs team can follow along.

Ian

Status
- Configured Markus to use a public key for the SSH calls.
- Discussed the current state of the test framework with Brian and Mina.
They referred Nick and I to a Google doc
(https://docs.google.com/document/d/1IFfGQ3VIYq8p2ZSy13GJPkOAVoaAw-zRcrQOj9LyKuQ/edit). Read the document to get an idea of what needs to get done.

Roadblocks
- Getting tests to mark based on the rubric is difficult and
needs some design work.

Next Week
- Try to fix functional tests for the test framework. There’s currently over 130 failing.

Daryn

Status
- Setup branch with Marc to begin collaborating on migration.
Roadblock
- Lost the weekend to the MHacks and a test today (tues.)
Next Week
- Continuation of jQuery migration.

Nick

Status
- This week was a bit busy for me so I didn’t get to do as much as I
wanted. I mostly just did reading and planning (not much coding).

- Received and read an extremely helpful documentation about the MarkUs Automated Test Engine from last term’s developers:

https://docs.google.com/document/d/1IFfGQ3VIYq8p2ZSy13GJPkOAVoaAw-zRcrQOj9LyKuQ/edit#heading=h.b00omv8gqmm1

Thanks a lot guys!

- Removed a stray comma in this one file, and made a pull request. Avery minor typo, but with the comma there the Config UI won’t load at all.

- Decided that I’d work on UI.

Roadblock
- The document might be a bit out of date. There are several “UPDATE”
remarks saying that certain features should be implemented. But those
features are already implemented (which isn’t a bad thing).

My concern is that there are several screenshots in the document, and
I was planning to just follow them when working on the UI. But now I’m
worried those screenshots may be from old design plans.

Next week
- Start working on the Config UI. The bullet points on page 13 look good
to me and I’d like to start implementing some of them.

Mike Stewart

Status
- Going through the Graders tab under assignments to see how much of it can be used for the grade entry form.
Roadblocks
- Need to connect with Oussama to create a plan so that we don’t duplicate work.
Next week
- Start setting up graders for the grade entry form.

Oussama

Status
- working on giving TA’s the ability to enter grades
- reviewing relevant existing code under the grader’s tab

Roadblocks
- Had to submit multiple assignments and write a term test this past week

Next
- Attempt to recycle parts of the code as the functionality already exists for a different user role
- Plan possible changes to database schema

Daniel

Status
- Continued work on issue 19, though now that I’ve realized notifications are occasionally displayed at different levels in the page, I may have to re-think the solution.
- Read up on RESTful API design
- Tested and reviewed the code of the current Markus API
- Started a discussion and analysis of the current API via issue 1012

Roadblocks
- I could not get /api/submission_downloads working yet – I keep getting a 422 response despite having what would seem to be a valid request.

Next
- Update the documentation for the existing API
- Discuss with Karen what other new features, aside from gradebook entry, she’s like to see in the API

I’m currently researching areas where Markus performance may see improvement. This involves reading through major parts of the wiki, particularly those relating to the schema and design, as well as previous issues and parts of the code base on Github. The blog is also a great source of information, as prior contributors have mentioned performance issues. Severin even did a complete benchmark of Markus: http://blog.markusproject.org/?p=3383

His findings, throughout the 4-part analysis, include the cost of subversion interactions. In part 3, he mentions that it’s likely that the primary contributor to heavy IO during peak traffic involves svn: subversion repository creation and file storage on existing repos. But can this be improved upon?

One of the future goals for the Markus Project includes a transition from Subversion to Git. I love Git, and Grit is a well-documented ruby gem for interacting with Git repositories. Much better documented than the svn ruby bindings. But given a switch from svn to git, could we expect improved performance?

Much of the git community claims that git is much faster than svn: https://git.wiki.kernel.org/index.php/GitSvnComparison The git website even presents some benchmarks for comparison: http://git-scm.com/about/small-and-fast The numbers are promising, but unfortunately, the commit operations include pushing, which I don’t believe is part of Markus’ operations. They also don’t involve grit nor the svn ruby bindings. So to research whether or not Markus’ peformance could improve from a transition to git/grit, I figured I’d have to write a small benchmark.

The script

The code can be found in a gist: https://gist.github.com/4658271

The code discussed completes 4 operations, and outputs their running time:

• Initializing 10,000 svn repos
• Initializing 10,000 git repos
• Initializing and commiting a small test file to 10,000 svn repos
• Initializing and commiting a small test file to 10,000 git repos

The test file that’s committed is called ‘test.txt’, and contains the string “hello world”.

The 40,000 repositories is a small enough sample size such that my laptop can complete the script in a bit over 10 minutes. I may come back to this with a larger sample, leaving it to execute over night, at a later time.

And note that the script keeps a reference to all those repository objects, so it uses a fairly large amount of memory. We aren’t setting things to nil to give the garbage collector time to go through while the process is running.

The Environment

Before getting to the results, here’s some details on the benchmark environment:

OSX 10.6 Snow Leopard
2.4Ghz Core 2 Duo
4GB ram
svn version 1.7.8 (r1419691)
git version 1.7.12.1
grit (2.5.0)

Results

And now for the numbers:

• Creating 10,000 svn repos: 119184.502 ms
• Creating 10,000 git repos: 120582.207 ms
• Creating and commiting to 10,000 svn repos: 285041.101 ms
• Creating and commiting to 10,000 git repos: 223160.437 ms

From this simple experiment, we can see that initialization was only minutely slower with git than svn, which comes as a surprise given the smaller size of git repositories. But we start to see a stronger contrast once committing files was involved, and svn showed to be ~22% slower in comparison. If initialization wasn’t included in those operations, the difference should be even greater.

I should also mention the size of the generated directories. 20,000 repositories were generated with both git and svn, making for 40,000 in total. The combined size of the 20,000 SVN repositories, of which half had our test file committed, is 594 Mb. And git? 278 Mb. That’s less than half the size!

And so given the benchmarks previously available, as well as the results from this simple benchmark, I agree with the majority: a switch from SVN to GIT should help improve performance. It may not be a high priority at this time, but with svn interactions being such a bottleneck under heavy load, it should help alleviate some of the strain. As such, though I’ll continue researching areas where Markus may see performance benefits over the coming week, I’ll keep this transition in mind as a possible goal for this work term. It would certainly be a large task to complete.

If we weren’t having so much trouble with svn-bindings today, we would really be having fun at the UCOSP code sprint at Facebook headquarters.

Here is some information (and hopefully pictures) of the team that is working on MarkUs this term:

Marc Bodmer

Hi,

My name is Marc Bodmer and I am a 4th year student finished up my last term at the University of Guelph. I am into front end web development, working on side projects with new web technologies at every chance I get. Along with MarkUs, I am also working on another project related to conference planning. I have written a short tutorial book on Ember.js called Learning Ember.js that will be released in February, 2013. I love attending conferences and meetups. In my spare time, I play squash, rollerblade and snowboard.

If you want to find out more about more or find out my accounts on various social networks, you can visit my website.

Daniel St. Jules

Hi, my name’s Daniel St. Jules, and I’m a 4th year student at the University of Ottawa. I joined MarkUs with an interest in working on a larger Rails code-base, and because I really enjoy developing web-based projects. Some of my projects can be found on http://danielstjules.com/ and on my Github. Aside from development, my hobbies include snowboarding and basketball.

Alysha Kwok

Hi everyone,

My name is Alysha Kwok and I’m finishing up my last term of Computer Science at the University of British Columbia. I am most comfortable working with Java and have worked on mobile applications (iOS and android) on co-op and course projects. I am new to Ruby on Rails but am eager to learn. My hobbies include graphic design, ballroom dancing, and yoga.

Here’s a picture from the code sprint at facebook so you can put a face to the name =]

Taken at Facebook, Palo Alto

Mike Wu

Hi, my name is Mike Wu, and I’m in my last term of undergraduate studies at the University of British Columbia. I grew up in Beijing, China, and I have been into computers and technologies ever since I was a kid. From co-op and courses, I have become familiar mostly with Java and desktop application development, so this project is a great opportunity for me to learn more about web technologies through Ruby on Rails.

My other interests include sports (especially basketball), photography, traveling, and music. I’m hoping to learn a few tricks to make my website more awesome!

Vancouver 2010 Olympic Torch Relay. Prince Rupert, BC.

Ian Smith

Mike Stewart

My name is Mike Stewart, and I’m in my fourth year of Software Engineering at the University of New Brunswick in Fredericton. I’ve completed 4 co-op software development terms at different companies, mostly working with Java and C++,  as well as some web development. I’ve also developed apps for Android and Blackberry and have put together a few websites over the years, though I’m completely new to Ruby on Rails. I look forward to working with this very talented student team, and learning even more about web development!

Other than development, some of my interests include skiing, SCUBA diving, and volunteering with a few organizations.

Nick Lee

Oussama Ben Amar

Daryn Lam

Hi everyone,

My name is Daryn and I’m in my last year at UofT studying computer science. I’ve worked on MarkUs in the past summer so feel free to email me directly if you have any questions (daryn.lam@gmail.com). I had an internship at IBM last year, also working with Ruby on Rails, developing an information management system that organized events and data which were crucial to my team. On my off-time I like to get active and play sports such as basketball, ultimate frisbee, and dragon boat. Any sports fans out there? Anyways, I hope to get to know everyone better over IRC even if I don’t get the chance to meet in person.

Daryn =)

As a developer coming from a  traditional PHP/MySQL stack environment, representing Object Oriented models in Rails can be a difficult concept to comprehend using the built in ActiveRecord abstraction layer. In this post I will attempt to outline most common way to implement an inheritance hierarchy and as a result limit opportunities to write repeated Model validation code,  create attribute repitition, and finally to limit the introduction of refactoring bugs into your rails project.

For example purposes, we will use a MarkUs specific case. As part of the student code testing framework, the MarkUs application is required to keep track of a number of files submitted by either TA’s or professors which contain either test scripts themselves or test support data required to drive their test scripts.

Test Support File

File Name – String, Description – String, Assignment ID – Integer

Test Script - I’ve abstracted some of the fields in this object for purposes of this post

Script Name – String, Description – String, Assignment ID – Integer

Display Results – Boolean, Run On Submission – Boolean

As you can see, there are three fields in both models that overlap. On top of this, since File Name and Description require user input that we can’t control, we are required to validate/sanitize these attributes before we can allow them to enter our database for security reasons. As you might imagine, these requirements could easily be implemented using polymorphism allowing the base implementation of both Test Scripts and Test Support Files to be handled in one ruby Model. This would roughly cut our validation code in half, make database schemas much easier to change and implement, and finally fewer database queries would be necessary.

Unfortunately our current implementation consists of two completely separate ruby Models. This has created the need for redundant code in the following places:

Controllers – Automated_Tests_Helper.rb: add_test_script_link, add_test_support_file_link, and process_test_form

Models – Test_Support_File.rb & Test_Script.rb: write_file, delete_old_file, sanitize_filename, delete_file, and many of the validation statements

Views – _test_support_file_upload.html.erb & _test_script_upload.html.erb & _form.html.erb

As you can see, this makes refactoring, upgrading, and bug hunting much more difficult. For this reason I have outlined below the simplest way inheritance can be implemented in the MarkUs project.

Single Table Inheritance(STI)

The method we will be covering is that of STI. “In a nutshell, STI allows you to create subclasses of a particular database table. Using a single table, you can cast rows to specific objects that extend the base model.”(Water Cooler) As you may have inferred from the name, this supports inheritance from only a single super class as well. This works mainly by adding an inheritance column to the table(model) you’d like to extend from. By default Rails looks for a column named “type” that is of type string to store this information in.

Essentially this works by using column “type” to store  model names which are represented in that table allowing us to determine which type of model we are dealing with for the given row. One very important note, and downside to consider, is that any additional columns that are needed to represent subclasses will also be included in the same table. This means that there could be potentially many additional null columns that aren’t used by the base class but would still play a factor if a table join is performed for any operations.

Taking a MarkUs specific example:

create_table “test_file”, :force => true do |t|

t.integer              ”assignment_id”

t.string                 “file_name”

t.string                 “description”

t.string                 “type”  //this handles the inheritance look ups

end

class TestFile < ActiveRecord::Base

#common file implementation here

end

test_script.rb:

class TestScript < TestFile

#specific implementation here

end

test_support_file.rb:

class TestSupportFile < TestFile

#specific implementation here

end

As you can see implementing inheritance is accomplished quite simply using this technique. Once you have implemented these model objects and have populated the database with different types of objects, you can access them using standard select or find calls using ActiveRecord. Rails will automatically convert the returned object into their specified model types.

That about concludes my basic overview of Single Table Inheritence. Hopefully this has been useful and will be considered for future versions of the MarkUs Testing Framework. For another great explanation, read: http://code.alexreisner.com/articles/single-table-inheritance-in-rails.html

Though this post is probably coming a little late in the term to help anyone out with their Rails debugging right now, I’m hoping it might at least give a few people some insight onto how you can quickly debug in Rails without using an IDE, and also to newcomers to the Markus project who are also experiencing Rails for the first time.

Probably the first thing you’ll want to know, is how the debugger is included in the Rails project, and how you will have access to it.  By looking at the “Gemfile”, which specifies what “add-ons” our rails project is using, in the root of the project we see that the debugger gem is included for the Development and Test environments:

gem "debugger", :platforms => :mri_19

We need to include the debugger to have access to it, and if we check the config files for development and test under /config/environments, the lines:

require 'ruby-debug' if RUBY_VERSION == "1.8.7"
require 'debugger' if RUBY_VERSION > "1.9"

Show us that it will always be available for every file in these 2 environments. This enabled us to use the “debugger” command, to stop execution in any file, and then inspect the current state, modify variables, and run commands both under “rails server” and when running unit and functional tests.
For example, let’s put a “debugger” statement in the assignments controller:

# Called when editing assignments form is submitted (PUT).
def update
debugger
@assignment = Assignment.find_by_id(params[:id])
@assignments = Assignment.all
@sections = Section.all

When you now submit an update on the assignments page, you’ll notice that the page response stalls. If you check the console containing the rails server, you’ll see that you now have an interactive prompt:

/home/ioev/Projects/Markus/app/controllers/assignments_controller.rb:188
@assignment = Assignment.find_by_id(params[:id])
(rdb:1)

Here we see a listing of the line number the debugger stopped at, and the contents of the line as well. If you type “help” at the prompt, you’ll get a listing of all of the available commands:

(rdb:1) help
ruby-debug help v0.10.4
Type 'help ' for help on a specific command

Available commands:
backtrace delete enable help method putl set trace
break disable eval info next quit show undisplay
catch display exit irb p reload source up
condition down finish kill pp restart step var
continue edit frame list ps save thread where

The most useful commands to us right now, are “next, continue, list, p and pp” but you should experiment with the others as well.
“next” or “n” will execute the next line of code.
“continue” or “c” will continue to the end of execution, or to the next breakpoint. You can use this to end debugging (unless you’ve got a debugger statement in a loop, where you’ll just end up at the next iteration)
“list” will list the line the debugger is stopped on, and the lines surrounding it.
“p” and “pp” allow you to print out variables, or execute any ruby code.

So if we type “l” for list, we get:
(rdb:1) l
[183, 192] in /home/ioev/Projects/Markus/app/controllers/assignments_controller.rb
183 end
184
185 # Called when editing assignments form is submitted (PUT).
186 def update
187 debugger
=> 188 @assignment = Assignment.find_by_id(params[:id])
189 @assignments = Assignment.all
190 @sections = Section.all
191
192 if !params[:assignment].nil?

And say we want to print out the current value of params, which are the parameters passed in by the form, we can type “p params” or “pp params”:
(“pp” stands for “pretty print” and will print with a little bit more formatting that the regular print, which makes it really useful for arrays and hashes. “ps” is similar.)
(rdb:1) pp params
{"action"=>"update",
"id"=>"1",
"authenticity_token"=>"EN8RfAGUY/EfKNepI6VZqWPs9rUaOGSMlCWPCS9x5Y4=",
"_method"=>"put",
"assignment"=>
{"display_grader_names_to_students"=>"false",
"marking_scheme_type"=>"flexible",
"message"=>"Learn to use conditional statements, and loops.",
"remark_message"=>"",
"group_name_autogenerated"=>"0",
"repository_folder"=>"A1",
"remark_due_date"=>"",
"description"=>"Conditionals and Loops",
"submission_rule_attributes"=>{"id"=>"1", "type"=>"NoLateSubmissionRule"},
"allow_web_submits"=>"true",
"due_date"=>"2012-11-21 16:17",
"short_identifier"=>"A1",
"allow_remarks"=>"true",
"student_form_groups"=>"0"},
"controller"=>"assignments",
"utf8"=>"?",
"locale"=>"en"}

Since “debugger” is a command, you can also use conditionals to only activate the debugger only under certain conditions. For example:

debugger if variable.nil?

Which is really useful if you’re looping through a large set of data, and are having trouble with only one piece, this will help you narrow things down.
So now that you have an idea how to use the Rails debugger, I hope you never have to use a “puts” statement and search the console for the output ever again!
Good luck!

What to Change:

• Submitted Assignment to Collected Assignments
• Order of Assignment by Descending Due Date

What to add to Assignment Information:

• Median
• Number of zeros
• Number of fails
• Number of Submitted Assignments

There is definitely more to add, but only with time and testing it will be clear what we need.

What’s C.R.A.P?

C.R.A.P stands for Contrast, Repetition, Alignment and Proximity.

Contrast

Different elements (titles, heading, body text, etc.) must look different from each other. This can be achieved by varying fonts, line thickness, colours, shapes, sizes, space, etc.

Repetition

Repeat an aspect of the design throughout the website, for example all titles will use the same font, size, weight, etc. This makes the website consistent.

Alignment

Don’t place elements arbitrarily on the page. Elements of the same importance should be aligned vertically and horizontally.

Proximity

Group related elements together, for example a heading should be closer to the text it belongs to than to the text of the previous paragraph. Physical close elements imply a relationship.

C.R.A.P at MarkUs

The Problems

Dashboard

The dashboard is overall fine. there small problems with the alignment of the assignment information.

Assignments

On the main page the proximity was violated when the class average was equally close to both assignments.It should be closer to assignment 1 and farther from assignment 2. It was also directly under the assignment. It is part of the assignment therefore it should be under the details of the assignment. The rounding is also different then the one on the dashboard. All pages should follow the same rounding rules.

The submission page has options right next to the name of the assignment. While they should be underneath. The options for sorting the table were using completely different style, using buttons rather than text as the options above them. I would suggest changing all the options to look like buttons. There is the rounding issue again.

On the groups page the options for each student are at different place depending on the length of the name. They all should be in the same place right underneath each other. The tabs for managing groups should be on the same line, if there is no space then at least the search should be on a separate line then the tabs.

Annotation

The annotation page the Help is the same size of the assignment title. They should be different size and style since they are different things. The options are directly next to the title. They should be underneath and changed to buttons.

Grading

On the Grading page The previous and next submission are higher than the rest of the options for some reason. They should be vertically aligned. Same applies to the check box for released. For unreleased assignments under the summary is a complete mess. There are issues with alignment and proximity. Most of the things there are done with tables but then it just stopped and things got out of alignment.

Users

The options on this page are all of a sudden on the right and not next to title like on other pages. This breaks the repetition between pages.

Student

On the student view the assignment rules are completely different than the rest of the section. Also all the details there are on the same line while the message is underneath. This is inconsistent.

The Pictures

The Good

Over all there is good contrast through out the website. The website is pretty consistent about its’ layout and there is pretty good proximity.

Here are the punchlines from our group (students from Centrale Nantes):

Amandine & Antoine :

Status :

• Seen how we could access to the grades, to which controller it was associated, in which views it was used and how to use it.

Next steps :

• Finish views.

Road block :

• Exam to prepare.

Alexy & Nora :

Status :

• Creation of models flexible_criterion_assignment_association and rubric_criterion_assignment_association
• Correction of migration
• Modifying flexible_criterion and rubric_criterion to add relationship with new tables
• Working on the radio button for the marking_scheme of a new assignement

Next steps :

• Modifying the UI to cope with the disappearance of the current radio button (see the status)

• (Nora) Working on the final report

Road block :

• Understanding the way to add the “new” criterion (skill/objective)

Nicolas & Sébastien :

Status :

• Worked on the controllers and views. Determine which should be used to display the new marks, and understanding what needs to be changed.

Next steps :

• Because there won’t be a specific marking_scheme_type, each time it is used we need to use critere.instanceof? (flexible_criterion) instead.

Road block :

• Exam to prepare

Step 1 – Make An Issue

Every bug you encounter needs to have an issue on Github. Even if it’s a big messy bug that you can not consistently replicate. Without an issue it will likely be forgotten. Forgotten bugs will never be fixed, and a two sentence Github issue will at least let someone know to look further. Ideally you will have time to move onto the next steps, but at the very least make an issue.

Make sure your title and brief description actually describe the bug.  Try to say more than just “it’s broken.”

Step 2 – Reproduce The Bug

You need to specify exactly what steps are needed to reproduce the bug. You are not writing a reminder for yourself. This information should be sufficient for someone else on a different machine to go and see the bug themselves. It should also contain only pertinent information, try to have a minimal number of steps required for reproduction. Clearly state what the proper outcome should be, and then explain how the current behaviour doesn’t match that.

Step 3 – Diagnostic Information

If the bug is a tricky one it is helpful to also include additional useful information, such as:

• Ruby Version
• Rails Version
• Database
• What version of the code is running

Step 4 – Comments

If you find additional information about the issue you should add it as comments. Again, this is for the benefit of others as well as a place to keep track of your own progress. If you are working on the bug you should indicate so, and post the results of your debugging.  If you disappear it’s nice if someone can pick up right where you left off.

Here are some good MarkUs issues to use as examples:

https://github.com/MarkUsProject/Markus/issues/872

https://github.com/MarkUsProject/Markus/issues/861

https://github.com/MarkUsProject/Markus/issues/755

The last one is a duplicate issue, but includes great detail!

Good luck bug hunting!