MarkUs Blog

MarkUs Developers Blog About Their Project

Archive for April, 2012

Recent MarkUs RESTful API Changes

without comments

As many of you may have heard I have been working on updating and adding on to the existing MarkUs RESTful API. In this blog I will cover some of my previous and upcoming changes to the API. For those who are not familiar with REST or looking for more background about our API, feel free to read up on Danesh’s blog post: http://blog.markusproject.org/?p=2866

 

Submitted Changes:

1.) Error messages that are created by us have been remade. Before we would reference the error specific HTML/XML files found in the /public directory. We now have HTML/JSON/XML/TEXT templates for various format types. A pretty straight forward example:

render 'shared/http_status', :locals => { :code => "200", :message => HttpStatusHelper::ERROR_CODE["message"]["200"] }, :status => 200

This type of call will return a 200 OK response with the given message in whatever format the user has specified through the URL or  the HTTP Accept header, which brings me to my next change.

2.) Responses are now returned in the user specified format. Currently this only works for collection/member GET requests for the users API and any error responses for any API controller. In addition I am using the same methods for the upcoming assignments API.

Upcoming Changes:

1.) In the past, to retrieve a specific user we would query: http://localhost:3000/api/users/1?user_name=Joe

In this case, the 1 is a completely irrelevant number and could really be any ( valid ) string. This may have been implemented so because of the restrictions placed on us by Rails for using the resourceful routing.

Consider the GET routing table entry for the users member: GET /api/users/:id(.:Format). As developers we would have to retrieve params[:id] to get the value passed in through the URL and this may seem misleading as this value would then be treated as a user_name inside the function*.

It is not easy to change :id to :user_name within the routing table and I see this as a necessary evil, introduced by Rails, that we will just have to cope with. As such, I have made the change to remove the need for the user to specify these garbage ids and the above example now looks much cleaner as:  http://localhost:3000/api/users/Joe

* Note that we only work with ids for URLs on the main website, but for the API we prefer to deal with the much more friendly “names”. Of course these names have to be set to be unique within the model.

2.) Reworking the routing to nest resources within each other. As an example, before we may have queried the following URL to retrieve test_results: http://localhost:3000/api/test_results?group_name=Group&assignment=A1

As suggested by Severin, I have made the change to change the above example to:  http://localhost:3000/api/groups/Group/assignmentA1/test_results

While the bellow URL may seem to seem more readable, I cannot help but feel that we are flirting with the dark side of Rails here. During my routing work I have found a recommendation made by Rails developers: “Resources should never be nested more than 1 level deep.” as can be seen on http://guides.rubyonrails.org/routing.html#limits-to-nesting. Are we really doing the right the right thing by making this change? I am open to suggestions.

3.) Last but not least, I will be adding the API for assignments. I will not be doing anything too crazy here, but instead stick to the functionality that we already provide through the website. For those brave enough, I have started uploading my changes here: http://review.markusproject.org/r/1206/

Written by ephilippov

April 7th, 2012 at 10:40 pm

Minutes for UCOSP meeting (02/04/12)

without comments

Sky

  • Blog post on shoulda! It will be a great start for new devs being introduced to testing.
  • Worked on #150, creating repos for students who join late.
  • Finished off moving custom validation into validation helper.

Hanson

  • Mostly in status report.

Aaron

  • Still questioning how Period.make pushes the new periods into the submission’s period array.
  • Meeting with Hanson and Christine to form a joint intro for their presentation on their work on MarkUs this term.

Jay

  • Cracked the reason behind extra grace day deduction. If A1 is handed in and then a submission for A2 is made within the timeframe of A1’s grace days, a grace day is deducted.
  • Debugging it further is problematic because the code starts a seperate thread/process to figure out the timestamps. This is unreachable via normal debugger line.

Christine

  • Received feedback on routes test review, figuring out how to implement the others.

Me (Danesh)

  • Changed bug reporter modal into a general help modal. This is now shown to all users, where non-admin users are pointed to their userguides. Only admins see the links to the bug reporting tools, on top of their userguide link.
  • Prototype removal slowly coming to an end!

Aimen

  • Status report says it all.
  • Presentation for the course needs to be a poster so there will most likely be no slides. 🙁

Sean

  • The modal for #689 is almost complete, it filters the columns properly. The per page button is broken when loading the view, it resets to 30 everytime it is changed.

Egor

  • MIA ( Egor: I must be going crazy…thought today’s meeting was cancelled due to Karen’s bday >_< )

Tobi

  • Researched integrated testing to be used for unit tests across different controllers.
  • Various reviews and merges.

 

Open floor

  • Post-mortem next week, fill out the doodle if you haven’t.
  • MarkUs party on Saturday, the 28th of April! See email shortly.
  • Research In Action showcase of MarkUs went well. Industry folks shocked as usual that MarkUs has loads of undergrad devs who do fantastic work. 😉

Written by Danesh

April 2nd, 2012 at 10:03 pm

Punchlines UCOSP Group – 4/2/2012

without comments

Sky
Status:
-Shipped #474
-Posted my blog writing: “understand unit test with gem ‘shoulda'”
-worked on #150
Roadblocks:
-stuck in code of #150, need time to clear my mind
Next steps:
-kill #150

Hanson
Status:
– Implemented fix for issue 717, it is sitting in RB
– Tested issue 683 and it is not reproducible (Severin said Egor might have fixed it – action: confirm with Egor)
– Started working on issue 730 (can someone formally assign the bug to me :))
– Also taking issue 729
Roadblocks:
– Tobi and Aimen gave some suggestions and tips for my search functionality issue in RB, but still waiting for a Ship-It 🙂
– Same case with my fix for 717 which has not been reviewed by anyone yet
Next Steps:
– Ship issues currently in RB
– Fix 729 and 730 issues

Aaron
Status:
– made blogpost about Mongrel
– working on outline for UCOSP presentation
– some UI rework discussion on 721 with Nelle (in Pull request state)
– 648 is in review board, has been reviewed and recommended fixes have been applied, still waiting for shipping cue
Roadblocks:
-none
Next Steps:
– ship 648

Christine
Status:
-Wrote route tests for Admin resource & API test results resource,
posted to ReviewBoard
-Looked into the helpful comments from Danesh in his review

Next steps:
-Complete route testing for all resources
-Figure out how to properly test routes that should not exist

Road blocks
-None

Jay
Status:
– Continue to debug through and investigate issue 656
– Thanks to Danesh, got a better understanding of the code but will still require some more work

Roadblocks:
– Swarm of assignments 🙁

Next Steps:
– Fix issue 656 asap!

Danesh
Status:
– Updated the bug reporter review, posted my changes that turned it
into a helper modal.
– Added the helper modal for TAs and students, it only links to the
userguide however.
– Changed assert_recognizes from prototype rework tests to
assert_routing since this is more desirable. Some issues popped up
with params though.
– Continued work on prototype helper removal, still quite a few helpers left.
– Helped Aaron with his issues, attempted to help Jay trace his test
(we didn’t get too far but I [think I] guided him in the right
direction). Went over many reviews, commented on a few and closed ones
that had code pushes already.

Next week:
– Try to get code stuck in reviews out the door.
– Not that much time on MarkUs, I have an exam on Friday.
– Post-mortem meeting?

Roadblocks:
– Exam on the 13th, I won’t be able to spend that much time on MarkUs.

Aimen
Status:
Worked on the presentation for MarkUs
Fixed the display of the Modal and working on Session for “Per Page”
Did some reviews for the issues posted on Review Board

Roadblocks:
-None

Next steps:
-Finish working on the presentation.
-Post my final diff for issue689 on Review Board

 

Sean
Status:
-worked on issue 689, almost done!

Roadblocks:
-Time; everything is due this week :S

Next steps:
-finish up 570
-finish up 689 with aimen

 

Tobi

Status:

  •  Pushed issue 669
  • Working on the integration unit test for 697 and 704
  • Did some reviews

Roadblocks:

  • When writing the unit tests for releasing assignment marks by admins and viewing the grades by students, I reached a road block. I previously used Webrat an additional Gem to write the test but after the reply to my email to markus-dev, adding a new dependency is not needed as I can use Rails integration tests for it. The test I’m currently writing will have to be a test of the entire process of an admin collecting assignments, the TAs marking them, and setting the status to completed and then releasing it.
  • Waiting for more “ship it”-s for issue 339
  • Logged some issues
Next Steps:
  • Finishing the unit tests
  • Closing other reviews

Written by alee

April 2nd, 2012 at 5:59 pm

A revisit to Mongrel, the pros and cons

with one comment

If you know about Mongrel already, chances are that you probably heard that we are moving away from using this in our production setup since it is not maintained anymore.  But before we completely depart from Mongrel (since we still use it I believe in production), I want to write a short educational blurb about it.

First off, here is some background for people unfamiliar with Mongrel.  Mongrel is an open-source HTTP library and web server which Ruby on Rails applications run on top of.  This setup is usually seen in the production environment for a Rails application.  Mongrel is written by a combination of Ruby and C, and can be run on Windows and any Unix-like systems (Linux, BSD, OSX, etc.).  Fun fact, Twitter used Mongrel as their very first web server!

The next question you’re probably wondering about is why are we moving away from Mongrel?  The simple answer is, its being maintained anymore and doesn’t support Ruby on Rails 3 which is what MarkUs will be running on soon.

Next I will discuss the pros and cons that Mongrel brings:

Pros:

– Can get good stability and speed when handling real-life(production) load when used in conjunction with other web server applications, a common configuration seen in production is having an Apache HTTP server working as a load balancer, proxy and handles concurrency delegation over a cluster of Mongrel servers

– Uses its own HTTP library which has a much higher performance and stability than conventional CGI scripts such as FastCGI and SCGI (this is what caused the original movement from CGI to Mongrel)

– Mongrel services use TCP which offers security and reliability in data transmission

Cons:

– Doesn’t support Ruby 1.9 and on!

– Does not handle concurrency well by itself (a big requirement especially if there will a lot of traffic coming in and out), needs to bring in other web applications such as Apache HTTP server which handles the load balance and concurrency tasks

– Awful speeds when it comes to serving static files (static html pages, CSS files, Javascript files, pictures) which is why configurations with other web server applications are commonly seen (Apache dominates in this area)

– Nowadays web server applications such as Thin (a web server which uses Mongrel’s library and parser), Passenger and Unicorn all provide much easier deployment processes as well as faster speeds through using Unix socket connections to the web server rather than TCP, not to mention you can easily run Rails 3 applications on top of these web servers

– No support for tracing, which is a useful ability that show developers the exact HTTP transactions that are going through, web servers like Thin can do this

I hope this post helps educate you a little bit about Mongrel, and its ups and downs.  I found this to be a very interesting topic to research on since Mongrel’s original author, Zed Shaw is quite the interesting character.  One of the main reasons why the releases of Mongrel stopped was because he left the Ruby on Rails community after making a public rant on rails programmers (he likes to use a lot of vulgar language, beware!).

Here is a link that I found was useful comparing Mongrel with Thin: http://www.wikivs.com/wiki/Mongrel_vs_Thin

Here is another link from stackoverflow  which talks a bit about different Rails web servers in production: http://stackoverflow.com/questions/1728978/recommendations-and-differences-between-different-ruby-on-rails-production-web

 

Written by alee

April 2nd, 2012 at 6:47 am

Understand unit test with gem ‘shoulda’

without comments

It was my first time to learn about unit test and the ‘shoulda’ gem in my recent ticket. I found it hard to learn it from scratch without any context. So I wrote this very simple introduction on unit test (specifically, with ‘shoulda’ gem) to provide you with a point where you can start to learn.

‘Shoulda’ is a meta gem that makes testing code looks simple and clear. Instead of writing ruby method with a lot of underscores, testers can use ‘context’ and ‘should’ keywords provided by ‘shoulda’.

One obvious advantage is that it makes test code reads more like natural language: to describe a specific test, first you need to mention the context, and then you must claim in this context, what behavior should appear to pass the test.

I will use the code in grouping_test.rb for example:


require 'shoulda'

context "A grouping" do
  setup do   
    @grouping = Grouping.make
  end
  
  should "not have any ta for marking" do
    assert !@grouping.has_ta_for_marking?
  end
  
  should "not have submissions" do
    assert !@grouping.has_submission?
  end
end

The keyword “context” is to describe the context of the test. Its following string do not have effect on the test result, only to make the test clear to read. The “setup” keyword is to setup any preparation work before each “should”. The code in “setup” will be executed every time before it run into another “should” test case. Similarly, we also have a “teardown” keyword to do cleanup after each “should” test case.

The string following the keyword “should” is the test case title. A good test case title should be clear and specific because it will appear in the output message if this test case fails.

Moreover, “context” can be nested to setup complicated situation while keeping the code clear. We can demonstrate this by adding a inner test case to the previous example:



require 'shoulda'

context "A grouping" do
  setup do
    @grouping = Grouping.make
  end

  should "not have any ta for marking" do
    assert !@grouping.has_ta_for_marking?
  end

  should "not have submissions" do
    assert !@grouping.has_submission?
  end

  context "and two unassigned tas" do
    setup do
      @ta1 = Ta.make
    end

    should "be able to add ta" do
      @grouping.add_tas(@ta1)
    end
  end
end

In the test case of “be able to add ta”, setup of both outer context and inner context will be executed in order before the test starts.

For more information about ‘Shoulda’, please take a look at:

A blog post from a previous MarkUs developer

A video tutorial about Shoulda on rails

Github page of Shoulda source code and document

Written by Sky

April 2nd, 2012 at 12:18 am