MarkUs Blog

MarkUs Developers Blog About Their Project

Archive for the ‘Remarks’ Category

From SVN to Git

without comments

I’m currently researching areas where Markus performance may see improvement. This involves reading through major parts of the wiki, particularly those relating to the schema and design, as well as previous issues and parts of the code base on Github. The blog is also a great source of information, as prior contributors have mentioned performance issues. Severin even did a complete benchmark of Markus: http://blog.markusproject.org/?p=3383

His findings, throughout the 4-part analysis, include the cost of subversion interactions. In part 3, he mentions that it’s likely that the primary contributor to heavy IO during peak traffic involves svn: subversion repository creation and file storage on existing repos. But can this be improved upon?

One of the future goals for the Markus Project includes a transition from Subversion to Git. I love Git, and Grit is a well-documented ruby gem for interacting with Git repositories. Much better documented than the svn ruby bindings. But given a switch from svn to git, could we expect improved performance?

Much of the git community claims that git is much faster than svn: https://git.wiki.kernel.org/index.php/GitSvnComparison The git website even presents some benchmarks for comparison: http://git-scm.com/about/small-and-fast The numbers are promising, but unfortunately, the commit operations include pushing, which I don’t believe is part of Markus’ operations. They also don’t involve grit nor the svn ruby bindings. So to research whether or not Markus’ peformance could improve from a transition to git/grit, I figured I’d have to write a small benchmark.

The script

The code can be found in a gist: https://gist.github.com/4658271

Screen shot 2013-01-28 at 4.26.14 PM

The code discussed completes 4 operations, and outputs their running time:

  • Initializing 10,000 svn repos
  • Initializing 10,000 git repos
  • Initializing and commiting a small test file to 10,000 svn repos
  • Initializing and commiting a small test file to 10,000 git repos

The test file that’s committed is called ‘test.txt’, and contains the string “hello world”.

The 40,000 repositories is a small enough sample size such that my laptop can complete the script in a bit over 10 minutes. I may come back to this with a larger sample, leaving it to execute over night, at a later time.

And note that the script keeps a reference to all those repository objects, so it uses a fairly large amount of memory. We aren’t setting things to nil to give the garbage collector time to go through while the process is running.

The Environment

Before getting to the results, here’s some details on the benchmark environment:

OSX 10.6 Snow Leopard
2.4Ghz Core 2 Duo
4GB ram
svn version 1.7.8 (r1419691)
git version 1.7.12.1
grit (2.5.0)

Results

And now for the numbers:

  • Creating 10,000 svn repos: 119184.502 ms
  • Creating 10,000 git repos: 120582.207 ms
  • Creating and commiting to 10,000 svn repos: 285041.101 ms
  • Creating and commiting to 10,000 git repos: 223160.437 ms

From this simple experiment, we can see that initialization was only minutely slower with git than svn, which comes as a surprise given the smaller size of git repositories. But we start to see a stronger contrast once committing files was involved, and svn showed to be ~22% slower in comparison. If initialization wasn’t included in those operations, the difference should be even greater.

I should also mention the size of the generated directories. 20,000 repositories were generated with both git and svn, making for 40,000 in total. The combined size of the 20,000 SVN repositories, of which half had our test file committed, is 594 Mb. And git? 278 Mb. That’s less than half the size!

And so given the benchmarks previously available, as well as the results from this simple benchmark, I agree with the majority: a switch from SVN to GIT should help improve performance. It may not be a high priority at this time, but with svn interactions being such a bottleneck under heavy load, it should help alleviate some of the strain. As such, though I’ll continue researching areas where Markus may see performance benefits over the coming week, I’ll keep this transition in mind as a possible goal for this work term. It would certainly be a large task to complete.

Written by Daniel St. Jules

January 28th, 2013 at 5:21 pm

Posted in Uncategorized

Migrating Prototype to jQuery – MarkUs Term Goal Winter 2013

without comments

My long term goal for MarkUs will be to port over our Prototype framework code to jQuery. I have had to do some research on this to find out how complex will this be and what is involved. Here are my findings.

Complexity

Since jQuery has such a huge developer community around it, finding replacement plugins for some of Prototypes features should not be hard. The MarkUs code base will have to undergo:

  • A complete Prototype code refactoring to jQuery. Refactoring javascript_include_tags to include jQuery, modifying Prototype creation code for Modals, Windows, Tabs (Included under the LivePipe UI Framework) and Prototype feature detection code blocks.
  • New plugins (jQuery equivalent) if no other substitutes can be found
  • Tests. It seems that as of right now only Functional and Unit tests are up to date in MarkUs. Acceptance tests may have to be written using Cucumber, in which case we can also include Javascript testing as a part of Cucumber.
  • Documentation. I want to document everything I create, most frequently used methods, and any plugins / extensions used for jQuery.

Major Differences Between Prototype & jQuery Syntax

Dollar Variable Selectors

  • Prototype uses $ for ID based selections
  • jQuery uses $ as a CSS based selector (Equivalent to $$ in Prototytpe)

DOM Ready Event

  • Prototype uses an Event object
  • 2 different ways to do this exist in jQuery

DOM Manipulation and Traversal

  • Different methods exist within each framework.

Events

  • Prototype uses Event class methods.
  • jQuery uses bind, unbind, and various other shortcut methods.

Ajax Requests

  • Different syntax between the two frameworks.

Tasks needed to complete

Going over the source code of MarkUs quickly, here are some tasks I see right away that need to be completed.

Replace UI Elements Created by LivePipe

The jQueryUI library may have to be used in order to replace what was provided by the LivePipe library. LivePipe and associated libraries, such as scriptaculous provided:

  • Tabs
  • Windows (Modals)
  • Dragging
  • Dropping
  • Sorting

As a result of this, some of the basic UI elements such as modal dialogs, draggable lists, tabs, may look just a little bit different.

Port over Prototype Classes to jQuery

Prototype is very OOP oriented and abstracts Javascripts prototype (different from the framework) functionality. jQuery requires you to use Javascript’s prototype functionality to extend jQuery and create your own custom objects.

Replace Prototype Feature Detection

Cases such as the following will need to be ported over to jQuery.

<%# Alert Internet Explorer users of incompatibility %>
      if (Prototype.Browser.IE) {

The Modernizer framework can be used in case jQuery lacks any advanced feature detection methods that Prototype may contain.

Written by Marc Bodmer

January 27th, 2013 at 5:14 pm

Posted in Uncategorized

Old Results not Showing when Remarking

without comments

This problem was raised in issue904. The design of Markus’s current remark request feature is, when a grader is remarking a submission, he/she should be able to see the old mark of each rubric category as well as the total before giving new marks(as shown below).

But old marks do not appear!

As it turns out, old marks will show up after the grader marks a category and refresh the page. When I first saw this issue and try to reproduce it on our demo server and my own local server everything works fine. I also learned that the server is running ruby 1.8.7 so I decided to setup RVM+ruby1.93 at home to test it which wasted me quite a few hours with no success due to svn binding.

Meanwhile, it was found out that this issue only happens when one is using PostgreSql. Lesson learned: although including configuration details in an issue might be tedious, those details sometimes helps. However I do like the fact that Markus project has no strict rules on how to “compose” an issue so one can decide what info is useful and what is not.

After yet a few hours of struggling with setting up RVM ruby 1.8.7 + PostgreSQL, I finally got Postgresql working by giving up RVM. I still believe RVM is great but somehow it doesn’t work for me.

In results_controller.rb, somewhere around line35 we have:

if @submission.remark_submitted?
@old_result = @submission.result

But debugger shows that @submission.result is not giving us the old result. As pointed out by Joey(ioev), this is because in submission.rb, the definition of the Submission class states:

has_one    :result, :dependent => :destroy
……
belongs_to :remark_result, :class_name => “Result”

The “has_one” relation means that the Results table has foreign keys that points to rows in Submission table. See the below example(from Rails tutorial):

But Submission also belongs_to remark_result which is also a row in the Results table. Below is an example of belongs_to relation(also from rails tutorial):

So we can have two rows in the Results table that points to the same submission. @submission.remark_result should give us the desired result because it’s one-to-one, but @submission.result can return the wrong result because database will just find any result in the Results table that has a matching submission id and return that.

Joey also noticed the time-stamps of the result entries and how they affect which result will be returned by the database. In results_controller.rb, update_remark_request is the method that creates the remark_result(a new result in the Results table). By swapping the order of the following two line:

@result.save
@old_result.save

to:

@old_result.save
@result.save

I am able to ensure that the old result row will have an older time-stamp and will be return by @submission.result. Testings show that this removes the problem with Postgresql without introducing problem back to Mysql or Sqlite. Obviously this is not a good solution at all and I will work on a better solution based on Joey’s comments.

 

 
 

Written by antez

December 6th, 2012 at 4:13 am

Posted in Uncategorized

Userdocs updated for remark requests

without comments

This term, Vivien and I finished the remark request feature.

Instructors can:

  • allow/disallow remark requests per assignment
  • set due dates
  • specify remark request rules/instructions
  • comment on each student’s remark request
  • view old and new marks as they remark

Students can:

  • create and save remark requests, to edit later
  • submit remark requests to the instructor
  • cancel submitted remark requests
  • view old and new marks once the remark has been processed
  • view instructor’s comments on the remark request once the remark has been processed

The MarkUs Userdoc has been updated, and is available here:

https://github.com/MarkUsProject/Markus/wiki/UserGuide#remark-request-rules

Instructor and Student Guides have also been updated, and are available here:

https://github.com/MarkUsProject/Markus/wiki/Doc_Admin

https://github.com/MarkUsProject/Markus/wiki/Doc_Student

Written by misa

April 14th, 2011 at 1:08 am

Posted in Uncategorized

Work Started, Work Completed, and Work Yet to Come

without comments

Whoops – we’ve already started off a brand new semester, and we forgot to write about what happened in the Fall!

So, without further delay, here’s what happened with the UCOSP team last semester for MarkUs:

Projects that we started…

There were 3 main projects last semester.  In no particular order, they were…

Re-mark Requests

Vivien and Misa were tasked with implementing a re-mark request feature.  This feature would allow students to request re-marks, and have graders / instructors issue re-marks, all within MarkUs.

    Dashboard Statistics

    Hora and Kurtis were asked to put some graphs into the MarkUs dashboard to display mark distributions and other useful data.

    Automated Testing Framework

    Evan was put to work tackling the long-awaited automated testing framework feature.  The idea is that students (and graders) should be able to run submitted code through a series of testing suites, and get the test results back, all within MarkUs.

    You can read about each project in more detail here.

      What got finished…

      Misa and Vivien put together some great mock-ups for the Remark request GUI, sorted out a new database schema to support it, and wrote the majority of the code to get the feature working.  There are still some leftover bits that still need to be merged, and Misa / Vivien will be finishing up that work in the next few weeks.

      Kurtis and Hora found a great Javascript graphing library to use (Bluff), and also figured out how to nicely cache statistics within MarkUs in order to keep the Dashboard snappy.  The code is more or less finished, and just needs to be merged.

      When he wasn’t helping us with user and developer support, Evan was kicking butt on the automated testing framework, and he got a lot done – especially with regards to the token system, which allows instructors to control how many times a student can run tests.  There is still some work to do before the automated testing framework can be considered finished, and will likely be pushed at again this semester.

      What’s left to do…

      So, for the re-mark requests and dashboard statistics, we need to do some last minute merging and polish.  For automated testing, we’re definitely going to need another round of focused development.

      Anyhow, that’s where we got to.

      Great job everyone!

      Written by m_conley

      January 11th, 2011 at 2:33 pm

      Posted in Uncategorized

      Remark Summary UI

      with 2 comments

      I am currently working on ways to display the mark summary during remarking and after remarks have been completed.

      This is what the instructor view currently looks like:

      And this is what the student view looks like:

      I am having trouble deciding on my own how the summaries should be laid out. In my opinion, the current views seem a bit cluttered and messy, but I haven’t come up with good ways to show this information. The highlighting of the old mark when remarking also seems a bit awkward to me. I am wondering if anyone out there has any suggestions that would make this look better? 🙂

      Some ideas I had:

      – Separate Remark Summary tab? We wanted to be able to compare old vs new marks on the same page though..

      – Collapsable information for old marks? Old mark info would show only if user clicks on or hovers over a button.

      Written by vivien

      December 8th, 2010 at 12:23 pm

      Posted in Uncategorized

      Revised revised DB Schema for Remarks

      with 7 comments

      Edit: the final decision was to add 2 columns to the Sumbission table. “Remark_result_ID” which links to a remark result, and “remark_request” to keep the request in text form.

      So, after a discussion during last Friday’s meeting, it seems that the best way to do this is to add a column to the Submission table and create a new Results object per remark. I think this then leads to adding two columns in the SubmissionFile table, to keep track of remark request files OR adding a blob column to keep remark requests.

      Please see previous post for the previous blog discussion.

      From what I understand of the discussion, the new schema suggested is as follows. There is a remark_results column added to the Submission table, which is the id of the new result object created for the remark.
      0) Only 1 remark_result ID may exist for one submission (no multiple remarks)
      1) If this column is NULL, then it means there is no remark requested.
      2) If there is an id there, it means there is a remark requested.
      3) If the remark result object’s marking state is “unmarked” it means that the request has still not been reviewed by the professor, and the student can still cancel his or her request. The original mark is still released to the student.
      4) (If the student cancels, the result object will be removed and the remark_results column in the Submission table will be NULL again).
      5) If the remark result object’s marking state is “partial” it means that the professor has started looking into the request. At this point, the student can no longer cancel the request. Both the original and remarked grade are unreleased to the student.
      6) If the remark result object’s marking state is “complete” then both the original and remarked grades are released to the student.

      The remark request would be saved in a text file on the server, and the file name and path saved in the SubmissionFile table under the columns “remark_filename” and “remark_path.” We could also standardize the filename so that we only need path. The other option is to save it directly into a “remark_request” column in the SubmissionFile table as a blob.

      Quick question… Do we want to save when (timestamp) the remark request was made? I guess this could be saved in the remark text file.


      Written by misa

      November 8th, 2010 at 9:49 pm

      Posted in Uncategorized

      Revised DB Schema for Remarks

      with 7 comments

      Edit: Please see updated post here

      In my previous post, I had mixed up Results/Marks with GradeEntryForms/Grades. Since assignments are linked to Results/Marks, that’s the one we should be changing.

      To the existing schema, I added a “RemarkExtraMarks” and “RemarkMarks” table (identical to the ExtraMarks and Marks tables). The “Results” table now has 2 more boolean columns “remark_requested” and “remark_begun.” Let me explain the addition of the 2 boolean columns.

      Initially, I had only planned on adding one boolean column “remark_requested” to keep track of whether there are remarks or not. However, with this schema, we run into the problem of the system not being able to distinguish between original marks being completed and released and remark requested but not processed, versus remark being completed and released. We want students to be able to see their marks until the professor has actually started looking into the remark, so the original marks need to stay ‘completed and released’ until then.

      Then, I also considered adding remark states to the existing “marking_states” column, but this would create a lot of UI complications (and a lot of bug potential!). We also wanted to avoid making a “remarking_states” column because it would create redundancy.

      After discussing with Mike, we decided on adding 2 boolean columns – “remark_requested” and “remark_begun.” Let me walk through the scenarios:
      1) Student has put in a remark request. At this point, “remark_requested” = true and “remark_begun” = false. Original grades are still completed and released to the student.
      2) Professor starts to look into the remark. This unreleases the mark to the student, marking state changes to pending, and “remark_begun” = true.
      3) Professor finishes the remark, re-releases mark, marking state changes back to complete, and “remark_begun” is still true. (“remark_begun” is still true because this helps us distinguish between stage 1 and stage 3)

      1b) If a student decides to cancel his/her remark request (which he/she can only do before the professor starts stage 2) “remark_requested” = false, and nothing else changes.

      Thank you Mike for helping me out with the schema!!

      Written by misa

      November 3rd, 2010 at 3:57 pm

      Posted in Uncategorized

      Remark Request GUI Mock-up

      with 2 comments

      Misa and I have been thinking about ways to implement the remark request feature. These are a few screenshots of our initial design.

      When a student wants to request a remark, he/she will first click on Request a Remark at the bottom of the source code. This will create a new tab ‘Remark’ on the right side where the student can make their case about a remark. Students can save, submit, or delete the request. Once a request has been submitted, it cannot be modified unless a student deletes and makes a new request. Instructor instructions (specified when they create an assignment) will also appear in this tab.

      When a student submits a request, their assignment gets a new marking state ‘remark’ with a new icon in the instructor/TA view. Then the instructor/TA can start modifying the students grade.

      This view was trickier to design because we want graders to be able to see the student’s justifications, their old marks, as well as new marks. For now, we have decided to keep the two tabbed sections and have old marks are highlighted in yellow while allowing the grader to click and assign new marks. (We still have to work out a way for instructors that use manual rubrics). The grader also has space to make his/her comments.

      After the instructor completes and releases the student’s mark, the mark breakdown and summary for both the instructor and the student will show the changes that have been made.

      Well, this is basically how the remark interface will work. I think there can be improvements, especially on the grader view, but we haven’t been able to think of an elegant way to show all the information at once. If anyone has any ideas or suggestions they’d like to add, please comment!

      Written by vivien

      October 24th, 2010 at 12:54 am

      Posted in Uncategorized