We are working on this feature, so here is my first attempt at describing the requirements.

In some of the large classes, we might have quite a few TAs grading an assignment. One way to divide them up is to assign several TAs to each part of the assignment. For example if we had 12 TAs grading an assignment we might put 4 of them on part 1, 4 on part 2, and 4 on part 3. The only mechanism we have in MarkUs to map graders to parts of an assignment is through rubric elements.  So the idea is to assign TAs specific rubric elements and then use some UI features to highlight the rubric elements when a TA opens a student submission.

Assigning rubric elements to a grader is separate from assigning groups to a grader.  A grader might mark one rubric element for all submissions, or might mark all rubric elements for a subset of the submissions, or might mark a subset of the rubric elements for a subset of the submissions.

I imagine a UI similar to the mapping of graders to groups or students.  In this case we would see a list of the rubric elements with checkboxes (?) and would use much the same mechanism to map the graders to the rubric elements.

If the instructor does not specify rubric elements for a grader,  then the default should be that the grader will mark all rubric elements.

After rubric elements have been assigned to a grader, when a grader view a submission, it should be clear which rubric elements they are supposed to mark.  I am imagining that these rubric elements would be expanded by default, and would also be coloured differently.  The graders should still be able to assign marks for other rubric elements, we just want the ones that they have been assigned to work on to stand out more.

As always, feedback and comments are welcome.  What am I missing?

The MarkUs team is pleased to present version 0.6.2!  Fantastic job everyone!

Major changes:

• For now, students who work alone do not have their repositories named after them
• “Allow Web Submits?” in Assignment Properties page defaults to REPOSITORY_EXTERNAL_SUBMITS_ONLY setting now
• Annotation Category dropdowns no longer close prematurely on mouseover-ing a tooltip
• Added “Reset Mark” capability to grader view

Grab a fresh copy of 0.6.2 here, or patch up 0.6.1 to 0.6.2 by clicking these words.

Brian Xu

• There’s a question about refactoring the FlexibleCriteria treatment to avoid if “rubric”/else statements.  Karen suggests that, since it’s a relatively complicated question, it be deferred to the end of the meeting
• Uploaded some new screenshots to ReviewBoard to display FlexibleCriteria functionality
• For FlexibleCriteria, a valid mark x is a float/integer where 0<=x<=max
• Victoria should play with the new FlexibleCriteria interface when it’s added to ReviewBoard, to give her input

Bryan Shen

• Karen thanks Bryan for his help with the dropdown problem experienced at UWaterloo
• Has been writing tests for FlexibleCriteria – ran into a problem with lots of repeated code
• Found an elegant solution to the above problem after posting a question on StackOverflow
• Will work on mapping TAs to marking scheme elements after finishing FlexibleCriteria
• Mike notes that, realistically speaking, work on mapping TAs to marking scheme elements will probably not get too far before term ends – should focus on foundation work and documentation for next developers to work on it
• Karen will get the specs for mapping TAs to marking scheme elements to Bryan sometime this weekend

Farah Juma

• The student interface for the GradeEntryForms is in pretty good shape now
• Just needs to add ability to release/unrelease marks, CSV upload/download, and tests
• Mike and Karen want a sneak preview so they can try it out

Joseph Mate

• Has put up a review request to switch submissions table from AJAX to server-side paginate
• Did some review requests, and wrote a blog post
• Next steps include testing, cleaning up the code to take advantage of will_paginate, possibly moving GradeEntryForm to server-side paginate, and getting the detailed submissions table view working again
• Mike notes that the old functions for the detailed table should still exist, and that getting it going again should (hopefully) not be too hard

Robert Burke

• Is putting test conversion to Shoulda off to one-side – taking longer than expected (has to rewrite many tests from scratch), and wants to focus on Notables instead
• Mike notes that it’d be a good general rule of thumb to avoid writing any new tests with Fixtures, but that converting old tests from Fixtures is a task that can be put off for a little while

Karen Reid

• Checks to ensure that everybody got their midterm report up on UCOSP blog
• Thanks to everybody who helped out with UWaterloo bugs this week!
• Classes at UofT end on April 1st.  End on April 5th for UWaterloo, and April 16 for SFU.
• Karen expects 4 more weeks of work from each team member, but is happy to see it stretch into April if needs be (marks will be sent to schools by end of April)
• Soft end-date for work is April 1st.  Brian Xu, Bryan Shen, and Robert Burke might continue working.
• Byron (UWaterloo) is coming to UofT on April 9th to discuss MarkUs plans – all are invited to attend

Brian Xu’s Database Question

• Original proposals for FlexibleCriteria from Laval students
• Brian explains the current situation:  In its current implementation, a method “get_criteria” has been added to Assignment model to retrieve collection of criteria (either Flexible or Rubric)
• Mike thinks it’d be a good idea to try to merge Flexible and Rubric tables into a single Criteria table, and then use Single Table Inheritance to let Assignments / Grader View / View Marks views be ignorant of which marking scheme is being used
• We are unable to find a scenario which breaks this model
• The design plan is approved – but delayed until the summer.  Brian Xu / Bryan Shen will continue developing so as to only support Rubric and Flexible Criteria.  Refactoring will happen afterwards.

Victoria

Status

• Filed new UI/UX tickets — mainly for Properties view.

• Wrote up UCOSP midway update.

• Reconnected with Nelle regarding Sections tab — which has now been implemented as well!

• Went through some ReviewBoard requests and provided some UI/UX feedback.

Next Steps

• Rework Grade Formation tab prototypes according to usability feedback we got from Diane and Jen.

• Brainstorm new designs for Dashboard view.

Roadblocks

• None

Mike

Status

• Helped with support for Byron / UWaterloo RE: database problem

• Fixed / patched 610 for 0.6 (tooltip annotation category bug)

• Prepped merge for Reset Mark feature for new 0.6 patch

• Did some email/irc support for teammates

Next Steps

• After Reset Mark feature gets Ship It, will release new 0.6 patch

• Continue reviewing code, answering questions, etc

Roadblocks

• None

Farah

Status

• Found and fixed the problem that was causing some test failures related to grade entry items (see Review Request #403)

• Mostly finished the student UI for grade entry forms (I hit a bug this week which slowed me down a bit but it ended up being a really silly thing related to a database inconsistency, which I have now solved.)

Next Steps

• Implement the ability to release the marks

• Tests for the student UI and releasing the marks

• CSV upload/download for the grades table

Roadblocks

• None this week

Brian Xu

Status

• Work on finishing functions of Flexible scheme and updating the prototype

• Most of the css are updated to support both schemes

• The input of mark of flexible scheme is validated now. Only float number not less than 0 and not larger than max can be input. Any invalid input will be ‘undo’. The updated result will be showed above the criteria.

Next Steps

• Discuss database design with Mike in the meeting

Roadblocks

• None

Bryan Shen

Status

• I’m writing the tests for the flexible marking scheme. I thought I would finish it much sooner but I ran into some problems. Some duplicate code are spotted within the current test cases which is not so DRY (Don’t Repeat Yourself). I’ll set out to refactor some test cases

• Investigated the UI issue regarding the disappearing dropdown list and confirmed the cause

• Submitted a minor patch that appends a missing tag in the view

Next Steps

• Finish the test cases for flexible marking schemes.

• Write a blog post of what I learnt from refactoring the test cases

Roadblocks

• Jet lag

Robert

Status

• Did not accomplish anything this past week.

Next Steps

• Halting work on moving Student Tests to Shoulda.

• Work on the Notables instead.

Roadblocks

• Being in California for 5 of the past 7 days hampered progress further than expected.

• Shoulda is great… if you aren’t porting tests. When I get back to this task, I may just write the Student tests from scratch, with inspiration from the former tests.

Joseph

Status

• finished moving from AJAX to serverside

• it’s on review board now

• some reviews

• blog post about using a dropdownlist without a form and without a button

Next Steps

• testing

• cleaning up the code to take advantage of will_paginate

• migrate the Manage Grade Entry Form: from AJAX to serverside as well

if you want it

• detailed submissions view

Roadblocks

• none

You can redirect using an html select without having to place it in a form, or have the user press a button. The user only needs to change the value.

Check out the following code that you can copy and paste into an html file and take for a test drive:

<html>

<head>
<title>Drop Down List Redirect</title>
</head>

<body>

<select onchange="top.location.href = 'http://www.google.com/search?q='
+ this.options[ this.selectedIndex ].value" >
<option value="">None</option>
<option value="cute+dogs">Cute Dogs</option>
<option value="lasers+beams">Laser Beams</option>
<option value="kitty+cat">Kitty Cat</option>
</select>

</body>
</html>

Breaking down the javascript code into more understandable chunks gives:

// 'this' points to the select object after change the item in the drop down list.
var drop_down = this;

// drop_down.selectedIndex contains the position of the item that was selected from the drop down
var selected_index = drop_down.selectedIndex;

// drop_down.options contains all of html option elements inside the html select
// we need to go to .value to get the 'value="something"' written in the HTML
var selected_value = drop_down.options[ selected_index ].value;

// changing top.location.href redirects ( unless you only append #blah )
top.location.href = 'http://www.google.com/search?q=' + selected_value

I hope you guys find this helpful!

Cheers,
Joseph

irc log can be find here

Agenda:
• Status report
• Q&A

People:
reid
m_conley
bryanshen
brianxu
robertb_frantil
jmate
bwinton

Meeting Minutes:
• Brian Xu has been working on Flexible scheme and has produced the prototype enable the flexible scheme for the views of grader/admin/students. The next step is to provide warning info for switching scheme and disconnect old records after switching scheme. Brian also finished ticket #531.
• Bryan Shen has done some manual tests and gave Brian his feedbacks. Now Bryan is focusing on the functional tests. Some issues related to view tests were discussed. Bryan will work on assigning TAs to rubric.
• Joseph has some great posts on Markus Blog (Setting URL using Javascript). Some issues related to fixing the nested form problem were discussed. He will work on AJAXifying Submissions Table.
• Robert is going to finish off doing the Student Tests. He will focus on the notables and move to converting tests afterwards.
• Mike will do a demo of the flexible scheme prototype for Karen.

Notes
Please remember to write your midterm review for the UCOSP blog.

Brian

Status:

• Finished ticket #531

• Flexible scheme prototype is created

• All views including admin/grader/students are created/modified for both flexible and rubric schemes

• database migration is updated (float number mark can be input now)

Next Steps:

• warning info for switch between different marking schemes (other related UI issues)

• “disconnect” old records after switch to different scheme

• more css files will be modified

• Bryan is working on test with me, test cases will be done at the same time we finish the components.

Roadblocks:

• Traveling back to Canada

Robert

Status:

• Converting Assignment and Student unit tests to shoulda and Machinist

Next Steps:

Roadblocks:

• Midterms sucked up more time this week than I wanted.
• Converting tests to shoulda requires more work than simply converting them to machinist. The tests need to be in a sensible order.
• I’m in California for 4 nights this coming week interviewing with companies that rhyme with Placebook and Bluegle. [Then again... I'll be stuck on a plane and in airports, with nothing to do but  whatever I've got on my laptop, which does include MarkUs....]

Victoria

Status:

• Together with Nelle, we discussed how to make the Sections feature more usable.  Suggested that we create a new top-level view/tab called “Sections” where the user can create and manage course sections.  This has been approved by Karen in last night’s UI/UX meeting.
• Also suggested that we move the “Sections” column in Users > Students view to the left of the “Grade Credit” column.
• Collected usability/design issues currently found in the Assignments > Properties page.  Again, discussed these with Karen in last night’s meeting and they have been approved/reassured.

Next Steps:

• UCOSP midterm blog post.
• File tickets for usability issues in “Properties” page.
• Rework Grade Formation tab prototypes according to usability feedback we got from Diane and Jen.
• Brainstorm ideas for Dashboard view.

Roadblocks:

None.

Mike

Status:

• Reviewed a bunch of code
• Wrote tests for Severin’s allow-web-submits negation bugfix (see http://review.markusproject.org/r/400/)

Next Steps:

• Same as always – continue to review code as it comes in.  Answer questions and help where I can.

Roadblocks:

• None

Farah

Status:

• Committed fix for Ticket #599 – a bug that was affecting pagination for the submissions table (see http://review.markusproject.org/r/387)
• Worked on Ticket #598 – calculate_total_mark needed to be refactored (see http://review.markusproject.org/r/399/)
• Worked on the student UI for grade entry forms

Next Steps:

• Changing “out_of” for grade entry items to type float (see http://review.markusproject.org/r/391/) has introduced some test failures so I need to investigate and update the tests
• Finish the student UI and take a look at how marks are currently released/unreleased for assignments in order to implement something similar for grade entry forms
• Remaining smaller tickets related to CSS, AJAX (#600-#602)

Roadblocks:

None this week

Bryan

Status:

• Manually tested Brian’s implementation of flexible marking scheme and gave him some feedback
• Studying and updating the existing functional and unit tests for rubrics and flexible marking scheme

Next Steps:

• Test the views related to flexible marking scheme

Roadblocks:

• None this week

Joseph

Status:

• wrote a couple blog posts about maintaining state between AJAX calls inside the URL using hrefs and document.location.href
• came up with two possible ways to fix the nested form issue on the submissions page
• easier way, use javascript on the dropdownlist’s onchange to set document.location.href. This will redirect the page, similar to a form’s GET method. As a result, the form around the dropdownlists are no longer required. We will not longer have the nested form issue.
• harder way, use javascript on each checkbox’s oncheckchange inside the table to update a hidden field in the unrelease/release mark form. This way, the form no longer has to contain almost the whole page. As a result, the form will not longer be nested with the dropdownlist’s forms.
• working on the easier idea to verify that it works

Next Steps:

• at the meeting verify that the solution is not too big of a hack
• ask about ways to test this. Should it be whitebox tests using Selenium?
• finish off the implementing the easier one
• if it does not work, try out the other idea

Roadblocks:

• None this week

I’m in the process of converting some unit tests (for Students and Assignments) away form fixtures and onto Machinist. I decreed at the weekly meeting that I was going to be finished this by the end of the weekend, and from the time this is posted, I’ve failed. I started later than I should have, and clearly didn’t account for enough debugging time.

“But wait!” I hear you clamour. “You’re just moving tests from one framework to another, right? Surely such things could be done by robots and other automatons?” Indeed! In an ideal world this would be true. But in this world, we will encounter exceptional circumstances and end up using our brain. Here is my tale…

This is a telling observation from the fixture tests:

Half the time, I’m not sure what’s being tested.

Here is a typical example:

<br />
def test_create_group_for_working_alone_student_existing_group<br />
    assignment = assignments(:assignment_1)<br />
    student = users(:student1)<br />
    # The test code...<br />
end<br />

The rote way of converting this to machinist is like so:

<br />
def test_create_group_for_working_alone_student_existing_group<br />
    assignment = Assignment.make<br />
    student = Student.make</p>
<p>    # The test code...<br />
end<br />

Pretty simple right? Awesome!

However, all is not well in testerland. Watch when I run it:

<br />
Testing started at 10:43 PM ...</p>
<p>NoMethodError: undefined method `allow_web_submits' for nil:NilClass<br />
app/models/group.rb:49:in `repository_external_commits_only?'<br />
    app/models/grouping.rb:333:in `update_repository_permissions'<br />
    app/models/student.rb:118:in `create_group_for_working_alone_student'<br />
    test/unit/student_test.rb:142:in `test_create_group_for_working_alone_student_existing_group'<br />
    /Applications/RubyMine 2.0.1.app/rb/testing/patch/testunit/test/unit/ui/testrunnermediator.rb:36:in `run_suite'<br />
    /Applications/RubyMine 2.0.1.app/rb/testing/patch/testunit/test/unit/ui/teamcity/testrunner.rb:215:in `start_mediator'<br />
    /Applications/RubyMine 2.0.1.app/rb/testing/patch/testunit/test/unit/ui/teamcity/testrunner.rb:191:in `start'<br />
1 tests, 0 assertions, 0 failures, 1 errors<br />
Test suite finished: 0.860263 seconds</p>
<p>Process finished with exit code 1<br />

Now there’s a tasty bowl of error soup!
What the heck happened? We were using a student and an assignment, shouldn’t it just work?

I look into the fixtures for both :student1 and :assignment_1 to see what went wrong.

<br />
# in users.yml<br />
student1:<br />
  user_name:      student1<br />
  last_name:      Mason<br />
  first_name:     Hollis<br />
  grace_credits:     5<br />
  type:           Student<br />
  section_id:     <%= Fixtures.identify(:section1) %><br />
  created_at:     <%= Time.now.to_s(:db) %><br />
  updated_at:     <%= Time.now.to_s(:db) %><br />
  hidden: false</p>
<p># in assignments.yml<br />
assignment_1:<br />
   short_identifier: Captain Nemo<br />
   description: "Code!!"<br />
   message:<br />
   due_date: <%= 30.days.from_now.to_s(:db)%><br />
   group_min: 2<br />
   group_max: 4<br />
   student_form_groups: true<br />
   instructor_form_groups: false<br />
   group_name_autogenerated: true<br />
   group_name_displayed: true<br />
   created_at: <%= Time.now.to_s(:db)%><br />
   updated_at: <%= Time.now.to_s(:db)%><br />
   repository_folder: CaptainNemo<br />
   marking_scheme_type: "rubric"<br />
   allow_web_submits: false<br />

Both look pretty routine, so now I’m at a loss of what to do. The only thing I’ve got to go on is the error message. But why am I getting it in the first place? The only thing that looks related to the error is the allow_web_submits property, which is being set to false. I update the make line to the following, to ensure consistent results.

<br />
    assignment = Assignment.make(:allow_web_submits => false)<br />

That’s not the big error though, why in the world is this being checked on a nil object? Where’s that happening? It’s raising in groups.rb, but where the heck were those set up?

Oh. They are… They are just in the yml files for the groups, groupings, and memberships fixtures.

A little tinkering around and my knowledge of how Assignments, Students, Groups, Memberships, and Groupings are organized gives me the final working test, that emulates the relevant conditions in the fixtures.

<br />
  def test_create_group_for_working_alone_student_existing_group<br />
    group = Group.make<br />
    assignment = Assignment.make(:allow_web_submits => false)<br />
    grouping = Grouping.make(:group => group, :assignment => assignment)<br />
    membership = StudentMembership.make(:membership_status => 'inviter', :grouping => grouping)<br />
    student = membership.user<br />
    # Same old Test code....<br />
  end<br />

Not at all the same, is it? The fixtures were hiding all the delicious dependancy information from me. In the end, I had to improvise a bit. From all the information, and time spent in the yml files determining the purpose of this particular test wasn’t easy. You needed to know exactly how the fixtures were set up in relation to each other, which involved look into half a dozen files. This is not ideal.

The purpose of a good test should be clear to anyone with some information about the system tested. In the case above, the test was that a student with a grouping already doesn’t cause an error when put into a separate group by themselves.

Beyond the assertion and the function name, I didn’t know what’s being tested. I didn’t know why those particular fixtures, :student1 and :assignment_1, were picked from the other students and assignment fixtures. The only way to find out is to directly delve into the .yml files that define them. That didn’t even guarantee that I understood the purpose of the test.

Before I submit this, I’ll likely need to check to see if each method is being adequately tested. Even if the tests were comprehensive under fixtures, it’s not clear that the same conditions are being met now. Thats not a vote to stay with fixtures though. Fixtures are bad because they move the preconditions away from the test, making it unclear under what circumstances the desired behaviour is to occur. Such a framework becomes brittle when writing new tests. Would you want add new fixtures, or see if some combination of the previous ones emulate the conditions of your test? With Machinist, you can construct what you need. When some other dev comes along, they might not know exactly why it was set up that way, but they will know what the setup is, and how you want the parts to be tested to interact.

Severin just pointed out a much easier way to update the URL using javascript:

<script language="javascript" type="text/javascript">
window.location.href = "/main/some_controller/some_action#some_anchor";
</script>

Thanks Severin!
I have extended this code into an example you can copy and paste into an html file and play around with:

<html>
<head><title>URL Updating</title></head>
<body>

<input    type="button"
name="cool"
value="cool"
onclick="window.location.href = window.location.href + '#value=param';" />

</body>
</html>

With Markus, this leaves lots of places for us to place the code. For a link_to_remote, we can place set the window.location.href in the :before or :complete variables. Example:
markus/app/views/ajax_paginate/_initial_paginate_links_alpha.html.erb

<%= link_to_remote "<< " + t('pagination.first'), :url => {
:action => action,
:id => assignment.id,
:filter => filter,
:page => 1,
:per_page => per_page,
:sort_by => sort_by,
:alpha_category => alpha_pagination_options[0],
:update_alpha_pagination_options => "false"
},
:before => "ap_thinking_start('#{table_name}');",
:complete => "ap_thinking_stop(); window.location.href = window.location.href + '#value=param';" %>

Notice that I stuck it right after the :complete =&gt; “ap_thinking_stop(); Additionally, notice that we are no longer limited to a’s. We can not apply this to any html objects.

Cheers,
Joseph

Markus’s submission page uses AJAX to grab the next or previous page of the table of submissions. However, this breaks bookmarks and the back button. So the goal is to do something similar to gmail. They update everything after the anchor (#) in the URL. Here is what we can do with Markus with Ruby on Rails, to do the exact same thing.

My examples are only going to cover the implementation of the Next &gt; button. However, this can apply to any AJAX link.

The part of code that generates the Next link is in:
_initial_paginate_links.html.erb:

<%= link_to_remote "Next >", :url => {
:action => 's_table_paginate',
:id => assignment.id,
:filter => filter,
:page => current_page + 1,
:per_page => per_page,
:sort_by => sort_by
},
:before => "ap_thinking_start();",
:complete => "ap_thinking_stop();" %>

When the page loads it generates:

<a onclick="ap_thinking_start();; new
Ajax.Request('/main/submissions/s_table_paginate/1?filter=none&amp;amp;amp;page=3&amp;amp;amp;per_page=30&amp;amp;amp;sort_by=group_name',
{asynchronous:true, evalScripts:true,
onComplete:function(request){ap_thinking_stop();},
parameters:'authenticity_token=' +
encodeURIComponent('hjLTZo+xwhfYVOjA6E6Cbt3mm0SoaJw3t+nQG9UF/iA=')});
return false;"
href="#">Next ></a>

However, we want the URL to update, so we need to get rid of the ” return false; ” at the end of onclick. I accomplished that by using link_to and the remote_function helpers that Rails provides.

_initial_paginate_links.html.erb:

<%= link_to "Next >", "#value=param",
: onclick =>
remote_function(
:url => {
:action => 's_table_paginate',
:id => assignment.id,
:filter => filter,
:page => current_page + 1,
:per_page => per_page,
:sort_by => sort_by
},
:before => "ap_thinking_start();",
:complete => "ap_thinking_stop();"
)
%>

Which generates:

<a onclick="ap_thinking_start();; new
Ajax.Request('/main/submissions/s_table_paginate/1?filter=none&amp;amp;amp;page=3&amp;amp;amp;per_page=30&amp;amp;amp;sort_by=group_name',
{asynchronous:true, evalScripts:true,
onComplete:function(request){ap_thinking_stop();},
parameters:'authenticity_token=' +
encodeURIComponent('hjLTZo+xwhfYVOjA6E6Cbt3mm0SoaJw3t+nQG9UF/iA=')});"
href="#value=param">Next ></a>

Notice that the return false is no longer there! If you try to click the Next &gt; button, then it will place #value=param at the end of the url without reloading the page.

Now all that remains is figuring out how to do this with an html form. For example drop downlists, with an onchange method that updates the page using AJAX but still appends #value=param to the end of the URL.

Cheers,
Joseph