MarkUs Blog

MarkUs Developers Blog About Their Project

Archive for the ‘Uncategorized’ Category

Assigning Graders to Rubic Elements

with 4 comments

We are working on this feature, so here is my first attempt at describing the requirements.

In some of the large classes, we might have quite a few TAs grading an assignment. One way to divide them up is to assign several TAs to each part of the assignment. For example if we had 12 TAs grading an assignment we might put 4 of them on part 1, 4 on part 2, and 4 on part 3. The only mechanism we have in MarkUs to map graders to parts of an assignment is through rubric elements.  So the idea is to assign TAs specific rubric elements and then use some UI features to highlight the rubric elements when a TA opens a student submission.

Assigning rubric elements to a grader is separate from assigning groups to a grader.  A grader might mark one rubric element for all submissions, or might mark all rubric elements for a subset of the submissions, or might mark a subset of the rubric elements for a subset of the submissions.

I imagine a UI similar to the mapping of graders to groups or students.  In this case we would see a list of the rubric elements with checkboxes (?) and would use much the same mechanism to map the graders to the rubric elements.

If the instructor does not specify rubric elements for a grader,  then the default should be that the grader will mark all rubric elements.

After rubric elements have been assigned to a grader, when a grader view a submission, it should be clear which rubric elements they are supposed to mark.  I am imagining that these rubric elements would be expanded by default, and would also be coloured differently.  The graders should still be able to assign marks for other rubric elements, we just want the ones that they have been assigned to work on to stand out more.

As always, feedback and comments are welcome.  What am I missing?

Written by Karen Reid

March 9th, 2010 at 10:33 am

Posted in Uncategorized

Meeting Minutes

without comments

The IRC log can be found at: http://www.cs.toronto.edu/~mconley/irc/markus/markus20100205_pg1.html

Brian Xu fixed three bugs:
512 -  UI: Show the graders name in the graders drop down —- http://review.markusproject.org/r/370/
531 – Graders need a way of “un-setting” Criteria —- http://review.markusproject.org/r/369/
571 – Edit Student: “Cancel” button links to something wrong —- http://review.markusproject.org/r/371/

Bryan Shen closed: #393. multi-level folder not allowed. Bryan also plans to blog about some things he has learned from working on Markus.

Brian Xu and Bryan Shen plan to get together and discuss how they are going to complete the flexible grading scheme.

Farah plans to return to working on the grade entry. She will update the review board so we can make comments on the code. The functional tests are still in the works.

Joseph has completed 564 – Validate files with a path with a space and check configuration on startup —- http://review.markusproject.org/r/358/. He hopes to get some time next week to look at will_paginate.

jerboaa figure out that urgent issue that arrived on February 4th.

Mike, feel free to assign this bug to someone else.  Everyone else, feel free to jump on it. :-)

The Download Detailed Criteria produces a stack trace when grace days are allowed.  It works when there are no grace days.

undefined method `find_by_rubric_criterion_id’ for #<Class:0xb6e475f8>
RAILS_ROOT: /data/markus/instance/csc209-2010-01

Application Trace | Framework Trace | Full Trace
/u/drprj/.gem/ruby/1.8/gems/activerecord-2.3.4/lib/active_record/base.rb:1842:in `method_missing_without_paginate’
/u/drprj/.gem/ruby/1.8/gems/will_paginate-2.3.11/lib/will_paginate/finder.rb:170:in `method_missing’
/u/drprj/.gem/ruby/1.8/gems/activerecord-2.3.4/lib/active_record/associations/association_collection.rb:380:in `send’
/u/drprj/.gem/ruby/1.8/gems/activerecord-2.3.4/lib/active_record/associations/association_collection.rb:380:in `method_missing_without_paginate’
/u/drprj/.gem/ruby/1.8/gems/activerecord-2.3.4/lib/active_record/base.rb:2143:in `with_scope’
/u/drprj/.gem/ruby/1.8/gems/activerecord-2.3.4/lib/active_record/associations/association_proxy.rb:206:in `send’
/u/drprj/.gem/ruby/1.8/gems/activerecord-2.3.4/lib/active_record/associations/association_proxy.rb:206:in `with_scope’
/u/drprj/.gem/ruby/1.8/gems/activerecord-2.3.4/lib/active_record/associations/association_collection.rb:376:in `method_missing_without_paginate’
/u/drprj/.gem/ruby/1.8/gems/will_paginate-2.3.11/lib/will_paginate/finder.rb:170:in `method_missing’
/data/markus/instance/csc209-2010-01/app/models/assignment.rb:400:in `get_detailed_csv_report’
/data/markus/instance/csc209-2010-01/app/models/assignment.rb:399:in `get_detailed_csv_report’
/data/markus/instance/csc209-2010-01/app/models/assignment.rb:384:in `each’
/data/markus/instance/csc209-2010-01/app/models/assignment.rb:384:in `get_detailed_csv_report’
/data/markus/instance/csc209-2010-01/app/models/assignment.rb:383:in `get_detailed_csv_report’
/data/markus/instance/csc209-2010-01/app/controllers/submissions_controller.rb:431:in `download_detailed_csv_report’

Thanks,
Karen

Robert finished off the tests for the random Grader assignment. He plans to work on issue 569 – Investigate possible Grader Assignment problem by manually hammering on the grader assignments to see if they always update correctly onscreen.

He also plans to work on 559 – Allow Rubric Weightings to be 0 or negative. However, Karen would like to discuss how it should be done with Byron. Mike believes that Byron will be able to use the Flexible Criteria if we push hard enough.

Mike will be running next week’s meeting. Additionally, Karen would like to have a meeting on the reading week’s Friday Feb 19th.

UI: Show the graders name in the graders drop down

Written by jmate

February 5th, 2010 at 5:54 pm

Posted in Uncategorized

Tagged with ,

January 28th Status Update

with one comment

Mike

Status

  • Fixed the functional tests!  Yeah!
  • Put the functional tests on markusproject.org homepage – though they act funny in an “off and on” sort of way…
  • Reviewed some code
  • Closed #530 (next/prev submission links in grader view no longer skip completed assignments in 0.6 – needs to be pushed to trunk)
  • Wrote a blog post about closing #531:  http://blog.markusproject.org/?p=1103


Next Steps

  • Review more code as it comes in
  • Turn the rest of Byron’s email into tickets (can’t believe I haven’t finished that yet)
  • Take a look at some other high-priority fixes maybe to patch 0.6
  • Brace myself for possible MarkUs support…CSC148 and CSC209 assignments due tomorrow.


Roadblocks

  • This is a heavy week and a half for me – that’s why I did most of my work over the weekend.  Might not respond to email this weekend, and will be pretty bogged down until next Wednesday sometime.

Brian

Status

  • 1 post on Markus blog
  • Finished #570 units test cases, change committed


Next Steps

  • Fix some bugs on weekends .


Roadblocks

  • Job hunting, 1 graduate course that requires intensive reading of papers, 1 research project

Victoria

Status

  • Completed first set of wireframes for the Assign Graders View.
  • Got useful feedback from Karen and Farah during last night’s UI meeting.

Next Steps

  • Start prototyping for the Group tab.  That is, the view users use to create new groups and modify existing groups.
  • Summarize new usability issues and ideas from last night’s UI meeting and write a blog post about it.

Roadblocks

  • Learning how to use a new application for prototyping.  I’m slowly getting the hang of it! :)

Joseph

Status

  • handling validate files with a space and checking configuration on startup is on reviewboard
  • ticketed the paging bug where the page number and page size are not saved in a bookmark or when hitting back/forwards
  • did rudimentary research on will_paginate – have not found anything so far

Next Steps

  • finish up the codereview
  • complete research on will_paginate
  • conclude which method to approach the pagination problem after research is complete

Roadblocks

Bryan

Status

  • Working on #393, Reviewing code, learning Ruby and Shoulda.


Next Steps

  • Solve more tickets.


Roadblocks

  • Not enough knowledge of Ruby, testing, and Markus’ structure.

Nelle

Status

  • Work on implementing creating section and to adding students to them
  • Started to look on how to migrate smoothly to Machinist


Next steps

  • Implementing the group members from same section limitation
  • Improving the UI on the different parts implemented this week


Roadblocks

  • None

Robert

Status

  • Working on tests for my fix for 510, otherwise Halted.
  • Reviewed changes from team members as they come in.

Next Steps

  • Find time to work on MarkUs.

Roadblocks

  • Many consecutive interviews [Google, Microsoft, Amazon, Facebook], midterms, and assignments all at once.

Farah

Status

  • Fixed a bug related to alphabetical pagination which was causing the alphabetical categories to get re-calculated each time the user switched between pages instead of only when the number of items per page is changed (and learned about session variables)
  • Re-factored code to make the GradeEntryFormsController lighter by moving some things to the model instead
  • Finished the unit tests for the grades table
  • Dropped in on the UI meeting on Wednesday

Next Steps

  • Finish functional tests for the grades table
  • Student interface for grade entry forms

Roadblocks

  • None this week

Written by Robert B

January 28th, 2010 at 8:58 pm

Posted in Uncategorized

Removing Marks

with 6 comments

Right now, MarkUs lets graders choose a mark from 0 – 4 to assign to a particular criterion on the rubric.  However, MarkUs does not currently allow graders to remove those marks once they’ve been made.  They can change the level, but they can’t flat out remove it.  This problem is filed as ticket #531.

My approach is to simply add a link that allows graders to remove the mark.  Simple as that.  I’ve thrown together a quick and dirty UI mockup – see below:

Remove that mark!

Remove that mark!

It really might be that quick and easy.  In fact, this problem might be ripe for one of my new teammates to sharpen their teeth on. It’ll be a good experience working with RJS, the Rails AJAX helpers, and the Grader View (which is one of my favourite parts of the entire app).  So I’m going to assign this ticket to “nobody”, and see who bites (and who’s reading the blog!).  :D

Written by m_conley

January 25th, 2010 at 5:58 pm

Posted in Uncategorized

Sections: specifications & ideas

with 5 comments

One of the functionality we would like to implement this semester is section management.

Here is the content of this post

Details specifications

  • Group management : the groups should be formed of two or three students of the same section ;
    • instructors need to be able to add the students section through the web interface or via the CSV file upload.
    • the student list should have filter to sort students per section.
    • instructors should be able to switch from two modes, in the assignment property tab:
      • groups have to be formed with students from the same section.
      • groups DO NOT have to be formed with students from the same section.
    • when switching to this mode, the application should display a warning message if some of the students don’t have a section associated to them.
    • students should see whether the groups have to be formed of students in the same section or not.
    • when the mode is activated, inviters should not be able to invited students in another section than theirs.
    • if some students don’t have a section, inviters with no section will be able to invited only students without any sections.
  • Deadlines : Each section needs to be associated with one specific deadline ;
    • this functionality can be enabled only if groups are limited to same-section students.
    • when enable, instructors can specify one deadline per section in the assignment property view.

If deadlines are specified per sections, it is logical that submissions rules could be too. For the first implementation, we will disable submission rules.

Database schema

section_db_schema

Sections table

This table is used to save all the sections. In the case of UofT, it should be from 2 to 3 sections. For Centrale Nantes (a French engineer school), 12 sections (groups).

id: int
name: string

Users table

We added a foreign key to sections. This is mostly used for students, but can also be used for Graders (we could then map graders to groups per sections).

Assignments table

There are a lot of options that can be added with the new section scheme.

section_due_dates: boolean, false
section_groups: boolean, false
section_submission_rules: boolean, false

Section_due_dates table

This links assignments and section, to add a specific due date to each section for an assignment.

id: int
section_id: int, fk
assignment_id: int, fk
due_date: date

If you have any thought, please chime in !

Written by nvaroqua

January 25th, 2010 at 10:08 am

MarkUs API Authentication Scheme

with 4 comments

The last couple of days I’ve been thinking of a way to do user authentication for MarkUs’ Web API. This is what I’ve come up so far. Feedback and suggestions are very welcome.

  1. Instructors and TAs will be able to get a “key” via MarkUs. When an instructor logs in the API key (a SHA2 512 bits in length) is displayed to him on the dashboard. TAs will see the key below the list of the assignments when they log in.
  2. The goal is that the private key never shows up in a script.
  3. In order to use this key for authentication from within a script the following steps are required:
    1. A MD5 hash of the private key has to be generated.
    2. This MD5 hash will be then encoded using Base 64 and the result will be used as a token and travels over the wire for each request.
    3. The token, generated as described above, will be sent to the MarkUs server by facilitating the HTTP header “Authorization”. I.e. each request sent to the MarkUs API has to include this header with the appropriate token.
    4. Since MarkUs knows about the private key and that the token is a Base 64 encoded MD5 digest of the key a matching user can be easily determined and the private key never has to be included in scripts on semi-private servers, where the scripts are run.

I was thinking that the Authorization HTTP header (which is otherwise used for Basic or Digest schemes) could have the following form:

Authorization: MarkusCustom MDk4ZjZiY2Q0NjIxZDM3M2NhZGU0ZTgzMjYyN2I0ZjYK

So far my thoughts. What do you think? Concerns? Am I on the wrong track? Thanks!

Written by Severin

December 30th, 2009 at 6:41 am

Posted in Uncategorized

Flexible Marking Scheme: What remains to do and screencasts….

without comments

You will recognize this report. It is the complete marking scheme analysis in which I have added one Chapter. Chapter 3 details the development that has been done and the development left to do in order to deliver  the complete flexible marking scheme feature.

By the way, eventhough it is published on our DrProject since December first, we forgot to let you know that our screencasts are done. Please follow this links to see theme:

- Select a marking scheme

- Add flexible criterion

Written by melagaud

December 7th, 2009 at 11:55 pm

Posted in Uncategorized

Simple Grade Entry: Table view performance issues

with 4 comments

As I’ve been working on the table view for grade entry last week and this week (there are still some kinks to sort out, more on that in my next post), I’ve been noticing some performance issues that I think we might want to deal with before we let instructors use this table. One thing that I needed to decide was whether or not each table cell should be saved right as the instructor enters the mark or if the instructor should enter a bunch of marks and then hit a “Save” button. I decided to go with the “Save” button option. My initial plan was to display the entire table on a single page. However, I’m finding that:

  1. It’s difficult to navigate
  2. This page can take a long time to load or save

Severin had suggested earlier that we could implement some sort of pagination. I really like this idea. Here’s what I’m thinking:

  • Maybe we could implement pagination alphabetically, by last name (eg. At the bottom of the table, we could have links like “A-D”, “E-J”, etc.). This would help to reduce the number of rows that get displayed on the page at a time. We probably wouldn’t want to implement pagination using numbers because if an instructor wanted to enter grades for a particular student, he/she would have a hard time figuring out which page contains the desired student.

Any thoughts about this?

Written by Farah Juma

December 3rd, 2009 at 12:39 am

Flexible marking scheme: PowerPoint presentation

with 2 comments

According to our QA cours requirement, we had to present a demo of the work done during  the session. Our demo was split in two parts.

1- PowerPoint presentation. This is the occasion to practice your french ;-)

2- Demo online of the application and of some code

The whole presentation went very well, and last around 30 minutes.

Written by melagaud

December 1st, 2009 at 12:41 pm

Posted in Uncategorized

Selenium is now integrated in MarkUs

without comments

Selenium, a nice framework used to test web interfaces, has been integrated into MarkUs. You can see how to run the tests in this page on the Wiki => How to run Selenium tests

The following links contain information about Selenium.

Other useful links :

Written by simonlg

November 25th, 2009 at 6:07 am

Posted in Uncategorized