MarkUs Blog

MarkUs Developers Blog About Their Project

Assigning Graders to Rubic Elements

with 4 comments

We are working on this feature, so here is my first attempt at describing the requirements.

In some of the large classes, we might have quite a few TAs grading an assignment. One way to divide them up is to assign several TAs to each part of the assignment. For example if we had 12 TAs grading an assignment we might put 4 of them on part 1, 4 on part 2, and 4 on part 3. The only mechanism we have in MarkUs to map graders to parts of an assignment is through rubric elements.  So the idea is to assign TAs specific rubric elements and then use some UI features to highlight the rubric elements when a TA opens a student submission.

Assigning rubric elements to a grader is separate from assigning groups to a grader.  A grader might mark one rubric element for all submissions, or might mark all rubric elements for a subset of the submissions, or might mark a subset of the rubric elements for a subset of the submissions.

I imagine a UI similar to the mapping of graders to groups or students.  In this case we would see a list of the rubric elements with checkboxes (?) and would use much the same mechanism to map the graders to the rubric elements.

If the instructor does not specify rubric elements for a grader,  then the default should be that the grader will mark all rubric elements.

After rubric elements have been assigned to a grader, when a grader view a submission, it should be clear which rubric elements they are supposed to mark.  I am imagining that these rubric elements would be expanded by default, and would also be coloured differently.  The graders should still be able to assign marks for other rubric elements, we just want the ones that they have been assigned to work on to stand out more.

As always, feedback and comments are welcome.  What am I missing?

Written by Karen Reid

March 9th, 2010 at 10:33 am

Posted in Uncategorized

4 Responses to 'Assigning Graders to Rubic Elements'

Subscribe to comments with RSS or TrackBack to 'Assigning Graders to Rubic Elements'.

  1. Hi, I doubt whether we should separate group-ta assignment from rubric-ta assignment. Using the scenario you mentioned, an admin wants 4 TA on part 1, 4 on part 2, and 4 on part 3, and each TA will be marking all the groups. The admin have to do this in two steps (separately):
    1. assigning every TA to every group;
    2. assigning each TA to a subset of rubrics.
    The first step is by itself not a meaningful operation, and it is currently not supported by Markus. It’s hard for the admin to ensure that each rubric of a submission is covered by at least one TA

    Now wouldn’t it be better if an admin can see a TA’s groups and rubrics at the same place, like in a table? This would help an admin to see which TA is in charge of which rubrics and which groups, which makes it easy for the admin to decide how to assign the TAs.


    11 Mar 10 at 12:24 am

  2. I think Victoria’s design for mapping graders to groups would also work well for mapping graders to rubric elements. For example, the graders could be in a scrollable box on the left-hand side and the rubric elements could be in a scrollable box on the right-hand side. An instructor could just check off which TAs should mark which rubric elements. Just like in Victoria’s AssignGrader prototype, rubric elements could also have the grader’s name listed next to them in the scrollable box. That way, it would be easy to spot which rubric elements have not been assigned a grader.

    Farah Juma

    11 Mar 10 at 9:32 pm

  3. Bryan makes some good points.

    We currently TAs to groups so I’d still like to keep that.

    However, the problem of figuring out if every rubric element of every submission is covered by a TA is one that I hadn’t thought of.

    The table seems like a good idea in some ways, but I’m imagining 300 groups and 20 rubric elements. It would be hard to scan the table to be sure that all the cells are filled.

    We do need some way to identify whether all the rubric elements for all the groups are covered. Maybe something as simple as an indicator on the page of how many are still left unassigned would be enough.


    11 Mar 10 at 11:22 pm

  4. […] admins are hoping for a feature that enables graders to mark a subset of criteria. According to Karen’s specifications and the discussions on IRC, I’ve written a draft version of User Stories: Assigning Graders […]

Leave a Reply