MarkUs Blog

MarkUs Developers Blog About Their Project

Archive for October, 2008

Rubric Weight Management

without comments

A few weeks back, I took my paper prototype for the Rubric Management control, and did some UI testing.  All went well, until last week, where I found a small flaw with my test and design.

The problem was that I tested the wrong user base – I tested my paper prototype on other computer science students, when I really should have been testing computer science profs.

It all comes down to weights.  As a student, I assumed that a professor takes a rubric, and distributes the weight among the criteria however they see fit:  Style is worth 5%, Creativity is worth 10%, etc.  My fellow CS students seemed to have the same impression.

Unfortunately, this is not the case.  CS profs, after creating the criteria for the rubric (where there are usually at least 10 criteria), add weight multipliers to the criteria that they deem more important.  So, for example, Style could get a weight multiplier of 2, and Creativity a weight multiplier of 1 (where 1 is the default weight).

So it’s a difference of orientation of thought:  I thought that 100% of weight was distributed amongst the criteria.  Instead, the criteria is created, and the weights are applied as professors see fit.  After the weight is applied, the 100% has been ‘created’, as opposed to the other way around.

So this means that the slider widget that I had originally planned for adjusting weights for the criteria no longer makes sense.  Instead, I plan to have a simple text input for inputting weight multipliers.  I’ll also put increment and decrement buttons on each side of the text input for quick bumps.

Just thought I’d share that.


Written by m_conley

October 26th, 2008 at 12:28 am

Posted in Uncategorized

Rebooting the Schema (part 2)

with 2 comments

Last time, I talked about the old model schema, and the problems it had that lead us to refactor the code. After refactoring, this is what the database looks like:


The relationships are now more concrete with the addition of memberships and assignments_groups tables. The assignments_groups is a Rails convention of declaring a many-to-many relationship between two objects by use of the join table. Thus, an assignment can have many groups, and groups can also have many assignments if a group persists throughout the course. A caveat though is to make sure that the join table is in alphabetical order, meaning it must be assignments_groups and not groups_assignments. That’s just the “convention-over-configuration” mantra of Rails at work.

Once we have the database schema set, we can then just go in and declare those relationships in the ActiveRecord classes respectively:

class Group < ActiveRecord::Base
has_and belongs_to_many :assignments
class Assignment < ActiveRecord::Base
has_and belongs_to_many :groups

However it is a different case if the join table contains extra information, which is our case with the memberships table. Here, not only does it reference the user and the group together, but it also contains extra information such as status of the member. Thus, we need to have a Membership class representing a member, and use has-many-through relationship. which sort of explicitly states that the association between a User and a Group uses memberships as its link. Here we declare the relationship as follows:

class Membership < ActiveRecord::Base
belongs_to :user
belongs_to :group
class User < ActiveRecord::Base
has_many :memberships
has_many :groups, :through => :memberships
class Group < ActiveRecord::Base
has_many :memberships
has_many :members, :through => :memberships, :source => :user


We’ve also separated the old submissions table to a submissions and submission_files tables. The new submissions table doesn’t seem to have much information and seems to be a waste of space. However, having this table allows us to delegate submission functions to a Submissions class rather than mixing them directly with either the User or Group classes. All we have to do now is just ask a User or Group for its Submission instance, and handle all queries related to submitted files from it.

Since we also want to avoid checking to see if it is a User or a Group submission everytime, we’ve abstracted the Submissions class and added separate classes for each type, UserSubmission and GroupSubmission – classes that are linked to Users and Groups respectively. Since instead of declaring the relationship with Submissions, we have:

class User < ActiveRecord::Base
has_many :submissions, :classname => UserSubmission
class Group < ActiveRecord::Base
has_many :submissions, :classname => GroupSubmission

class UserSubmission < Submission
has_many :users
class Group
Submission < Submission
has_many :groups

This allows us to call either user.submissions or group.submissions and return with an instance of the appropriate Submission subclass type.

Final Results

The refactored models with the appropriate associations gave way to a much cleaner code in the end. With the schema set in place, I’ve revisited the old code and heeded the advice in the first post, stuffing all the business logic in the appropriate models and leaving workflow control to the controllers. The result turned several functions with 200+ lines into a single function with less than 50 lines. I was also able to create more thorough unit testing while code was being written. Here, we can see that we’ve improved our stats quite a bit:

$ for f in app/controllers app/models app/helpers; do echo $f
`find $f -name "*.rb" |xargs wc -l |tail -n1`; done
app/controllers 563
app/models 591
app/helpers 60

In retrospect, I think the refactoring decisions suits us very well with what we have in mind and gives us room for modifications at the same time…until we actually start porting OLM. Stay tuned.

Written by Geofrey

October 14th, 2008 at 6:32 pm

Rebooting the Schema (part 1)

with one comment

Starting off this project, we’ve decided to start fresh with our model schema. When I started with Rails, and my first time developing on an MVC framework, I’ve always wondered how to decide what goes in the model and what goes in the controllers. After our summer iteration, a Rails reviewer came in to see the status of the code. With a little bit of console magic, he showed us this:

$ for f in app/controllers app/models app/helpers; do echo $f
`find $f -name "*.rb" |xargs wc -l |tail -n1`; done
app/controllers 720
app/models 390
app/helpers 32

He suggested that we should try to put most of the controller functions in the models as much as possible. That way, we can do a more thorough and fine-grained unit testing without mixing data functions to the workflow that controllers are suppose to do. To answer my question, he gave this advice:

One way to force oneself to use models more:
controllers should only have the methods pertaining to RESTful resources.

That way, the controller’s job would only be tying up together the business logic through the models and use the models to share code between controllers. Andrew, one of the students working on the project, also suggested to make use of the ActiveRecord relationships rather than having customized SQL queries in the code.

Before refactoring, the DB schema looked like this:

Old Schema

There were a lot of redundancies with this schema. For example, the group table not only contains group information but also maintains a record for each member of a group. Thus, a group with four members would have four records in the group table containing the same group_number and assignment_id. Also, if the same group persisted throughout the assignments in the course, then we would have to duplicate those four records for each assignment.

The simplicity of the tables also made it hard to actually manage it on the models layer. With no join tables to work on, I was using functions to wrap hard-coded ActiveRecord queries.

In the next post, I’ll talk about what the new schema looks like and what improvements we’ve made in the code.

Written by Geofrey

October 10th, 2008 at 5:04 pm

Posted in OLM on Rails

Developing a Rubric for OLM on Rails

with 2 comments

Hey all – I’m Mike, and I’ve just started working on the OLM on Rails project.  My first task is to develop the Rubric models, controllers, and user experience, and I’m going to share some musings on it.

Ok, so first off, here’s the idea:  A professor creates an Assignment, and has to have a marking scheme to go along with it.  The Rubric that she develops will be used by the TA’s to mark the assignments.  Sounds pretty simple.

I’m still getting my thinking wrapped together here, so I’m just going to muse for a bit.  Here are some thoughts, rules, and caveats:

  • Creation of a Rubric is a separate event from creating an Assignment.  So if an Assignment exists, it does not imply that an associated Rubric exists yet
  • A Rubric consists of an arbitrary number of criteria to grade the student
  • Each criterion has a weight assigned to it (ie: 0.25, 0.5, etc)
  • Each criterion has a number of levels that can be achieved by a student.  By default, these levels are 0, 1, 2, 3, and 4 – with 0 being the worst performance, and 4 being the best.  We’re considering having the ability to add additional levels, to improve flexibility
  • An assignment cannot be marked until a Rubric has been created
  • A Rubric is not valid unless all of the weights of its criteria sum to 1.0 (100%)
  • The total number of marks achievable on an Assignment equals the sum of each criterion weight, multiplied by the maximum level of that criterion.  This does not include possible bonus marks, which are not handled by the Rubric
  • Assigning weights to criteria is going to be a little tricky, interface-wise.  While it’s tempting to just use a simple text-box, and trust the user to input the weights manually, it’d be nice for something a little more elegant and user friendly.
  • Some professors may want a simpler way of inputting this information quickly, for instance, by uploading a file describing what they want.  Andrew has suggested that users upload a YAML file… something worth thinking about.

That’s all I’ve really got rolling in my head about it right now.

So, what have I done?  I’ve created two new models:  RubricCriteria and RubricLevel.  A RubricCriteria is assigned to a single Assignment.  A RubricLevel is assigned to a single RubricCriteria.  Piece of cake.

I’ve also created some basic unit tests for these models.  I think I’m going to start thinking about user interfaces now, because I think it will give me some hints on what I’ll want the Controller to do.  I might upload some of my sketches later.



Written by m_conley

October 8th, 2008 at 12:43 pm

Posted in OLM on Rails

Tagged with , , , , , ,