MarkUs Blog

MarkUs Developers Blog About Their Project

Archive for March, 2010

Meeting Minutes – March 12th, 2010

with one comment

The meeting started at 3:36 and ended at 4:20.

The IRC log can be found here

Round table

Farah

  • Has submitted a review request for the student UI for grade entry forms
  • Has started implementing the ability to release the marks
  • Asked about the level of granularity we want for the “Marking State” column for grade entry forms. Karen said we should just indicate whether or not the student’s mark has been released (as opposed to also indicating unmarked, partially marked, and completed rows).

Joseph

  • Has done a lot of reviews and implemented Mike’s suggestions from his first review
  • Is currently working on implementing the changes Mike suggested in his second review
  • Joseph is going to meet with Mike next week to discuss how to test his changes

Bryan

  • Has been writing a tutorial about how to use Machinist within MarkUs
  • Has been working on switching from fixtures to Machinist in ResultsControllerTest and has submitted a review request
  • Has run into speed-related issues with Machinist
  • Has started thinking about assigning TAs to rubric elements
  • Karen said we need to be able to assign TAs to groups as well as rubric elements. However, we can treat these as essentially independent operations.

Brian

  • Has submitted a review request for the Alpha 1.0 version of the flexible criteria marking scheme
  • Has also been working on switching to the generic Criteria model with single-table inheritance

Robert

  • Is done with interviews
  • Has been reading the review requests and is ready to get back to working on MarkUs

Nelle

  • Has been implementing the ability to have one due date per section and it is going smoothly so far
  • They’ve also been trying to install MarkUs on a server in France (without root access)
  • Benjamin has a bunch of patches ready to be uploaded to Review Board

Other issues

  • Karen wants everyone to be thinking even harder than usual about documentation since there are only 3 weeks left now
  • Everyone should think about these things:
    • Is there anything that you have been working on that really deserves a blog post?
    • Are there tickets that need to be closed/commented on/written?
    • What will make it easy for the next crew to pick up where you left off? Think back to how you got started in January, and what you wish you had seen.
  • We’ll be doing the usual post-mortem at the end of the semester so we should start thinking about that as well

Written by Farah Juma

March 12th, 2010 at 9:38 pm

Posted in Meetings,Minutes

March 12th Punchlines

without comments

Victoria

Status

  • Met up with Karen in person on Monday to discuss feedback we got back from usability session held 2 weeks ago.
  • We also discussed the basis for the Dashboard view — what should be included, what to look into..etc.

Next Steps

  • Come up with a list of design and function requirements for Dashboard view.
  • Draft possible designs/layouts that could be used for the Dashboard view.
  • Possibly go through more UI/UX Review Board requests as needed.

Roadblocks

  • Things are a little crazy at work.  Hopefully everything will start settling down in a few weeks.

Mike

Status

  • Finally merged changes from 0.6 into trunk
  • Reviewed a ton of code
  • Some support via email

Next Steps

  • Continue what I’m doing – I’m bracing myself for the end of term crunch

Roadblocks

  • None

Farah

Status

  • Made averages for grade entry forms also appear on a student’s main page
  • Submitted a review request for the student UI (see Review Request #430 )
  • Started to work on releasing the marks

Next Steps

  • Finish implementing the ability to release the marks
  • Work on unit tests and functional tests for the student interface and releasing the marks
  • Implement CSV upload/download for the grades table

Roadblocks

  • Ran into issues in a couple other courses. Will spend extra time on MarkUs this weekend.

Brian

Status

  • Update Flexible scheme alpha 1.0 with all desired functions

Next Steps

  • Changing database scheme for criteria

Roadblocks

  • None

Joseph

Status

  • Completed a bunch of reviews
  • went over the recommendations from the review: http://review.markusproject.org/r/407/ – Switching to Serverside Pagination

Next Steps

  • testing

Roadblocks

  • figuring out the test and creating the proper groupings

Bryan

Status

  • Opened review request for ResultControllerTest. Considering how to reduce the performance overhead of Machinist.
  • Wrote a tutorial of using Machinist in Markus.
  • Thinking of some problems related with Assigning TAs to Rubrics

Next Steps

  • Discuss with Karen about the use cases of Assigning TAs to Rubrics
  • Finish the ResultControllerTest

Roadblocks

  • None

Robert

Status

  • Have offers from Google, Amazon, and Microsoft to consider!
  • Kept up with reviews, but didn’t do very many.

Next Steps

  • Do/Finish notables, as it has been all term.

Roadblocks

  • Interviews in Seattle. -> From here on in, I have guaranteed certified Free Time(tm).

Written by Robert B

March 12th, 2010 at 4:58 pm

Posted in Uncategorized

How would Markus be used at Ecole Centrale de Nantes

with 3 comments

For our project at Ecole Centrale de Nantes, Christian, Benjamin and I had to meet several professors from the different CS courses that are taught our school during our two first years in order to understand how would Markus be used at Ecole Centrale de Nantes, and list the desired new functionality.

We met Guillaume Moreau, in charge of supervising Markus’s deployment at Centrale Nantes, and head professor of the Object course of third year (during which he would like to test Markus next year), Didier Lime, current head Professor of the first year CS course on algorithm and C programming, and last but not least, Vincent Tourre, head prof of this same course next year.

A few concerns and ideas where raised. Here are the three main ideas.

  • Metrics: Guillaume Moreau likes to run metrics on students code for project: number of classes, name of this classes, number of methods, number of variables etc. These metrics are very useful to detect cheating: indeed, in more than 80% of the case, two groups who have exactly the same number of classes and methods just copied/pasted their code. Another type of metrics would be interesting, for third year students at Ecole Centrale de Nantes: metrics on the code itself, that analyses the code, and the use of some key word, and give automatic feedbacks on the code quality (http://pmd.sourceforge.net/).
  • PDF export of the results: Didier Lime would like the students to be able to retrieve the results and comment on their code. A simple way to do this is to implement a pdf export of the results page, with the annotation in a PDF. Students could therefore keep all traces of their assignments on their computer.
  • Possibility to upload reports & screenshots: Vincent Tourre was concerned about reports. Our first year course is about algorithm, and not code. Students have to write algorithm before starting to code anything, and write reports, with their algorithm. In order to use Markus to grade, we could ask students to only write plain text report. But, in second year, we also have to make the object schema. If students could upload reports and images, it would solve both of this problem (a bit like in review board).

Written by nvaroqua

March 10th, 2010 at 4:28 am

Posted in Uncategorized

Getting Started with Machinist in Markus

with 3 comments

When running, we almost always have to set up some objects to test. We used to set up these objects by loading them from fixtures. There is a problem with fixtures. It’s hard to know the attributes of the objects or the relationships between different objects unless we delve into fixture files. This is not good, because when doing tests and making assertions, we want to know well about the objects we tested against, but looking into fixtures is a time-consuming job. Now don’t you want a mechanism that allows you to explicitly create objects and build up the relationships among them, but without having to specify their attributes that you don’t care about? This is what Machinist does! With Machinist, you can create and save an assignment object by writing one short line in your code: “assignment = Assignment.make”. Isn’t that great?

In his post, Fixtures are not as clear as Machinist, Robert has a deep insight on how Machinist is better than fixtures.

Yes, Machinist is great, and I promise you’ll find it even greater if you spend a little time reading through Machinist’s documentation. Make sure you understand what a blueprint is and how we can pass attributes to “make” method and override its default behaviour.

The documentation pretty much tells all you need to know about Machinist. I’m not going through every details here, but there are some stuffs you may want to know about how we can use Machinist with Markus.

1. Setting up the playground for Machinist

Before using Machinist, please make sure you’ve installed two gems: Machinist and Faker. If they are not on your machine, you can run the following command in a terminal:

 gem install faker machinist --source http://gemcutter.org 

Now that the gems are installed, you want to build some confidence in Machinist by making some easy objects. The easiest way to play around with Machinist is to open a script/console with test environment. You can do this by opening a terminal, change the directory to the root directory of your Markus’ project, and then run the following command:

 script/console test 

Make sure you append the “test” option, which loads test environment for you. Otherwise you’ll be at the risk of screwing up data you use for dev.

If you see something like



Loading test environment (Rails 2.3.5)
>>

you’re already in the script console. In script console, you can execute any Ruby code as you would normally write in .rb source files. If you’ve not used script console before, try to “new” an empty group object to get yourself familiar with it.



>> group = Group.new
=> #<Group id: nil, group_name: nil, repo_name: nil>


“group = Group.new” creates a new empty group, whose information is displayed in the console.

Now you probably cannot wait to use Machinist. OK, let’s start by making a group which is the easiest model I can find in Markus. Try to execute the following line in the script console.

>> group = Group.make

But you’re likely to get this:



NoMethodError: undefined method `make' for Group(id: integer, group_name: string, repo_name: string):Class
 from /var/lib/gems/1.8/gems/activerecord-2.3.5/lib/active_record/base.rb:1959:in `method_missing_without_paginate'
 from /var/lib/gems/1.8/gems/will_paginate-2.3.12/lib/will_paginate/finder.rb:170:in `method_missing'
 from (irb):5
 >>


Why? If you’ve read the Machinist’s documentation, you should’ve already known the reason. You cannot use the “make” method of a model unless you’ve defined the its blueprint. OK, then let’s write the blueprint. Wait, in fact, you don’t need to, because blueprints for most models in Markus are already defined in /test/blueprints/blueprints.rb, which is “required” by /test/blueprints/helper.rb. To make a reference of these blueprints, you only need to run the following line in script console:

>> require "test/blueprints/helper.rb" 

Now try to “make” your group object again, and see what happens. If you still get an error, please don’t panic, and skip ahead to read the next section, where you’ll find the answer. But most likely you will be lucky to see a sign of success in your script console like the one I get here:

=> #<Group id: 930503258, group_name: "group1", repo_name: "group_930503258"> 

See, with only one simple call of “make”, you have already created a group object with all the attributes set properly. That’s why we all like Machinist!

Congratulations. You’ve already set up the playground for Machinist. At this point, you are good to go. Just try out any of your ideas with Machinist.

2. Common Issues

Sometimes, when you make a make an object, you get an error message like this: “ActiveRecord::RecordInvalid: Validation failed: Group name has already been taken”. This message means in the database there is already a group object with the same group name as the one you are just making. The test database can persists objects as a result of your last test, so you should clear the database before playing around with Machinist. To delete all the objects of a model in database, you can call “ModelName.delete_all”, where ModelName is the name of the Model whose objects you want to delete. But it’s is more desirable to clear all the table at the same time. Here is the code to do that:



(ActiveRecord::Base.send :subclasses).each {|c| c.delete_all}


I recommended you to use script console. But it has an issue. Everytime you modify and save blueprints.rb, you need to restart the script console for it to detect the change. This makes it inconvenient for you to do test your blueprints. As an alternative to script console, you can create a .rb file at your project’s root directory, and put the following lines at the beginning:



ENV["RAILS_ENV"] = "test"
require File.expand_path(File.dirname(__FILE__) + "/config/environment")
require File.join(File.dirname(__FILE__), "test/blueprints/helper")


This should also set you up a test environment where you can write code to play around with Machinist. The benefit of this approach is that it makes it easy to test blueprints.

3. Making Objects with Associations

There are three common types of associations: “belongs_to”, “has_many” and “has_one”.

belongs_to“: if A “belongs_to” B, a.b should be not nil and unique. It’s identical to a foreign key relationship from A to B. B should be made prior A. A’s blueprint should take care of making B. For example in Markus, Student belongs Section. The blueprints for Section and Student are:



Section.blueprint do

name {Sham.section_name}
end

Student.blueprint do
type {'Student'}
user_name {Sham.student_user_name}
first_name
last_name
section
end


The blueprint of Student implicitly calls Section.make. Therefore, when you call Student.make, a section will be made before student. If you already have a section object my_section and want to make a student that belongs to that section, you can call Student.make({:section => my_section}), which overrides the default behaviour specified by Student blueprint and doesn’t make a new section. When you make an object, you may end up making lots of  other objects because the “belongs_to” association “recursively” makes the dependencies.

has_many“:  if A “has_many” B, then B “belongs_to” A. Machinist encourages you to first create the object on the right hand side of “belongs_to” association (seeking for some better wording here). Using the example of Student and Section. Section belongs to Section, and thus Section has many Students. When you want to make a section with many students, you need to make a section first. We are still using the blueprints from the previous example:



section = Section.make

section.students.make


When section.students.make is called, Machinist is smart enough to call  Student.make({:section => section}). So you don’t need to worry about accidentally creating a new section. After the two lines, you have section.students.first.section == section.

has_one“: “has_one” is a special case of “has_many”. if A “has_one” B, then B “belongs_to” A. Same as “has_many”, A should be made prior to B. Suppose we have two models: Coach and Team, where Team “has_one” Coach and Coach “belongs_to” Team. You may attempt to make an object of Team in a similar way to “has_many” association:



team = Team.make

coach = team.coach.make


But it doesn’t work. In the “has_many” example, section.students is a collection which is an object not nil, but here team.coach is nil. There is no way for Machinist to know which blueprint to use for making a nil object. So what you have to do is:



team = Team.make

coach = Coach.make({:team => team})


Then you can use the team object.

Submission.blueprint do
grouping
submission_version {1}
submission_version_used {true}
revision_number {1}
revision_timestamp {1.days.ago}
end

Written by bryanshen

March 10th, 2010 at 3:51 am

Assigning Graders to Rubic Elements

with 4 comments

We are working on this feature, so here is my first attempt at describing the requirements.

In some of the large classes, we might have quite a few TAs grading an assignment. One way to divide them up is to assign several TAs to each part of the assignment. For example if we had 12 TAs grading an assignment we might put 4 of them on part 1, 4 on part 2, and 4 on part 3. The only mechanism we have in MarkUs to map graders to parts of an assignment is through rubric elements.  So the idea is to assign TAs specific rubric elements and then use some UI features to highlight the rubric elements when a TA opens a student submission.

Assigning rubric elements to a grader is separate from assigning groups to a grader.  A grader might mark one rubric element for all submissions, or might mark all rubric elements for a subset of the submissions, or might mark a subset of the rubric elements for a subset of the submissions.

I imagine a UI similar to the mapping of graders to groups or students.  In this case we would see a list of the rubric elements with checkboxes (?) and would use much the same mechanism to map the graders to the rubric elements.

If the instructor does not specify rubric elements for a grader,  then the default should be that the grader will mark all rubric elements.

After rubric elements have been assigned to a grader, when a grader view a submission, it should be clear which rubric elements they are supposed to mark.  I am imagining that these rubric elements would be expanded by default, and would also be coloured differently.  The graders should still be able to assign marks for other rubric elements, we just want the ones that they have been assigned to work on to stand out more.

As always, feedback and comments are welcome.  What am I missing?

Written by Karen Reid

March 9th, 2010 at 10:33 am

Posted in Uncategorized

Introducing MarkUs 0.6.2!

with one comment

The MarkUs team is pleased to present version 0.6.2!  Fantastic job everyone!

Major changes:

  • For now, students who work alone do not have their repositories named after them
  • “Allow Web Submits?” in Assignment Properties page defaults to REPOSITORY_EXTERNAL_SUBMITS_ONLY setting now
  • Annotation Category dropdowns no longer close prematurely on mouseover-ing a tooltip
  • Added “Reset Mark” capability to grader view

Grab a fresh copy of 0.6.2 here, or patch up 0.6.1 to 0.6.2 by clicking these words.

Written by m_conley

March 7th, 2010 at 7:28 pm

Posted in Releases

Tagged with ,

Meeting Minutes: March 5, 2010

with one comment

Brian Xu

  • There’s a question about refactoring the FlexibleCriteria treatment to avoid if “rubric”/else statements.  Karen suggests that, since it’s a relatively complicated question, it be deferred to the end of the meeting
  • Uploaded some new screenshots to ReviewBoard to display FlexibleCriteria functionality
  • For FlexibleCriteria, a valid mark x is a float/integer where 0<=x<=max
  • Victoria should play with the new FlexibleCriteria interface when it’s added to ReviewBoard, to give her input

Bryan Shen

  • Karen thanks Bryan for his help with the dropdown problem experienced at UWaterloo
  • Has been writing tests for FlexibleCriteria – ran into a problem with lots of repeated code
  • Found an elegant solution to the above problem after posting a question on StackOverflow
  • Will work on mapping TAs to marking scheme elements after finishing FlexibleCriteria
  • Mike notes that, realistically speaking, work on mapping TAs to marking scheme elements will probably not get too far before term ends – should focus on foundation work and documentation for next developers to work on it
  • Karen will get the specs for mapping TAs to marking scheme elements to Bryan sometime this weekend

Farah Juma

  • The student interface for the GradeEntryForms is in pretty good shape now
  • Just needs to add ability to release/unrelease marks, CSV upload/download, and tests
  • Mike and Karen want a sneak preview so they can try it out

Joseph Mate

  • Has put up a review request to switch submissions table from AJAX to server-side paginate
  • Did some review requests, and wrote a blog post
  • Next steps include testing, cleaning up the code to take advantage of will_paginate, possibly moving GradeEntryForm to server-side paginate, and getting the detailed submissions table view working again
  • Mike notes that the old functions for the detailed table should still exist, and that getting it going again should (hopefully) not be too hard

Robert Burke

  • Is putting test conversion to Shoulda off to one-side – taking longer than expected (has to rewrite many tests from scratch), and wants to focus on Notables instead
  • Mike notes that it’d be a good general rule of thumb to avoid writing any new tests with Fixtures, but that converting old tests from Fixtures is a task that can be put off for a little while

Karen Reid

  • Checks to ensure that everybody got their midterm report up on UCOSP blog
  • Thanks to everybody who helped out with UWaterloo bugs this week!
  • Classes at UofT end on April 1st.  End on April 5th for UWaterloo, and April 16 for SFU.
  • Karen expects 4 more weeks of work from each team member, but is happy to see it stretch into April if needs be (marks will be sent to schools by end of April)
  • Soft end-date for work is April 1st.  Brian Xu, Bryan Shen, and Robert Burke might continue working.
  • Byron (UWaterloo) is coming to UofT on April 9th to discuss MarkUs plans – all are invited to attend

Brian Xu’s Database Question

  • Original proposals for FlexibleCriteria from Laval students
  • Brian explains the current situation:  In its current implementation, a method “get_criteria” has been added to Assignment model to retrieve collection of criteria (either Flexible or Rubric)
  • Mike thinks it’d be a good idea to try to merge Flexible and Rubric tables into a single Criteria table, and then use Single Table Inheritance to let Assignments / Grader View / View Marks views be ignorant of which marking scheme is being used
  • We are unable to find a scenario which breaks this model
  • The design plan is approved – but delayed until the summer.  Brian Xu / Bryan Shen will continue developing so as to only support Rubric and Flexible Criteria.  Refactoring will happen afterwards.

Written by m_conley

March 6th, 2010 at 1:03 pm

MarkUs March 5th Punchlines

without comments

Victoria

Status

  • Filed new UI/UX tickets — mainly for Properties view.
  • Wrote up UCOSP midway update.
  • Reconnected with Nelle regarding Sections tab — which has now been implemented as well!
  • Went through some ReviewBoard requests and provided some UI/UX feedback.

Next Steps

  • Rework Grade Formation tab prototypes according to usability feedback we got from Diane and Jen.
  • Brainstorm new designs for Dashboard view.

Roadblocks

  • None

Mike

Status

  • Helped with support for Byron / UWaterloo RE: database problem
  • Fixed / patched 610 for 0.6 (tooltip annotation category bug)
  • Prepped merge for Reset Mark feature for new 0.6 patch
  • Did some email/irc support for teammates

Next Steps

  • After Reset Mark feature gets Ship It, will release new 0.6 patch
  • Continue reviewing code, answering questions, etc

Roadblocks

  • None

Farah

Status

  • Found and fixed the problem that was causing some test failures related to grade entry items (see Review Request #403)
  • Mostly finished the student UI for grade entry forms (I hit a bug this week which slowed me down a bit but it ended up being a really silly thing related to a database inconsistency, which I have now solved.)

Next Steps

  • Implement the ability to release the marks
  • Tests for the student UI and releasing the marks
  • CSV upload/download for the grades table

Roadblocks

  • None this week

Brian Xu

Status

  • Work on finishing functions of Flexible scheme and updating the prototype
  • Most of the css are updated to support both schemes
  • The input of mark of flexible scheme is validated now. Only float number not less than 0 and not larger than max can be input. Any invalid input will be ‘undo’. The updated result will be showed above the criteria.

Next Steps

  • Discuss database design with Mike in the meeting

Roadblocks

  • None

Bryan Shen

Status

  • I’m writing the tests for the flexible marking scheme. I thought I would finish it much sooner but I ran into some problems. Some duplicate code are spotted within the current test cases which is not so DRY (Don’t Repeat Yourself). I’ll set out to refactor some test cases
  • Investigated the UI issue regarding the disappearing dropdown list and confirmed the cause
  • Submitted a minor patch that appends a missing tag in the view

Next Steps

  • Finish the test cases for flexible marking schemes.
  • Write a blog post of what I learnt from refactoring the test cases

Roadblocks

  • Jet lag

Robert

Status

  • Did not accomplish anything this past week.

Next Steps

  • Halting work on moving Student Tests to Shoulda.
  • Work on the Notables instead.

Roadblocks

  • Being in California for 5 of the past 7 days hampered progress further than expected.
  • Shoulda is great… if you aren’t porting tests. When I get back to this task, I may just write the Student tests from scratch, with inspiration from the former tests.

Joseph

Status

  • finished moving from AJAX to serverside
  • it’s on review board now
  • some reviews
  • blog post about using a dropdownlist without a form and without a button

Next Steps

  • testing
  • cleaning up the code to take advantage of will_paginate
  • migrate the Manage Grade Entry Form: from AJAX to serverside as well

if you want it

  • detailed submissions view

Roadblocks

  • none

Written by Brian Xu

March 5th, 2010 at 3:50 pm

Posted in Status Reports

Javascript to Redirect a Drop Down List (select)

without comments

You can redirect using an html select without having to place it in a form, or have the user press a button. The user only needs to change the value.

Check out the following code that you can copy and paste into an html file and take for a test drive:


<html>

<head>
<title>Drop Down List Redirect</title>
</head>

<body>

<select onchange="top.location.href = 'http://www.google.com/search?q='
+ this.options[ this.selectedIndex ].value" >
<option value="">None</option>
<option value="cute+dogs">Cute Dogs</option>
<option value="lasers+beams">Laser Beams</option>
<option value="kitty+cat">Kitty Cat</option>
</select>

</body>
</html>

Breaking down the javascript code into more understandable chunks gives:


// 'this' points to the select object after change the item in the drop down list.
var drop_down = this;

// drop_down.selectedIndex contains the position of the item that was selected from the drop down
var selected_index = drop_down.selectedIndex;

// drop_down.options contains all of html option elements inside the html select
// we need to go to .value to get the 'value="something"' written in the HTML
var selected_value = drop_down.options[ selected_index ].value;

// changing top.location.href redirects ( unless you only append #blah )
top.location.href = 'http://www.google.com/search?q=' + selected_value

I hope you guys find this helpful!

Cheers,
Joseph

Written by jmate

March 4th, 2010 at 8:13 pm