MarkUs Blog

MarkUs Developers Blog About Their Project

Archive for November, 2009

November 27th Status Update

with 2 comments

Fernando

Status

  • system almost done
  • polymorphic option to our model
  • Added some tests

Next Steps

  • Add the modal dialog to the TA view

Roadblocks

  • TIME

Mike

Status

  • Reviewed more code

Next Steps

  • Review even more code! We’re on the final stretch!

Roadblocks

  • Roadblocks are beginning to clear – things are starting to ease up, schedule-wise.

Farah

Status

  • Made some final changes to the grade entry form models based on feedback from my grade entry form creation review request and modified the unit tests accordingly. (see review 217)
  • Finished the functional tests and submitted a review request. I spent a lot more time on this than I had originally planned to but I wanted to make sure the tests were quite thorough. (see review 245)
  • Worked on the table view (60% complete)

Next steps

  • Finish the table view and test it. I had wanted to do the “attaching graders” view as well but I don’t think there’s enough time remaining (maybe “simple” grade entry isn’t the best name for this feature :))
  • Documentation for the Wiki
  • Screencast

Roadblocks

  • None this week

Severin

Status

  • Had a meeting with Karen regarding automated testing on Monday
  • Finished up some tickets
  • Created some tickets
  • Did another blog post about automated testing

Next Steps

  • Fix reviews for which I got feedback for
  • Continue writings/analysis regarding automated testing
  • Talk with Mike and Karen about must-fix things for the spring courses round.

Roadblocks

  • None (I’d like to have a 7-day-weekend, though)

Gabriel

Status

  • Fought with cucumber to make it behave around MarkUs… Not much success up to now
  • Published my post-mortem on the UCOSP blog

Next Steps

  • Close the cucumber chapter
  • Write Selenium tests for the feature I implemented (Flexible criteria edition)
  • Document the Flexible criteria features state (wiki)
  • Introduce MarkUs to our colleagues (and share our experience) during our next QA class

Roadblocks

  • Time (because of work and a certain course notoriously known –around here– as Design 3)

Melanie

Status

  • Worked on and finished Ticket #493
  • Worked on Ticket #457
  • Published my Thumb up Thumb down on the UCOSP blog
  • Published on Who is Who of the Markus project
  • Created 2 screencasts to explain the flexible criterion implemented features. Will publish theme soon
  • Created a PowerPoint for our QA course that presents Cross-Canada, Markus and our implication

Next Steps

  • Finish Ticket #457
  • Write some Selenium tests
  • Document the Flexible criteria features state with my teammates
  • Introduce MarkUs to our colleagues (and share our experience) during our next QA class

Roadblocks

  • None

Simon

Status

  • Wrote a guide how to run Selenium tests on the wiki and blog
  • Wrote Selenium tests for the Assignment properties’ view
  • Created a migration making Rubric the default value for the marking scheme type — Ticket #441

Next Steps

  • Create tickets for what is left to do on the Flexible criteria feature
  • Document the Flexible criteria features state (wiki)
  • Introduce MarkUs to our colleagues (and share our experience) during our next QA class

Roadblocks

  • Time, I am working on an intense project finishing on December 3rd

Tara

Status

  • Finally committed code for tickets #358 and #405 (adding multiple students to a grouping as an instructor)
  • Finished the index view for the Notes tab (for groupings)
  • Created “new” and “edit” actions and views for the Notes tab (for groupings)
  • Met with Fernando to finalize Notes system, including polymorphic assocations
  • Changed notes to use polymorphic associations for the “noteable” object

Next steps

  • Commit Notes tab code
  • Extend Notes system create form for assignments and students
  • Investigate ticket #436 – Grace periods used are not reported anywhere in the UI
  • Investigate ticket #361 – Grader View: shoudn’t unmarked rubric be expanded by default?
  • Roadblocks

    • None

    Written by gabrielrl

    November 27th, 2009 at 2:55 am

    Posted in Uncategorized

    Selenium is now integrated in MarkUs

    without comments

    Selenium, a nice framework used to test web interfaces, has been integrated into MarkUs. You can see how to run the tests in this page on the Wiki => How to run Selenium tests

    The following links contain information about Selenium.

    Other useful links :

    Written by simonlg

    November 25th, 2009 at 6:07 am

    Posted in Uncategorized

    Automated Testing: Mockups and some more ideas

    with 5 comments

    The more that I read up on this and think about it, the more I think this will be a huge project. There is really a lot to concern about. In an ideal world, this is what automated testing would do once implemented.

    1. Students have a system at hand, which they can use to learn writing good tests. For instance they would check if their tests are correct, by running them against a reference implementation an instructor would put up at some point before the assignment due date.
    2. Students would learn which parts of the code they are testing (coverage). This would – again, we are in an ideal world – be reported visually in an Grader View like environment. With little boxes, as to where is unchecked code. See the mockup below.
    3. Students would also learn which areas of their code is particularly fragile. This could be achieved by running a static analysis tool on the code
    4. Students would have the opportunity to see public test results of tests the instructor has created and are marked public. The instructor uploads a public test file and could additionally specify a filter file, which is run on the test output. I can imagine these kinds of things would be shell scripts or Python scripts.
    5. Also, students have the possibility to run some more tests on their code, the restricted test suite as specified by the instructor. Say the instructor specifies that a student or group can run these tests at most 1 or 2 times a day. The idea is to encourage students to start their assignment early and to have lots of opportunity to run those restricted tests. For each restricted test, students would have to log into MarkUs and trigger their execution. We store in the database, how many restricted test runs are left for each grouping. Restricted tests can be filtered similarly as public test. By using some sort of script run on the test output.
    6. An instructor could specify a file, which has to successfully run in order to have the submission graded. This would be useful, if an instructor does not allow submissions which do not compile. Such failing “acceptance tests” would be displayed visually in the submissions view and the grader view.
    7. Graders have a visual report ready, when they start grading. They would see the percentage of failing and passing tests, respectively. See again the mockups for more information.
    8. Tests created by students (for instance to run them on a reference implementation), would be submitted via the web interface in a separate section, “Tests” say. The reason for this is to keep tests and code separate. I can imagine that these tests would go into a different folder in the groupings repository (e.g. /tests/a1). Alternatively, students can submit these tests via SVN command line client into the appropriate folder. Tests would be triggered via MarkUs’ interface in any case. This makes tracking of students testing activity and restricted test runs easier.
    9. Students control their test runs via MarkUs. They trigger runs, and get immediate feedback if they are not allowed to run the desired test or get feedback as to where in the testing queue their run is schedulred. Students can wait a while and check back later if there are test results available for them. Maybe we could plan for the option of email notifications for students once the test has been run and results are available.
    10. Automated testing would support graders in many aspects. All tests will be run which the instructor provided for the assignment. This includes public, restricted, private and basic acceptance tests. If students are required to write tests for their code, too, MarkUs could be configured to also run tests in a specific folder in the grouping’s repository (e.g. /tests/a1). All these tests results would be available to the grader in the grader view. Ideally, failures are already pre-annotated in the Grader View. If basic tests are failing (e.g. program does not compile), automatic deductions could be triggered.

    I’ll stop for now. Here are some mocks:

    assignment_creation_overview

    assignment_creation_detailgrader_view_integrationIf you have thoughts, please feel free to chime in. As I mentioned previously, I think this whole topic would be enough material for a master thesis. The big question remains: How would you make sure that you can integrate test runs into the Grader View and other points? It would be nice if test tools would have a standard output format. I can imagine some built in integration points in MarkUs, though. That’s it for now, stay tuned 🙂

    Written by Severin

    November 22nd, 2009 at 2:13 am

    Posted in Uncategorized

    Meeting Minutes: November 20, 2009

    with one comment

    Main points (see IRC log):

    • Meeting starts at 12:06 p.m.
    • Everybody seem to make good progress
    • The flexible marking scheme group has integrated Selenium for some test of MarkUs
    • The Laval students plan to add Cucumber tests, since it’s a requirement for their course
    • Regarding factory_girl, Gabriel isn’t very excited about it (he plans to look into Machinist)
    • The flexible marking scheme is basically missing Cucumber tests for 2 user stories (the rest is implemented, by and large)
    • Farah submitted reviews regarding grade entry form (assignments page, properties view, unit tests)
    • She is 50% complete with functional tests
    • Fernando reports the notes system is slowly moving forward
    • Tara says that the “Notes” tab is mostly done (commit should follow shortly). The delete action works and she plans to work on “Add new” and “Edit”.
    • Fernando uploaded a first version of the modal dialog for creating notes.
    • Tara states that notes will be implemented for groupings only for now.
    • Tara wants to have her tasks of the notes system done by November 28. The following week will be spent on maintenance ticket work and she’ll see what she can do in the days until December 7
    • Severin reports he is mostly doing programming work regarding MarkUs at the moment
    • Karen is looking for good students to work on MarkUs next term. Farah, Fernando and Gabriel are interested. Gabriel has yet to find a way of doing it as part of a ULaval course.
    • The plan is to do a extended post mortem meeting on December 4 (1-2 hours)
    • ReviewBoard activity seems to increase on Thursdays (it has been mentioned that small diffs for review are preferred)
    • Severin mentions that posting code and test in one review request helps
    • Meeting ends at about 12:45 p.m.

    Written by Severin

    November 20th, 2009 at 10:45 pm

    Posted in Uncategorized

    Status Reports, November 20th

    without comments

    Fernando Garces

    Status

    • Committed the notes model
    • Fixed a bug concerning the name of the column type in the notes model
    • Uploaded v1 of the notes modal dialog for review
    • Uploaded locale message for the log system

    Next Steps

    • Modify the modal dialog based on feedback in review board
    • Add documentation and testing

    Roadblocks

    • Time

    Farah Juma

    Status

    • Finished incorporating the option to create a new grade entry form onto the Assignments page and created the Properties view
    • Worked on some changes to the GradeEntryForms controller based on feedback from my review request
    • Worked on unit tests for grade_entry_form.rb, grade_entry_item.rb, and grade_entry_student.rb and have submitted a review request
    • Learned some more about “shoulda” and started working on functional tests for the GradeEntryForms controller (50%  complete)

    Next Steps

    Roadblocks

    • None this week

    Mike Conley

    Status

    • Showed off MarkUs with Severin and Karen during the DCS Research In Action Showcase
    • Reviewed lots of code!

    Next Steps

    • Review even more of the code that’s going through

    Roadblocks

    • Again, just time, and other courses.

    Severin Gehwolf

    Status

    • Was presenting at RIA on Tuesday
    • Fixed some bugs (Changesets: 1073, 1071, 1070)
    • Tried to solve a problem for a MarkUs production instance 

    Next Steps

    • Automated Testing (No change, since last week. I plan to hand a  first version in on December 5)
    • Work on my other tickets (60% done)
    • Review code

    Roadblocks

    • None

    Gabriel Roy-Lartie

    Status

    Next Steps

    Roadblocks

    • Problems with another school project and at my job

    Mélanie Gaudet

    Status

    Next Steps

    • Continue with ticket #457
    • Continue with ticket #174
    • Continue with ticket #309

    Roadblocks

    • Had to work on some other prjects.
    • Had to give some time to my work(need money to continue study ;-))

    Simon Lavigne-Giroux

    Status

    • Integrated Selenium into the project.
    • Did Selenium tests for the login.
    • Did Selenium tests for the warning displayed when changing marking scheme.

    Next Steps

    Roadblocks

    • None.

    Tara Clark

    Status

    Next Steps

    Roadblocks

    • None at the moment

    Written by melagaud

    November 20th, 2009 at 9:50 am

    Posted in Uncategorized

    Minute 13/11/09

    with one comment

    The meeting started at 12:07 and ended at 12:49

    The IRC log can be found here

    Agenda

    • Status Reports
    • News

    Status Reports

    Notes System

    1. The model for the notes system is done
    2. Mock Up for the general view of the note system is done
    3. Nothing yet implemented
    4. Necessary to decide what can be done and what must be left for the next term
    5. Preferred well documented and tested code over finished non documented or tested code

    Foreign Keys Issue

    1. Our current policy of not to use foreign keys at all will be kept for now

    Grade entry

    1. Creation form is done now
    2. Currently working on incorporating it onto the Assignments page
    3. Working on adding the Properties View
    4. Focus will be given to the UI for TAs and instructors.
    5. Student UI will be saved for next term.

    Flexible marking scheme

    1. Instructors’ UI is almost ready to ship
    2. Focus will be in tests rather in adding more functionality

    Logging

    1. Decision made of what to log and what not to log

    News

    1. The University of Waterloo is planning to use MarkUs in January for their CS135 class

    Written by fgarces

    November 16th, 2009 at 7:19 pm

    Posted in Uncategorized

    Status Reports, November 13th

    without comments

    Fernando Garces

    Status

    • Worked on bug #337
    • Started making locale messages for the logger
    • Mock-up for the modal dialog of the notes system
    • Request review of code for the notes model

    Next Steps

    • Implement the modal dialog
    • Log more information

    Roadblocks

    • Lack of time

    Farah Juma

    Status

    • Looked into Foreigner and created a blog post on my findings
    • Resolved the issues I was having with the grade entry creation form! An instructor can now add as many questions as desired and can edit the question names/totals or remove them too.
    • The creation form is now done except for a minor UI issue related to the size and maxlength attributes of some of the text fields and I am getting help from Mike and Severin on this
    • Decided to work on incorporating the grade entry forms into the drop-down menu and Assignments page and creating the instructor’s Properties view before working on the tests (completeness: 70%)

    Next Steps

    • Finish incorporating grade entry forms into the menu / Assignments page
    • Work on tests for creating grade entry forms and editing them too (so far, I’ve done manual tests)
    • Table view for instructors

    Roadblocks

    • None this week

    Mike Conley

    Status

    • Reviewing mountains and mountains of code

    Next Steps

    • Keep chugging through all of this code

    Roadblocks

    • Just time.

    Severin Gehwolf

    Status

    • Fixed assignment test (MySQL issue; 100% complete)
    • Finished a quite large merge (100% complete)
    • Closed ticket 424 (Tests of repo library, was part of the merge; 100% complete)
    • Wrote reviews
    • Fixed bug described in ticket 468 (syntax highlighting of docstrings; 100% complete)

    Next Steps

    • Finish the last ticket (Make it configurable per assignment if external commits are allowed only).
    • Proceed with automated testing paper (~10% complete)

    Roadblocks

    • None

    Gabriel Roy-Lartie

    Status

    Next Steps

    • Try out machinist. See in what it differs from factory_girl and make suggestions on the most appropriate for our context.
    • Fully integrate cucumber.

    Roadblocks

    • None this week

    Mélanie Gaudet

    Status

    Next Steps

    • Continue with ticket #457
    • Continue with ticket #174
    • Continue with ticket #309

    Roadblocks

    • Some SQL commands are applied, with a migrate, in Dev DB but not in Test DB. I can apply those specific SQL command on test DB only via the SQL Editor.

    Simon Lavigne-Giroux

    Status

    • Looked a bit at the warning in the Assignment properties’ view
    • Worked on my other projects.

    Next Steps

    Roadblocks

    • None.

    Tara Clark

    Status

    Next Steps

    Roadblocks

    Written by Tara Clark

    November 13th, 2009 at 12:33 pm

    Posted in Uncategorized

    Foreigner Findings

    with 2 comments

    I’ve been looking into Foreigner to see if this will help us with our foreign key issues and I am liking it so far! Here’s what I’ve found:

    Overview

    Foreigner can be installed as either a plugin or a gem and provides us with methods that we can use in our migrations for adding and removing foreign keys. A great thing about Foreigner is that it works with both PostgreSQL and MySQL. This differs from the MigrationHelpers class we currently use, which was designed to work with PostgreSQL only. Migrations that use Foreigner will also work on databases that do not support foreign keys (eg. sqlite3).

    Migrations

    There are a couple of different ways foreign keys can be specified in a migration:

    • If we want to specify that there should be a foreign key from the GradeEntryItem table to the GradeEntryForm table, we could add the following line:
      • add_foreign_key(:grade_entry_items, :grade_entry_forms)
    • The add_foreign_key method can also take some options. For example, we could include the :dependent option as follows:
      • add_foreign_key(:grades, :grade_entry_items, :dependent => :destroy)
    • Alternatively, foreign keys can be specified as part of the create_table statement (t.foreign_key takes the same options as add_foreign_key):
      • create_table :grades do |t|
        ...
        t.foreign_key :grade_entry_items
        ...
        end

    With Foreigner, we’ll even be able to see the foreign keys in schema.rb!

    It seems like Foreigner would provide us with a simple, straight-forward way of adding foreign keys to our migration files. I’ve installed the plugin on my machine and am going to try it out to see how it works.

    Fixtures and foreign keys

    One thing to keep in mind is that the order in which fixtures are loaded becomes important when foreign keys are involved. Things can start to get kind of tricky. This is why alternatives to fixtures (like factory_girl) are good things to investigate.

    Written by Farah Juma

    November 11th, 2009 at 11:04 pm

    Posted in Uncategorized

    Notes Modal dialog

    with 3 comments

    This is the modal dialog for the notes system. I stole the idea from review board and just change it a little bit.

    One thing that i am planning to do but I couldn’t right now was to put the username in bold.

    This dialog is going to be available in the assignment-> groups & graders view and in the submission view of the graders

    notes view1

    Written by fgarces

    November 11th, 2009 at 2:57 pm

    Posted in Uncategorized

    Meeting Minutes – Nov. 6th, 2009

    without comments

    The meeting started at 12:00 and ended at 12:53.

    The IRC log can be found here

    Agenda

    • Talk about whether everyone feels that they are making sufficient progress to finish their assigned tasks by the end of the term
    • Follow up on any questions

    Progress

    • Karen liked seeing the percentage numbers in the status reports
    • From the flurry of review requests on Thursday, it seems that everyone is making good progress

    Simple Grade Entry

    • Farah said that finishing the creation form is taking longer than she thought it would but she’s learning a lot in the process
    • She said the one ticket she is concerned about not getting to is the one for the student view
    • Karen said that it might be helpful for her to sit down with Severin or Mike to get some tips and pointers
    • Farah said that she will ask Mike or Severin for help

    Tickets

    • Tara said she is stuck on how to do tests for Ticket #358 (see https://stanley.cdf.toronto.edu/drproject/csc49x/olm_rails/ticket/358 ) since the controller tests are in bad shape
    • She doesn’t want to fundamentally change the add_member action without writing proper tests
    • Karen said that it may be worth spending the time to fix up the tests for the controller
    • Tara has written out the test cases she should cover, but the fact that the tests are already so broken makes her feel like she should start from scratch
    • Tara’s going to work on the tests and consult with others if she runs into trouble

    Notes System

    • Tara is less worried about the Notes system since she’s working with Fernando and they have been collaborating and meeting
    • Fernando said that since they’ve just started the Notes System, it’s hard to say whether or not they’re on schedule but the notes system for the groupings should be finished

    Foreign keys issue

    • Mike said that there seems to be a two-way split on this issue
    • Mike said that our migration helpers are designed to work with Postgres and not MySQL
    • Farah and Tara have been able to run the migrations successfully on MySQL but Fernando, Severin, and Mike have had problems running these on MySQL (Severin said this is likely a version issue)
    • Mike said that the Foreigner gem/plugin that Mike G. pointed out works with both Postgres and MySQL
    • Mike said that it’s just a matter of including Foreigner, and writing a migration to add foreign keys where they’re missing
    • Farah and Melanie are going to look into Foreigner and report back on their findings
    • Simon asked if there is another technique instead of fixtures to build objects for tests
    • Gabriel brought up factory_girl and he has been looking into this already and likes it a lot

    Flexible Marking Scheme

    • Gabriel feels like he’s right on schedule
    • Gabriel said that writing the flexible criterion edit view (for admins) is making him learn a lot of little things that were still not clear
    • Simon said that the marking scheme type can now be changed from the Assignment view
    • Simon added a dynamic warning message for the case where an instructor attempts to change the marking scheme after it has been set. It might be overkill though and he’ll look into a static message or JavaScript (no communication with the server)
    • Karen said we definitely need a warning to appear
    • Simon asked Karen if she prefers a static message (the message is always in the page while editing) or a dynamic message (it only shows up when clicking on another marking scheme)
    • Karen said that she prefers a dynamic message and that consistency with other warning messages is important
    • Simon’s going to work on this and will ask for help, if necessary

    Recruitment for next term

    • Karen said that recruitment for next term has begun and we should let her know if we know anyone who is good and might be interested
    • It also looks like we might be getting another summer intern from France again this summer!
    • Karen just put in a grant proposal to see if she can get money to hire two students to build the automated test system this summer
    • Many of us said we want to continue with MarkUs in one way or another even if it’s not for credit!

    Time management

    • Karen reminded everyone that the end of the term is approaching quickly so we should all keep track of time

    Written by Farah Juma

    November 6th, 2009 at 7:36 pm

    Posted in Uncategorized