# MarkUs Blog

MarkUs Developers Blog About Their Project

## Status Update – Jan 29 2015

Christian

• This week
• Figured out what was going wrong with displaying remarks to markers
• Next week
• figure out why the save button was deleting marks and fixing that

Peter

• This week
• Fixed issue-1760 – Converted Student Interface’s marks spreadsheet table to React.js
• Next week
• Work on issue #1974
• Others
• It’s my first time working with React.js so any feedback on implementation would be appreciated

Paymahn

• This week
• Finished off #1795. PR still hasn’t been merged but I’m confident it’s done.
• Next week
• Start working on integrating MathJax. Will be mostly investigative work and maybe some playing around.

Ryan

• This week
• Finished the first iteration of text annotation for specific text selections
• implemented it by using javas execCommand function
• this presented several new issues, including breaking original text selection method
• decided it would be best to redo the highlight function and add classes to the text nodes between the end and start column then split the end/start into two spans, one to leave plain the other to highlight
• fixed up the way the column offsets were calculated when there were multiple spans in the line
• Next week
• I need to finish implementing the new highlight method
• all that’s left is to clean it up and handle a few edge cases.
• Problems
• struggled to parse all the DOM nodes, specifically finding the applied styles

Maryna

• This week
• learning respec (BDD, Red/Green testing, basic tests, built in matchers, stubs vs mocks, fixtures and factories
• examined present markus code
• Looking at models and associations and attributes to understand DB structure
• Write the tests for calculate_collection_time and calculate_grouping_collection_time to close ticket #218
• Next week
• to continue working on assigned tickets (adding new tests to some implementations)
• to start looking into translating existing rake tests into RSpec
• Problems
• understanding the reasoning/meaning behind some of the code
• assignment.latest_due_date + hours_sum.hours and hours_sum is always 0. Type casting?

Josh

• This week
• NA
• Next week
• NA

January 30th, 2015 at 12:28 am

## Status Report – Jan 23 2015

Christian

• Last week
• Confirmed existence of #2014
• Refactored a bit of the remark display in the results controller
• Made old marks populate the marks fields when remarking assignments (issue 1975)
• This week
• Fix issues 2014 and 1869

Maryna

• Last week
• worked on issue 76: Disallow non-AssignmentFiles option in Assignment Properties. done and pushed. The major challenge: work on the front end (styling). Had to learn the basis of jQuery to make options pop up in the right places.
• learning RSpec from the scratch by video tutorials. Reading report provided by one of the students who worked on Markus previously. So far so good.
• This week
• review the RSpec tickets assigned by David and find the easiest one to start with.
• If I would require more time to learn RSpec, my plan is to find some unsigned issues that I could work on in the meantime.

Ryan

• Last week
• worked on getting the text annotations to display on only the selected text and not the entire line
•  managed to get the start and end columns saved into the database while handling a few edge cases
• made progress getting the data back out of the DB and to the client side.
• This week
• Finish off annotations.
• Having trouble with
• working with bare bones JS to be consistent with the rest of the annotation class and keep the client side processing to a minimum
• staying awake after experiencing jet lag and getting back into the swing of things
• It turns out that not all of my changes were committed to github, I would assume due to my own user error. It will put me back at least a few hours but I should be able to redo the work and meet my goals regardless. I’ll consider this a lesson learned.

Paymahn

• Last week
• Fixed issue 1917. There was an issue with links in the dashboard for spreadsheets and assignments going to settings instead of what would be expected
• Fixed issue 1884. Flexible criteria could not be downloaded. This was happening because the client was making a POST instead of a GET
• Worked on issue 1795. The error message for criteria created with a duplicate name was awful. I’ve gotten the message fixed but this broke some tests. I’m in the process of converting the broken tests (and their respective files) to RSPEC
• This week
• Looking to finish off issue 1795 (third bullet point above)
• Will start investigating my project
• Will need to learn javascript (I’ll probably do some tutorials or something and incessantly annoy Ryan with questions)
• Having trouble with
• RSPEC doesn’t make sense (maybe I can’t find the right docs?)
• Specifically trying to understand how to properly use helper methods for the tests

Peter

• Last week
• Learning react framework – tutorials, studying implementations in codebase
• Updating Student Marks Spreadsheet Table to React (Issue 1760)
• This week
• Complete updating Student Marks Spreadsheet Table to React (Issue 1760)

Josh

• Last week
• few more rails tutorials, began (re)familiarizing self with vagrant, bash, virtualbox, etc
• Not much substantial yet, catching up on classes and stuff
• This week
• Begin research/work on #1804 (refactoring Flexible/Rubric criterion models)
• If possible, begin on issue #1973, but that’s probably a low priority feature atm, and 1804 will likely take at least a few weeks given how little I know of Rails.

January 23rd, 2015 at 2:08 am

## Winter 2015 Project List

• Convert tables to use React
• Outstanding issues (ordering of columns is still a problem)
• UI improvements: Anchoring the first column for better horizontal scrolling

The semantics of the different uploading features is varying and unclear.  We especially need to take a close look at the uploading of data in to the spreadsheets, groups, and graders views. Working on this project will require a deep understanding of the operation of each of these uploading functions, looking at the database and repository operations, and documenting the behaviour in the help system.

The other area where CSV uploading needs improvement is that we sometimes encounter performance problems where Apache seems to time out.

3. React refactoring (smallish)#1696

In the past two terms, we have converted tables to use the React library.  As we near the end of this process (spreadsheets are the last ones), the code for the tables could use some refactoring to simplify it and pull together common functions.

4. Get rid of the rest of Prototype #1496 (small)
We have slowly been removing all the Prototype code, and replacing it with JQuery. Aside from the tables that still need to be converted to use React, there are still a few places where Prototype has not been fully replaced. This would be a good project for a student with an interest in learning (more) Javascript/JQuery.

5. Fixing issues with and improving the remark system #1933, #1869, #1975, #1976

We encountered some bugs with the remark request system, and have some feature requests. This will involve the remark request view and the marking scheme models.

5. Git backend #1698 (large)

Alex worked on the git backend last term, and is planning to finish it this term.  The main piece that is left is incorporating authentication so that students can use git directly.  He is planning to work on it again this term.

A lot of work has been done to upgrade MarkUs to use Rails 4.x.  There is still some work remaking to complete this upgrade. This project would be a great one for students with some understanding of Rails, and have an interest in how the Rails framework works.

7. Rspec tests (large, but an ongoing effort)
This summer saw a major effort to change to Rspec tests.  In the fall we were successful in getting all students to write some Rspec tests, and Kuba also spent the term refactoring and testing the Assignment model. A continuing goal is to have everyone on the team write some Rspec tests. It will be a good way to really learn what the models and controllers are supposed to do, and will move the project forward. We plan to set aside a few hours at the sprint for writing tests. There are some outstanding Github issues related to missing tests that can easily be closed with some work.

8. Improve seed data for the development environment

In the dev environment, there is data to populate the data base with users and assignment submissions.  This seed data helps with manual testing and understanding how the system is supposed to work.  We need to increase the size of the seed data to include more students, more assignment submissions, and to increase the diversity of initial scenarios such as different due dates for submissions that affect more of the system.

9. Summary page of all the marks for all students (smallish)
There is currently no view that combines marks from all of the different assignments and spreadsheets. This table would look a lot like the Submissions table or the Summaries table, but would have one column for each assignment and spreadsheet. A new feature would be a way for the administrator to specify a weighting for each piece of work to produce a total. (Good for someone not familiar with Rails.)

10. MathJax support for annotations? #285 (medium?)
It would be nice to be able to use math symbols in annotations. The MathJax library seems to be what we want, and some work has been done on this.

11. UX Refresh of the submissions table (Includes #75) (medium)
We haven’t taken a serious look at what is in the submission table for a long time. For example, we probably don’t need the “can begin grading” field. We would also like to be able to show the grader(s) for each group.

12. Section due dates don’t work #1676 (small)
Some courses would like to have a different due date for each section. This feature seems to have numerous problems with it. There is also a proposal to change the UI for how sections are added.

13. More flexibility in the marking schemes

There are lots of ways in which the two available marking schemes could be improved.

14. API improvements

It would be nice to make the API more powerful so that we could take more advantage of the API for automated operation.  This will also involve writing some scripts and consolidating scripts that interact with the API.

Written by Markus

January 13th, 2015 at 1:23 pm

## Tagging Backend

This blog post will discuss the decisions made surrounding the backend of the Tagging Project.

Database Schema

As this is the initial implementation of the Tagging feature, we strove to keep the schema changes as simple as possible.  MarkUs uses ActiveRecord as the M in MVC, and thus we had to create ActiveRecord migration files to implement changes.

We decided to create a Tags Model which contained the following explicitly declared pieces of information:

1. name (string)
2. description (string)
3. user (string)

and an integer id that is implicitly declared.  Furthermore, we created a groupings_tags association to create a many to many association between groupings and tags.

That migration file explicitly associates and a tag id with a grouping id and adds a index of the two ids that must be unique.

An interesting thing with ActiveRecord migration files, and increased its ease of use, is that each migration file must specify the action being done(creating table, adding an attribute to a table etc) but also the action that must be done to rollback these changes.

In this case, the rollback operations for each files solely consisted of dropping each table(and removing the index for the grouping_tags association).  This made it quite easy to create and edit the schema in subsequent iterations.  Editing the schema file would consist of the rollback command, editing schema file, and re-migrating.

Models

In editing the model files, we only had to create a tag.rb and modify the grouping.rb to reflect the new schema.

Controllers

In creating the tag controllers, we had to split up the logic between a tags_controller.rb and a tags_helper.rb module.

The interesting between the tags_helper module and the tags_controller is that other controllers can include the module and directly call its method.  This is not the case for tags_controller; routes.rb needs to consulted to call its methods.

The methods in the tags_helper modules are used by controllers (e.g. tags, results) whereas methods in tags_controller are called from views.

For example, methods in the controller include index, edit, create, destroy.  The module has methods that do work with groupings and tags, which are needed by the results_controller.

Written by Nathan Chow

December 15th, 2014 at 1:31 am

## Status Report – December 5

Chris

This Week:

• Completed the PDF annotation editor/viewer (i.e. view, load, save, delete)

Next Week

• Fix an issues from the last pull request.

Jakub Subczynski

Final Week

• renamed methods throughout codebase:
• Assignment#past_due_date? to #past_all_due_dates?
• Assignment#what_past_due_date to #section_names_past_due_date
• Assignment#section_past_due_date?(grouping) to #grouping_past_due_date?(grouping)
• created Issue #1926 regarding the intended functionality of Assignment#past_all_due_dates? when there the assignment has sections
• created RSpec tests for:
• Assignment#get_detailed_csv_report

Yusi

This week:

• Reported some issues

Next time:
Fix

• https://github.com/MarkUsProject/Markus/issues/1924
• https://github.com/MarkUsProject/Markus/issues/1923
• https://github.com/MarkUsProject/Markus/issues/1922
• https://github.com/MarkUsProject/Markus/issues/1920

Nathan

Last Week:

• have a code fix updating the submissions table to display real associated tags
• final exams and end of term projects have kept me pretty busy

Next Week

• Will be done finals Thursday of next week
• will work on implementing RSpec tests, any remaining features for tagging and writing a blog post

Written by Chris Kellendonk

December 5th, 2014 at 3:52 pm

## PDFJS Annotations II

For this term I have been working on replacing the old image based PDF annotation system with a new one that uses the native file format. The new system allows the students and the markers to view PDF files in a much smoother native fashion. As well as allows markers to annotate the PDF files directly in the viewer. I believe the overall user experience has improved as a result of this change.

In order to implement this feature the [PDF.js](https://github.com/mozilla/pdf.js/) library by Mozilla was used. This library is designed to render PDF files in the browser using only Javascript and HTML5. But was not designed for adding annotations to a PDF file.

This change has been very beneficial for a multitude of reasons:

1. It has increased the speed of the assignment collection by completely removing the long process of converting each submitted PDF into a series of images.
2. Allows for a smoother workflow when viewing and annotating PDF’s. There is now support for changing the zoom level, and there are page number controls for navigating through the pages.
3. It is now resolution independent. It doesn’t matter what resolution the PDF file was saved as, it will always render correctly to fit within the screen nicely. While still allowing the user to zoom in on pages if need be.
4. Annotations can be drawn directly in the viewer and thus will change with the zoom level of the document. This is beneficial when small images or specific areas need to be annotated.

The main challenge with working on this component was implementing the annotation system on top of the PDF.js library. The main viewer code was largely undocumented so a fair amount of reverse engineering had to be done in order to understand the rendering process and insert the hooks for annotations. The PDF.js library itself was not modified at all during this process, instead hooks were added into the rendering process so that the annotation system could sit on top of PDF.js without being coupled too heavily to the library. This should make it simpler to upgrade to new PDF.js versions as they are released.

Currently the system does not support downloading PDF files to a user’s computer with the annotations rendered in them. They can only be viewed online. An improvement that could be made in the future would be to render these annotations to the actual PDF file before a user downloads them. [Origami-PDF](https://github.com/mobmewireless/origami-pdf/) is a library that may be able to do this.

Overall I think the system works quite well. The annotation system behaves exactly the same as the image annotations so the user experience will be seamless when transitioning from one system to another. Thus a very small learning curve. And having an actual PDF viewer makes it much simpler to navigate through and markup PDF files.

Written by Chris Kellendonk

December 5th, 2014 at 3:26 am

## Rails 4 Upgrade Notes II

This term, I have been working on the upgrade from rails 3 to rails 4. The upgrade process consisted of two phases. Firstly, I upgraded MarkUs from rails 3.2 to rails 4.0. Then, I upgraded it from rails 4.0 to rails 4.1. Each phase consisted of some parts that need to be changed by referring to two upgrade guides in Rails 4 Upgrade Notes

Right after each upgrade phase, I ran both unit test and Rspec test. There were many deprecated warnings and errors showing up. In order to make pull requests small, I
ran each test from unit test and Rspec test separately and updated the code.

After all warnings and errors from the tests were fixed, I tested MarkUs manually. There are some issues after the upgrade and need to be fixed.

https://github.com/MarkUsProject/Markus/issues/1924
https://github.com/MarkUsProject/Markus/issues/1923
https://github.com/MarkUsProject/Markus/issues/1922
https://github.com/MarkUsProject/Markus/issues/1920

Also, The following tasks still need to be done.

Extracted Gems
Since many features that were present in earlier versions of Rails were removed from Rails4 itself and extracted to gems, I have added the following Gems in order to make the upgrade process as smooth as possible.

gem ‘actionpack-action_caching’, ‘~>1.0.0′
gem ‘actionpack-page_caching’, ‘~>1.0.0′
gem ‘actionpack-xml_parser’, ‘~>1.0.0′
gem ‘actionview-encoded_mail_to’, ‘~>1.0.4′
gem ‘activerecord-session_store’, ‘~>0.1.0′
gem ‘rails-observers’, ‘~>0.1.1′
gem ‘rails-perftest’, ‘~>0.0.2’

But not all of them are needed in our application. Those gems that are not used in our application should be removed. For more information about what the gems do, Please refer to upgrading to rails 4 page 71.

Since the final exams are approaching, I didn’t have time to finish the above tasks yet. And I will try to finish them after all finals have completed.

Written by Yusi Fan

December 4th, 2014 at 5:49 pm

## RSpec testing in MarkUs

My focus for this term has been testing the Assignment model with RSpec. The major components of this testing included:

1. Migrating existing rake tests to RSpec tests,
3. Improving the Assignment model itself (through adding additional validations, removing unused methods and improving existing methods).

I have been creating tests for the model using a unit-method approach. This is what is suggested when testing models. I created a separate describe block to test each Assignment method and only that method. Every method should have an associated describe block upon completion of the Assignment model testing. When I finish testing a method, I delete the associated tests from the rake test suite. Therefore, the old Assignment tests should also all be removed by the time all of the RSpec tests are in place. It is worth noting that there are many methods in the Assignment model without tests for them. One should not rely on simply migrating the tests to ensure good test coverage.

I realized early on that, realistically, there is a good chance no one will inspect the coverage of these tests for a long time after I’ve written them. This is why I was very thorough when testing. I focused on the methods one-at-a-time instead of thinking about the amount of work left to keep myself from just speeding through them. My boiled-down process looks something like:

1. Focus on a method that is in the current test suite.
2. Inspect the method to see what it does and how it does it.
3. Create RSpec tests for that method.
4. Ensure all tests in the old test suite have been covered and remove them.
5. Commit changes.

I make changes to the method throughout the process if I see anything that could be improved.

I suggest drawing out a diagram of the associations between the models you are working with. Jumping into testing in a new codebase can be a bit daunting and drawing out a rough diagram really helped me understand how the models fit together. In addition, be sure to play around with the application to find where each model matters. It will help create a mental picture of models as you work with them.

I learned RSpec while working on the tests. Here are some useful links I’ve gathered on RSpec testing:

Written by Jakub Subczynski

December 4th, 2014 at 1:05 am

## Status Report – December 2

Irene

Last Week:

• Added test cases to Assignments rspec tests
• Updated Main Controller functional test

This Week:

• Make some final visual changes in assignment summaries

Tori

Last Week:

• looking over React tables to determine which need a custom sort
• custom sort function for grace credits (sort by fractional value remaining/total)
• small refactoring of last week’s sort functions

This Week:

• making the compare functions more general; moving some of the parsing work back into table.js

Bryan

Last Week:

• Fixed up some of the changes with the Tags view.
• Moved the Tags view link up to the sub menu. It was previously located in the sub sub menu. Tags are global in scope as opposed to confined to an assignment.
• Got most of the Results view working. Tags can now be assigned and unassigned from particular submissions. This was done using Ajax to prevent page loads every time a tag is assigned.

This Week:

• Bug fixes?
• Finishing up the Tag feature.

Written by Irene Fung

December 2nd, 2014 at 9:50 pm

## Status Report – November 28

Jakub Subczynski

This Week

• Refactored Assignment spec:
• grouped certain tests (i.e. association and validation tests)
• replaced .to be and .not_to be with .to be true and .to be false when expecting a boolean value
• other small improvements
• RSpec tests for:
• Assignment#section_due_date (and rewrote actual method)
• Assignment#past_due_date?
• Assignment#latest_due_date
• Assignment#section_past_due_date? (and modified actual method)
• Assignment#what_past_due_date
• Started putting RSpec blog post together

Next Week

• Complete blog post and
• You guessed it — more tests!

Chris

This Week

• Completed drawing/displaying annotations.
• Completed saving annotations.

Next Week

• Update/Write blog post on the status of annotations and how they work.

Yusi

This Week

• Worked on TODOs in the comment that is related to rails 4
• Change:
find_all_by_… to where(…),
find_or_create_by_ to find_or_create_by,
find_last_by_ to where(…).last
• Manual tests

Next Week

• Write a blog post
• I found some errors caused by the new “all()” api today, I will update the code related to it.
• Manual tests and discover more errors

Written by Jakub Subczynski

November 28th, 2014 at 2:04 pm