# MarkUs Blog

MarkUs Developers Blog About Their Project

## Weekly Status Update: Oct. 19, 2015

Jeremy

1) Carried on looking into the test-framework branch, specifically the code for form and file uploads. Worked on delete grader functionality
2) Continue working with Wilson on test script and test support file uploads, ensuring attributes for assignments are updated correctly
3) Just figuring out how all the form/test code fits together and identifying where errors are coming from.

Wilson

1. Spent this week working on getting the view for the test file upload working. Solved a few errors caused by missing functions in the helper, forms, and in the navigation bar. Gained a much better understanding of how everything goes together.
2. Ran into the issue with AssignmentRules, but David already addressed those for me in the PR.
3. Going to continue addressing issues of broken functionality in the file upload section and figuring out what still needs to be fixed / what works.

Shaughn

1.
– Addressed some comments in PRs to allow them to be merged
– Added a functionality to calculate # of annotations per marked submission on a per assignment basis for TAs
– This is present in the submission status column for TAs and teachers right now, but plan to move this to dashboard
– Admins see as a whole how many annotations there are per submission (i.e. across every TA)
– Plan to add a quick reference to each Grader’s average annotation per submission using the function created for TAs

2. No problems, been a bit tough to juggle with studying for exams, and assignments this week.
3. This week, I plan to add some functions to calculate distribution of marks for each TA. If time permits, I will look into adding visualizations for these metrics to the dashboard and any other areas where appropriate.

Melissa

1. I was able to complete the removal and testing of the manager_graders.js and manage_groupings.js files in the app/views/javascripts folder. I also researched some Prototype specific syntax so I know what to look out for when I am looking for inline Prototype to remove. I also started to look for where things break when I remove the inclusion of Prototype from content.html.erb.
2. This week was a little busy with assignments for other courses, so I wasn’t able to do as much as I would have liked.
3. Next week I will continue looking for inline Prototype code and checking to see what removing the inclusion of Prototype may break. I also would like to look into the FilterTable.js file, and try to figure out if it is still in use and what it’s for, as I think it’s related to Prototype.

William

So these last couple weeks I fixed a few PDF issues and also a bug regarding newlines not showing up in remark requests. There weren’t any major obstacles. Next week, I will start on the grading interface.

Binuri

So last week I was still working on the repository set up when groups are set up through csv files. I created outline code but the code and the logic is very confusion. So I met up with Karen for and we decided that some of the previous code that already exists may not work/outdated and certain steps need to be reconsidered. So I went back home, I reformatted the flow charts etc that I drew to figure out the code and I have to meet up with Karen again and see if my understanding is correct and to see if the behaviour is appropriate and then finish up the new code

Written by William Kwan

October 19th, 2015 at 3:20 pm

Posted in Uncategorized

## Weekly Status Update: Oct. 5, 2015

Jeremy

1)  Looked a little more into the the results of the branch merge. There’s definitely a few things which are broken but it’s not immediately clear what has/hasn’t been implemented. Wilson and I have informally documented a couple of things which aren’t working right now
2) –
3) Schedule a meeting with Karen, David and Wilson to set some milestones for the project. It’s still not super clear to me what has already been built and where we’d like to be at the end of the term. So I think meeting up to set a few intermediate markers will be helpful (I’ll email).

Shaughn

**This Week**

– One spreadsheet is visible, with marks entered, released to students, dated immediately

– Another is hidden, has no marks entered, has not been released to students, dated two months from seeding time

– Investigated rubocop pre-commit file

**Troubles**:

– Learned that the precommit script will only run on files that have been added (staged) for a commit

**Next Week**:

– TA Analytics

Melissa

1) This week I worked on converting the last .rjs files in the views folder into .erb files. This is almost done now, 7 files have been deleted/converted and only 2 more remain.

2) I ran into a roadblock that came from my lack of Rails knowledge. I realized that a lot of the .rjs files I had to convert weren’t even being used anymore. Originally I thought they were just not working properly, but after investigating the current codebase and checking in the project’s git history I found that in some cases the method or action that the .rjs file was related to in the controller was removed. This resulted in me realizing I could just delete a lot of the .rjs files that still exist (especially in the grades and assignments view). After that I was back on track.

3) This week I will finish up those last two .rjs files and submit a pull request for them (hopefully by mid-week). Then I’ll be moving onto the app/assets/javascript/ folder and working on the manage_groups.js and manage_graders.js files. I also want to spend some time looking into the Gitolite issue I had with my setup that hasn’t been fixed after the sprint weekened. David sent me a helpful resource so I’ll start there

Wilson

1) Implemented a different fix to fix the broken tags list. Also setup Jeremy’s merged test branch and investigated into what is working / what is not and documented it.
2) Mostly just getting used to how RoR handles migrations. Ran into a few issues with dependencies on the test-framework branch but got those worked out.
3) Jeremy and I are going to have a meeting with David and Karen this week regarding the exact scope and features wanted for the testing framework so we can figure out how to tackle it.

Binuri
The one I posted before seemed a little extra so I narrowed it down.
1) Last week with the little assessment i did, my plan to tackle the full issue of groupings in one go was unsuccessful, so I decided to break it down. This week I came up with an approach to taking the repo_name in to consideration when creating groups through a csv file upload.
2) I ran in to a lot of road blocks where I didn’t know which step to take, for example, do I reuse the group when a repo name is found etc. Also I didn’t know the existence of hooks, so it took me some time to figure out the actual flow of methods/objects.
3) This week I  plan to continue with the repo names in the csv file group uploads (edited)

William

Last week I made the annotations rotate when the JPG/PDF gets rotated. This was trickier than rotating just the image because the annotations are generated by coordinates, so some math needs to be done. For PDF, it wasn’t too bad because the coordinates are stored as percentages local to the PDF boundaries, but for JPG, the coordinates are in pixel space relative to the entire page. Also, rotating JPGs that aren’t square can cause them to overlap with the panel boundaries, so I added an extra translation to align them with the edges of the panel.Although I got the annotations rotating to the correct positions, I found some bugs in the annotation/image viewing code(#2308, #2307, #2306). For next week though, I’d like to fix some issues with my own code first. In no particular order:

1.
If the grader tries to make an annotation while the image is rotated, the coordinates are stored in the db as if the image isn’t rotated.
2.
For rotated annotations, the text sometimes doesn’t show up when the mouse is hovering over it.
3.
Large JPG files with scrolling have wonky behaviour when the img element is rotated. The scroll bars aren’t updated to swap the width/height, and rotating the annotations on the img can bring them outside of the iamge viewing panel, which is right, but I need to figure out how to hide them until the grader scrolls over and update their positions as they scroll.
4.
I think there’s some sort of mask at the bottom of the JPG view that needs to be resized upon rotation. When an image is uploaded in landscape orientation and rotated to portait, the bottom of the image is cut off by blank white space.

Written by William Kwan

October 5th, 2015 at 3:25 pm

Posted in Uncategorized

## Weekly Status Reports

Sept. 21 – 28, 2015

Jeremy

1) Merged master into test-framework branch. This is now on my local fork.
2) Couldn’t figure out why the server wouldn’t start after my merge – but David helped me figure this one out!
3) Going more carefully through things that were merged to verify and test existing functionality. Start to look at at what test-framework has to offer. Coordinate with Wilson

Wilson

Was a little busy this week, so didn’t get a lot done. Mostly worked on fixing the issue regarding tags not showing up. After spending a lot of time figuring out the debugging tools for Ruby on Rails, I figured out that we are storing the memory address for the user instance creating the tag instead of the actual user in the database. It was actually pretty hard to find this error since no exception was raised. I’m a little confused at how this ever worked since it seems like it should’ve always been broken. When the controller goes to try to find the user who created the tag, it encounters an error and 404s. I have fixed this but there’s a schema change I wanted to consult David about before changing that should probably be done.
Next week I will be talking to Jeremy about what we need to do in order to get the existing automated testing code working.

William

Last week I made a minor fix to the spreadsheets and added a button that lets you rotate JPGs and PDFs. The main issue was that it did not rotate the annotations as well so I’ll be investigating that this week. After I’m done that, I’ll either do another one of the PDF issues or change the UI of the grading interface.

Shaughn

**This Week**:
Investigated rubocop and its usage/config in Markus project
-Added rubocop-git to project to be used as a tool to check for style conflicts before committing
-Created a ruby script (lib/tools/pre-commit) that uses rubocop and rubocop git to automatically check diffs and rake files to make sure they will not cause an issue with Hound when PRs are created.
– Dependencies are not installed on local machine when using sshfs so if I try to use some of the tools in our project, it failed.
**Next week**:
– Add marks for each student in the new seeds for marks spreadsheets.  Going to look at how this is done randomly in the assignment seeding.

Melissa

1) This week I was playing catch up with school, but managed to finish some things I didn’t get done during the code sprint. I completed generating seed data for remark requests, and fixed a 404 error I found when instructors tried to comment on remark requests.
3) Next week I’m going to start working on my project, removing the last traces of Prototype. The first thing I’m doing is converting the last .rjs files in the views folders into .erb files. So far this has been going pretty smoothly.

Binuri

I was a little busy catching up on school although i have no idea how i got behind in the first place. I started looking at the code to creating repositories.

Written by William Kwan

September 28th, 2015 at 3:54 pm

Posted in Uncategorized

## Projects for Fall 2015

Here is a list of things we would like to work on.

Issues: #2237#2180#1560

There are several problems to address here, and anyone working on this should consult closely with us to make sure they understand the issues thoroughly.  Groups and their repos are created either when a student logs in and navigates to an assignment, or when an instructor uploads a group list.

One issue is that the group upload using a csv file ignores the repo name in the csv file and auto-generates the repo name (group_xxxx).  This is done to avoid name collisions, but it would be better if we can check to see if the repo name already exists, and if the membership is already the same.

Another issue is performance.  We get a 502 proxy error when we run it through Apache for large numbers of groups (>400).  The CSV upload deletes existing groups when it does the upload, so we can’t even do it incrementally.  Two possible solutions to this problem are to refactor the code so that we can do the upload incrementally, or refactor the code so that the operation is run as a background task, so that the proxy error does not cause the transaction to abort.

We also want a way to create the repos and/or the directory in the repos in an automated way for individual assignments so that instructors can add starter code.  Issue #2180 describes this in more detail

Complete work on the Git backend

We are nearly completion on a Git backend to allow student submissions to be stored in a Git repository.  The remaining todo list is in #1855.

Remove the last calls to the Prototype library

Issue: #1496

We switched to jQuery some time ago, but there still remain a few places where Prototype code remains.  We would like to finish getting rid of this.

Report on TA activity

It would be nice to see summary data of what TAs have done.  Metrics such as ratio of assignments marked,  average number of annotations per submission, and a distribution of marks for each TA would help instructors identify differences in how the TAs are grading.

4. UI of student interface

When a student navigates to an assignment, the landing page could use some UI work.  The tab is labeled Assignments which isn’t correctly because it is a particular assignment.  We also need to look at what information is displayed and whether there is a better way to organize it.

We could also look at issues with the results view of the assignment.  Binuri mentioned that the students don’t see the weights in the rubric view.

5. Working on the production setup instructions

We occasionally run into problems that we can’t replicate on our development environment, and we also don’t do a very good job of supporting our system administrators who deploy several instances of MarkUs running under the Apache web server.  We would like to put together a set of instructions for both these purposes.  It would be great to be able to easily configure a virtual server running a production instance (or multiple instances of MarkUs).

Change marking status to a “complete” button. Include more info for TAs (x/y submissions complete)

(Relevant issues: #2118)

Auto-testing interface

Dig into and update existing code.  The goal is to reach a proof of concept stage where a single test can be run by clicking a button. There is likely a lot of refactoring to do starting with getting the test-framework branch up to date with master.

The existing design has results being sent back in XML.  At least one of the auto-testers we are working with uses JSON, which seems like a more sensible format, so modifying our design to use JSON is the ultimate goal.

PDF issues

There are a few issues that with PDF annotations that would be worth looking into (#2011#2043#1936#1935#11092042).  A user has also reported a wish list of additional features including: switching between portrait and landscape, moving and resizing annotations.  We also notice some problems with annotations on Chrome that could be investigated.

General issues

There are a number of medium sized issues that really need resolving, but are too big to qualify as an “easy fix”.  Examples include (in no particular order): #2050#2109 #2130#1705, and others.

Written by Markus

September 19th, 2015 at 5:32 pm

Posted in Uncategorized

## Remark System

I went in to this project with no knowledge of anything in it. Markus runs on Ruby, Rails, JQuery, HTML, MVC pattern, RESTful services, and raw Javascript. Basically, I had none of these things. I’d never touched web development before (barring a small ‘computers’ class in elementary, where a high school student taught me how to link to images on other people’s web sites by reading and writing pure html in the 90’s). I had to give myself a crash course in Ruby and Rails in order to understand enough to begin reversing behaviours through observation. On top of that, I had to learn enough HTML to understand what was going on in the views, and pick up the Javascript and JQuery, since so many of my changes were concentrated on the views – also new to me, since I usually stick to back-end development. Overall, I’ve become a huge fan of Rails, and particularly Ruby. I found this project gave a great understanding of how (and why) MVC systems work the way they do, and really came to appreciate how easy it was to find what needed to be changed when I went looking – you could tell just by looking at if the issue was with data, behaviour, or appearance. Ruby has become one of my favourite languages, barring a few quirks which can lead to consistency issues (you can use begin/end or {/], your boolean logics can be !/&&/|| or not/and/or, and some other similar problems). However, this project was able to mitigate these problems with Hound, a program that hounds you if you try to submit changes that don’t follow the consistency standards. This was a great term for broadening my horizons and getting to experience web development, which I don’t really see taught at SFU.

Written by Christian Millar

April 16th, 2015 at 1:01 pm

Posted in Uncategorized

## RSpec testing in MarkUs II

MarkUs is the robust and reliable application where testing is used as the main component to make sure those qualities persist. I knew nothing about testing before starting UCOSP this term in January of 2015. Besides the fact that testing is very important for the known reasons (Duh). When I began learning RSpec I realized the philosophy behind writing robust code: consider all the possibilities of how your code can be wrecked before writing it. Are you writing a form with a field prompting for user’s date of birth? Think about dates like 99/99/9999 or javascript code that might end up in that field. Crazy stuff happens, so it’s faulty to write code only assuming the best-case scenario where user actions are always coherent with the functionality implemented.

The Behavior Driven Development (BDD) seems very logical and even optimal for applications like MarkUs. But adopting it means that all the developers should have in-depth knowledge of how testing works. It’s seems unlikely to happen due to the distributive nature of the application and the fact that many people who work on it are coming and going just like me. After spending my term on MarkUs working with RSpec I can say for sure that I’m not even close at becoming pro at testing. So forget about BDD and let’s see what I’ve done over this term.

The tickets I was working on mainly concentrated around different sorts of testing. I started with the easiest kind that involved unit testing where you test a single function to make sure it produces appropriate results using arbitrary scenarios. Sometimes it’s difficult to come up with a good scenario. You have to think about what function supposed to do and test that, but then you also have to consider wild (not too wild) possibilities that could break the function. Writing passing a test is easy, but most of the time I was only satisfied when I was able to break something that resulted in failing test and required fixing.

That happened when I began working on CSV upload functionality for Simple Grade Entry form. The upload relies on rails library parsing methods that are not ideally robust to begin with. So it’s very important that the file passes all the checks before entering the parsing stage. My wild (but not too wild) failing scenarios included checking for: presence of EOF character, appropriate EOL characters, non-CSV files that are pretending they are CSV (sneaky spies!), etc. I also discovered that any file passed through CSV upload would overwrite the original template. Even though this functionality is required for other parts of MarkUs, I thought that it could potentially cause some troubles along the way. Now this concern is handled and corresponding RSpec tests are in place.

RSpec is a powerful testing framework that possesses both great readability and writability aspects. Although it’s not good enough on its own for html parsing (requires Nokogiri), and javascript parsing (requires Capybara/Selenium pair which is not a stable solution for vagrant box developers).  I also found out that RSpec is not multi-thread-friendly and because of that I was unable to finish the last issue that required testing forked processes. Nevertheless, there are currently efforts made by the RSpec core team and, hopefully, we will soon get the capability of testing parallel processes.

I greatly enjoyed working with this test framework and its object-building sister – FactoryGirl. The biggest challenge I was facing is understanding of sometimes complex relationships between different MarkUs entities. And in-depth understanding is required, because there is no way to even start testing without a good idea of how different elements co-operate with each other. And then you start building factories, which have to follow the rules of multiple associations, validation and other model requirements. Testing helps a developer to get almost intimate with the code, so I believe that this term I became not just a better coder, but improved as a programmer.

Written by Maryna Moskalenko

April 10th, 2015 at 8:22 pm

Posted in Uncategorized

## Integrating MathJax

A feature commonly requested by markers was the ability to annotate student submissions with LaTeX like math. This has now been implemented!

Anywhere annotations exist markers can enter LaTeX like math thanks to MathJax. When creating an annotation the marker just needs to surround the math in a pair of dollar-sign symbols. The entire annotation shown in the image above is:

should actually be $$\lt \frac 1 2$$

As would be expected, students also see LaTeX like math when viewing an annotation with math in it. This math rendering system expands (and reinforces) the utility of Markus for theory-based courses such as calculus, discrete math or linear algebra.

In addition to the MathJax integration, markers can now preview their annotation prior to submitting it.

The preview functionality will help speedup the marking process for users unfamiliar with MathJax syntax. Instead of submitting an annotation, hovering over it and then editing it, markers can now preview exactly how their annotation will look once submitted. This also allows markers to preview rich HTML annotations with bolded and italicized text.

April 10th, 2015 at 12:54 pm

Posted in Uncategorized

## Status Updates – March 27

Peter

• This week
• Git cleanup of spreadsheets branch
• Began work on CSV override marks option
• Next week
• Continue work on CSV override marks option

Paymahn

• This week
• Started working on annotation previews
• Fixed issue 1727
• Next week
• Continue work on annotation previews

Maryna

• This week
• wrapping up with file upload testing.
• All tests are passing except one:
• There is an error thrown when an uploaded file has no newline at the end of file or non-standard newline
• Worked on displaying the error message when a user tries to upload the file that would change Simple Grade Template. More information: #2098
• Next week
• finish up 2078/2079 and start 1199

Christian

• This week
• Changed the seed so A2 has a flexible scheme and some submissions. Also, discovered an interesting bug: if you change an assignment that isn’t due yet to a due date in the past, collecting submissions breaks. Completed issue 2030. Found that reordering works on assignment criteria, but was still broken on annotation categories, so I rejiggered things to work there
• Next week
• The only issue currently assigned to me is 2087, so that’s on the list.
• I’ll also be asking for some more tickets during the meeting.

Ryan

• This week:
• Cleaned up the changes I made last week, included fixing url generation
• Removed some of the unused RJS files
• Converted some RJS files to erb from the graders page
• Issues
• Got stuck debugging why jQuery wasnt working… needed to use jQuery() not \$()
• Took a while to ensure I wasnt removing any functionality
• Next week
• Fix up git issue
• Continue converting RJS to erb files

March 27th, 2015 at 11:34 am

Posted in Uncategorized

## Status Update – Feb 27

Paymahn

• This week
• started writing UTs for MathJax integration. Started having difficulty testing well, took a step back and realized the integration could be significantly simplified.
• Simplified MathJax integration (deleted lots of code)
• Finished mathjax integration and put up a PR
• Next week
• Unknown. Either pick up a new project or fix any bugs found while checking the PR

Christian

• This week
• Wrapped up the last issues with the remarks (adding an extra mark not working and deleting database records for extra marks when they aren’t needed).
• Started creating a modal that’ll popup when you’re close to expiry (currently thinking it’ll popup, say “You’re about to expire; press here to cancel that”, something like that). Currently got the javascript that’ll check every minute or so hooked up, and figured out how we’re tracking the session timeout.
• Next week
• More of the modal. If it turns out way easier than I think it’ll be, I’ll be able to start work on the double login bug too.

Ryan

• This week
• Searched though react docs to find way to hide a column for hours, was unsuccessful
• Figured out how to get data from the conroller to the view with rails
• Searched for all the section columns and set them to hide if there are no sections set
• Started to investigate ajax update issue with the marks_graders tables
• Next week
• Try to find a solution to the ajax issue, ensure react is implemented correctly on the marks_graders page
• Issues
• There doesnt seem to be much documentation on how to use the react tables so I spent a lot of time trying to figure them out
• This was my first time using the rails MVC style so there was learning curve there
• Finding all the section columns wasnt as simple as I had hoped

Maryna

• This week
• worked on issue 664: Add tests for our routes
• Re-writing the test from this outdated patch: https://gist.github.com/benjaminvialle/4055208
• Completed:
‘Routing to main page’
‘An Assignment collection’
‘An Assignment member’
‘An Assignment’s rubric’
‘An Assignment’s flexible criteria’
‘An Assignment’s automated test’
• Next week
• continue working on tests for csv upload and routes tests
• Issues
• as the routes.rb file got changed over the year, some routes became outdated and new routes were added. Have to trace the changes to write new tests and remove outdated

Peter

• This week
• Completed the implementation of the Grade Entry Table using React
• Search, Sorting, and Selecting all working now
• Next week
• Work on Issue 1762: “total_grade” should be a calculated field in the database.

Josh

• This week
• NA
• Next week
• NA

February 26th, 2015 at 10:11 pm

Posted in Uncategorized

## Status Update – Feb 20

Paymahn

• This week
• Did manual testing of annotation system with MathJax. Found several bugs specific to my work (PDFs and something else I forget) and several on development
• Opened issues: 2045, 2043 and 2042
• Next week
• Unit testing, finally

Maryna

• This week
• Finished issue 218: adding tests for “calculate_collection_time” and “calculate_grouping_collection_time”
‘expects a call to send_data’
‘sets disposition as attachment’
‘returns vnd.ms-excel type’
‘passes naming conventions’
• Upload is much more complicated. Here are the tests that I have started:
‘does not accept an empty file’
‘should gracefully fail on non-csv file with .csv extension’
‘does not accept .csv file with wrong column names’
• Next week
• Right now if we try uploading a non-csv file with csv extension – the application crashes. To catch this exception functional testing must be implemented. I would also have to think about other testing for the upload
• Problems
• every time no matter if the file is correct or not, it gets redirected
• all calls to this upload function return status 302, go somewhere else and respond with the appropriate status code. Unit testing cannot trace such response

Christian

• This week
• Refactoring in and around the remarks and results screen, namely changing how (and when) remark results are generated
• reworking the views in the results screen, since they shouldn’t be checking the user role if they’re in a role-specific folder.
• Started the add extra mark issue, which’ll likely be fast
• Next week
• I’m going to need to pick a new area and set of issues to work on, which I’ll be bringing up in this week’s meeting

Ryan

• This week
• Investigated annotation bugs
• Applied and thoroughly tested the fixes, found a few hidden issues that were already exisiting and fixed them as well
• Next week
• Take a big stab at the react stuff! I hope to put in double time next week to make up for this week
• Issues
• Hound
• I didn’t have a lot of time to put in this week (Other classes, V-Day, birthdays, huge hassle getting my washing machine replaced, etc etc)

Josh

• This week
• NA
• Next week
• NA

Peter

• This week
• Updated the React table input textboxes for grades to dynamically save
• Next week
• Continue working on and debugging the spreadsheets table implementation using React
• Potentially start on other issues currently assigned to me
• Issues
• There are still issues with the sorting, checkboxing, and searching that I need to look into
• Notes
• I was away travelling for most of this week as it was reading break at UBC, so I wasn’t able to get as much work done.