MarkUs Blog

MarkUs Developers Blog About Their Project

Archive for November, 2014

Status Report – November 28

without comments

Jakub Subczynski

This Week

  • Refactored Assignment spec:
    • grouped certain tests (i.e. association and validation tests)
    • replaced `.to be` and `.not_to be` with `.to be true` and `.to be false` when expecting a boolean value
    • other small improvements
  • RSpec tests for:
    • Assignment#section_due_date (and rewrote actual method)
    • Assignment#past_due_date?
    • Assignment#latest_due_date
    • Assignment#section_past_due_date? (and modified actual method)
    • Assignment#what_past_due_date
  • Started putting RSpec blog post together

Next Week

  • Complete blog post and
  • You guessed it — more tests!

 

Chris

This Week

  • Completed drawing/displaying annotations.
  • Completed saving annotations.
  • Worked on loading existing annotations.

Next Week

  • Complete loading existing annotations.
  • Update/Write blog post on the status of annotations and how they work.

 

Yusi

This Week

  • Worked on TODOs in the comment that is related to rails 4
  • Change:
    find_all_by_… to where(…),
    find_or_create_by_ to find_or_create_by,
    find_last_by_ to where(…).last
  • Manual tests

Next Week

  • Write a blog post
  • I found some errors caused by the new “all()” api today, I will update the code related to it.
  • Manual tests and discover more errors

Written by Jakub Subczynski

November 28th, 2014 at 2:04 pm

Custom Sort Functions for React Tables

with one comment

Recently, we discovered that some columns in the new React.js Tables were not sorting the rows as expected. In particular, the GROUP NAME and REPOSITORY columns in the Submissions and Summaries views exhibited consistent but very strange behaviour, as shown in the figures below.

sort_by_group_name

sort_by_repo_name

(The COMMIT DATE column was also sorting incorrectly, but this issue was less mysterious in that it was obviously performing string comparison on the dates. In order to correctly parse the date strings into a numerically comparable value, a customized sorting method was definitely required.) I was assigned the task of refactoring the React Table class (and related classes) in order to allow customized sorting methods for columns that required special treatment.

As it was, sorting was handled by sort_by_column(). The cell values we converted to lowercase and used the dangerouslySetInnerHTML.__html property for React Components. Values of types other than string were not interpreted.

function sort_by_column(data, column, direction) {
  function makeSortable(a)
  {
    if (typeof a == 'string') {
      return a.toLowerCase().replace(' ', '');
    } else if (a.hasOwnProperty('props')) {
      // Is a react component, get innerHTML
      return a.props.dangerouslySetInnerHTML.__html.toLowerCase();
    } else {
      return a;
    }
  }
  // sorts column id
  var r = data.sort(function(a, b) {
    if (makeSortable(a[column]) > makeSortable(b[column])) {
      return 1;
    } else if (makeSortable(b[column]) > makeSortable(a[column])) {
      return -1;
    }
    return 0;
  });
  // flips order if direction descending.
  if (direction == 'desc') r.reverse();

  return r;
}

 

table.js ]

In some table columns, we use a React <span> components with dangerouslySetInnerHTML.__html set in order to properly render html contents such as links and icons in a cell. (Though not shown above, the cells in the COMMIT DATE column will contain an icon if the submission commit date is past due.)

Of course, when we attempt to sort by dangerouslySetInnerHTML.__html, we are comparing strings of html code rather than the text displayed in the table cells. Naturally, the unexpected behaviours exhibited by the GROUP_NAME and REPOSITORY columns were the result of string comparison between the hrefs for the anchor elements, rather than the hyperlink text. Since both links differed by group number, we were effectively performing an alphanumeric sort on the unformatted group numbers.

For example, group_0007 would appear below group_0011 on an ascending sort because we were comparing “11” and “7” rather than the repository names. In the first figure, the group names did not appear to follow any kind of logical order because the display text is completely unrelated to group number.

My solution was to leave the existing function makeComparable() intact, for  primary processing of sortable data, while optionally allowing a custom compare function to be passed in as needed for any particular column.

function sort_by_column(data, column, direction, compare) {
  //...
  // determine sort behaviour
  compare = compare || compare_values;

  // sort row by column id
  var sorted = data.sort(function(a, b) {
    return compare(makeComparable(a[column]), makeComparable(b[column]));
  });

  // flip order if direction descending
  if (direction == 'desc') {
      sorted.reverse();
  }

  return sorted;
}

table.js ]

The compare function to use when the table rows are rendered is stored in the Table state variable sort_compare. When the selected sort column changes sort_compare is updated to the compare member of the appropriate column.

// Header col was clicked. Adjust state accordingly.
synchronizeHeaderColumn: function(sort_column, sort_direction) {
  var compare_func = this.props.columns.filter(function(col) {
      return col.id == sort_column;
  })[0].compare;
  this.setState({
    sort_column: sort_column,
    sort_direction: sort_direction,
    sort_compare: compare_func
  });
}

table.js ]

Since we have the default comparison function compare_values, we are able to leave compare undefined for any given column.

function compare_values(a, b) {
  if (!b || a > b) {
    return 1;
  } else if (!a || a < b) {
    return -1;
  }
  return 0;
}

table_sorts.js ]

Now we define functions such as compare_anchor_text() to handle special cases, and assign the custom sort functions to column objects. In the code below, notice that the SECTION column does not define a compare member since the data is just a string and so the default compare_values will suffice.

function compare_anchor_text(a, b) {
  function parse_anchor(a) {
    var open_tag_end = a.indexOf('>', a.indexOf('<a'));
    var close_tag_start = a.indexOf('</a', open_tag_end + 1);
    if (open_tag_end !== -1 && close_tag_start !== -1) {
      return a.substring(open_tag_end + 1, close_tag_start);
    }
    return a;
  }

  return compare_values(parse_anchor(a), parse_anchor(b));
}

table_sorts.js ]

var SubmissionsTable = React.createClass({
  getDefaultProps: function() {
    // Defines the columns used for the table and whether they
    // are sortable searchable. The default initially sorted
    // column is the first sortable column in the array.
    return {
      columns: [
        {
          id: 'group_name',
          content: '<%= j raw I18n.t(:'browse_submissions.group_name') %>',
          sortable: true,
          compare: compare_anchor_text,
          searchable: true
        },
        {
          id: 'repository',
          content: '<%= j raw I18n.t(:'browse_submissions.repository') %>',
          sortable: true,
          compare: compare_anchor_text,
          searchable: true
        },
        {
          id: 'section',
          content: '<%= j raw I18n.t(:'browse_submissions.section') %>',
          sortable: true,
          searchable: true
        },
//...

_submissions_table.js.jsx.erb ]

Written by Victoria Verlysdonk

November 25th, 2014 at 10:51 pm

Posted in Uncategorized

Status Report – November 25

without comments

Irene

Last Week

  • Fixed Ruby development environment after it was broken from revert merge from master
  • Learning how to write tests for class methods, confused about how to create the test data. Worked on rspec tests and updating rake test for new method in Assignments model
  • Manual testing on dashboard

This Week

  • Finish rspec tests and make pull request
  • Add spreadsheets summary to dashboard

Nathan

Last Week

  • Opened PR with changes to associate grouping and tags upon tag creation (in one specific workflow), moved methods to a TagsHelper module to allow access from different controllers, set up a variable in the resultscontroller to display tags not associated to current grouping
  • Manual testing to show the association created between tags and groupings work
  • reading and research into routes.rb, still very confused!

This Week

  • continued work on the submissions table and the results view.

 

Tori

Last Week

  • new compare functions for React tables custom sorting
    • some time spent doing it the wrong way (passing extra data to the view) and then discussing/clarifying the right way (relevant commit; other work was not pushed to github before I changed it).
  • writing up work on sorting for blog

This Week

  • add custom sorts where needed in other tables (?)
  • look into filtering problems

 

Bryan

Last Week

  • Added features for the Tag view. These include edit modals, tag delete buttons, create new modal, download yml.
  • Worked on the routes file. Created routes to allow for the functions defined above. There are still issues with the routes table and certain things aren’t working properly.
  • Added some features to the controller to help assist with CSV upload and download, YML download, tag creation, etc.

Next Week

  • Bug fixes for the tagging features.
  • Start on the tagging features on the Submissions view.
  • Fixed routes.rb so that routes are defined for the tagging features.

Written by Victoria Verlysdonk

November 25th, 2014 at 1:14 pm

Status Report – November 21

without comments

Chris

This Week:

  • Continued work on the drawing system for annotations in the PDF view. Originally I was trying to use as much of the existing image annotation system as possible, unfortunately I have discovered that the way they work are too different and I am not creating a specific annotation manager for the PDF’s.
  • Read through a ton of Javascript and HTML to decipher how the current annotation drawings are done on images.

Next Week:

  • Finish the drawing, saving, and loading of annotations on the PDF.
  • Clean up as much code as possible before the final week.

 

Jakub Subczynski

This Week

  • Migrated testing of the following methods: Assignment#assigned_groups, Assignment#unassigned_groups and Assignment#add_group
  • Updated rspec matchers that checked if two arrays were equal from eq() to match_array() so that order is not considered. This caused intermittent failing when the order of the retrieved records varied with the local array. Order does not matter in the modified cases.
  • Improved readability and slightly optimized Assignment#add_group method. Both tests suites still pass with my modifications.

Next Week

  • More RSpec testing/migrations
  • Publish blog post: “RSpec tips for Markus”

Yusi

This week:

  • Updated and removed some gems
  • Upgraded rails from 4.0 to 4.1
  • Fix errors caused by the upgrade

Next week:

  • Merge master to rails4_new branch

Written by Chris Kellendonk

November 21st, 2014 at 3:58 pm

Status Report Nov 18th

without comments

Nathan

Last Week

  • Modified existing create function to handle adding a tag from tag view and from individual submission view, (individual submission view will also associate the created tag with the grouping right away)
  • Add operation to grab tags not associated with the provided grouping

Next Week

  • Test workflows and fix bugs encountered.

Irene

Last Week

  • Modified assignments model, main controller and main index view to display the most ‘recent’ assignment upon load. I mistakenly merged master into my dashboard branch and David asked me to undo the merge and make a new pull request. During that process, however, I messed up my development environment (getting errors) so I am working on fixing that.

Next Week

  • Add spreadsheets to dashboard
  • Rearrange elements based on wireframe design

 

Tori

This Week

  • Pull Request #1878 (last week’s accomplishments)
  • Opened Issue #1877
  • Pull Request #1892: removed old references to populate_repo_browser
    • still addressing some HoundCi comments on that
  • added capability for custom sort functions on React Table columns

Next Week

  • fix more tables that don’t sort correctly
  • possible write-up on sort behaviour… it wasn’t actually a hard fix and I’m not yet sure if there is enough info there for a ‘significant’ piece of writing

 

Bryan

This Week:

  • Modified the Tag controller to allow for tags to be created and for tags to be uploaded.
  • Got the React table in the Tags view to display tags that are stored in the ActiveRecord. Also allowed for users to create tags using the Create New Tag dialog.
  • Started work on the Individual Submission page by adding in code to display tags that were assigned to a particular submission. Also started working on the ability for users to tag submissions with a particular tag.

Next Week:

  • Completion of the Tag view. The React table will have the “Edit” and “Delete” links working as well as the CSV upload and download dialog boxes working.
  • The individual submissions view will be fully complete. Users will be able to assign submissions with tags, remove assignments of tags and view the top used tags in the system.

Written by Nathan Chow

November 18th, 2014 at 2:06 pm

Posted in Status Reports

Status Summary – November 14, 2014

without comments

Jakub Subczynski

This Week

  • Pull Requests:
    • Removal of unused Assignment#no_grouping_students_list method.
    • Renaming of Assignment#graded_submissions method to ‘graded_submission_results’ (as it returns a set of Results)
    • Test migration/improvement of 2 Assignment methods #groups_submitted and #add_csv_group

Next Week

  • Continue migration/improvement of Assignment model tests

Yusi

This Week

  • Added strong parameters of assignment model and update helper file
  • Updated some test cases according to the new rails API
  • Did some manual tests under admin

Next Week

  • Continue working on manual tests

Irene

This Week

  • Merge pull request #1857 (Split view on dashboard)
  • Working on the assignment summary view (according to the wireframe)

Next Week

  • Submit pull request for the revamped assignment summary view
  • Implement logic to display the most “current” assignment

Chris

This Week:

  • Continued work on integrating the annotation system.
  • Removed unnecessary files from pdf.js and optimize the viewer loading time

Roadblocks:

  • Tried to use the current annotation system for the pdf’s. I think it is incompatible with the needs for creating annotations on a pdf. I am still trying to use the existing code for selection of areas to draw annotations and then implement a annotation rendering part on top of that.

Next Week:

  • Finish javascript side of creating annotations.
  • Begin working on the server side saving of annotations.

Written by Jakub Subczynski

November 14th, 2014 at 3:51 pm

Status Report – November 11th

without comments

Bryan

This Week:

  • Coordinated with Nathan regarding the Tagging feature back-end. Started looking into linking the views with the controller.
  • Cleaned up view code and continued to add in features that didn’t require the Tagging controller.

Next Week:

  • Linking the Tagging controller with the views.
  • Have a working prototype of the Tagging feature that involves the simple creation and deletion of a tag.

 

Tori

This Week:

  • Sorting repo files table by revised-by or last-modified no longer breaks the table
  • Current path breadcrumb no longer breaks when assignment repository  root folder is selected. It was appending the assignment repository folder name repeatedly (e.g. /A1/A1 each time you selected /A1).


Next Week:

  • Sort last-modified dates as dates instead of strings
  • Fix revision selection
  • File Manager view


Issues:

  • I think I have a better understanding of the revision selection problem as described here. I spent a while investigating this. I thought perhaps there was an off-by-one error with the controller using 0-based revisions and the view using 1-based, but now I think revision 0 is just a weird case that *does* exist, but isn’t actually accessible.

Written by Bryan Muscedere

November 13th, 2014 at 11:22 am

Posted in Status Reports

Status Report – November 7th

without comments

Chris

This Week:

  • Worked on the text drawing annotations. Currently I am trying to use interface the image annotation system with the pdf viewer so that the same code can be used in both places.
  • Removed unnecessary code in the viewer and added more optimizations.
  • Fixed more display and css bugs.

Next Week:

  • Continue work on the annotation system.
  • Save annotations from the pdf to the database.

Yusi

This Week:

Fixed:
test/functional/grade_entry_forms_controller_test.rb (20 errors)
test/functional/groups_controller_test.rb (1 error)
test/functional/marks_graders_controller_test.rb (3 errors)
test/functional/notes_controller_test.rb (18 errors)
test/functional/main_controller_test.rb (12 errors)
test/functional/role_switching_test.rb (2 failures)
test/functional/groups_controller_csv_upload_test.rb (2 failures)
test/functional/submissions_controller_test.rb (1 warning)
test/functional/results_controller_test.rb (4 errors)

Next Week:

TODO:
test/functional/automated_tests_controller_test.rb (13 tests, 12 assertions, 5 failures, 2 errors, 0 skips)
test/unit/helpers/ensure_config_helper_test.rb (9 tests, 5 assertions, 0 failures, 1 errors, 0 skips)
test/unit/markus_logger_test.rb (20 tests, 20 assertions, 1 failures, 0 errors, 0 skips)
test/unit/submission_collector_test.rb (16 tests, 50 assertions, 1 failures, 0 errors, 0 skips)
spec/models/grade_entry_student_spec.rb (12 examples, 1 failure)
spec/controllers/groups_controller_spec.rb (16 examples, 3 failures)

Jakub Subczynski

This Week

  • Migrated more rspec tests. Assignment rake tests down to 800 lines
  • Identified codebase improvements consisting of removing an unused method and renaming a method
  • Faced issue creating an rspec test to test Assignment#graded_submissions. I did not realize I needed to set `version_used_submission` for Submission in order for it to be picked up as a valid submission so I kept on receiving zero submissions To solve, I removed FactoryGirl and created all of the necessary records manually, using data from development. When that worked, I started removing attributes and swapping in FactoryGirl for each record to narrow down the source of the issue.

Next Week

  • Create more tests for Assignment methods and remove rake tests that relate the to method
  • Remove invalid_override from Markus
  • Remove unused Assignment#submissions_by(user)

Written by Chris Kellendonk

November 7th, 2014 at 3:40 pm

Status Report – November 4

without comments

Tori

This Week:

  • Figured out how to add to student repos with svn
  • Added subdirectories and previous directory link to repo files table
  • Fixed current path/breadcrumbs display (was showing html instead of actual links)
  • Removed some unneeded code/files

Next Week:

  • Fix sorting in repo files table (probably need a custom sort after all; will most likely refactor table.js to make custom sorting definitions work similar to custom filers, as Lawrence Wu suggests)
  • Fix revision selection
  • Start on File Manager table conversion

Roadblocks:

  • Some time spent fixing svn ruby bindings on my desktop
  • Some time spent looking at how the db is seeded. I don’t really get it.
  • Sorting by revised_by and last_modified don’t work. I didn’t notice before modifying the repo since all the files had the same values for those columns.
  • Selecting a revision by number or with the date picker don’t appear to update the files table. With some caveman debugging, I can see that @revision_number is changed temporarily, but seems to go back to the most recent revision after the update pdfs script is used. The page says it’s displaying revision N(timestamp of N) but the table data is still still for the most recent revision every time.
  • Spent a considerable amount of time figuring out the flow of control in the existing FilterTable; it seemed to bounce around between the view and controller a lot. The new flow models the submissions table more, but I wonder if the old flow had something to do with why the revision selection and sorting aren’t updating appropriately.
  • It took some doing to get the file download links to appear, just because I’m not a ruby expert and I wasn’t sure where the links should be formed (it’s in the controller now). Also, I was using the old *_table_row.html.erb for reference and accidentally copy/pasta’d the directory row link info from file manager instead of using the repo browser action.

Nathan
Last Week

  • Open a pull request with the following changes
    • change to tags migration file to capture name and description content
    • addition of assignment_tags migration file to capture many to many association
      • this change came up looking at Bryan’s UI, may need to sync with him if we actually want this
    • when submissions table is loaded, tags for each grouping is being pulled now (not shown in UI yet)
    • Add tags functionality is the tag view and in the individual submission view should create a tag now associated with the assignment, not yet associated to the individual submission/grouping
    • a few other tag operations in tag_controller I anticipate will support other tagging features.
  • created a google doc with Bryan to go sync over what’s needed for each tagging operation

This Week

  • continue going through the list of features and implement the backend operations needed
  • also I want to revisit some of the operations i wrote this week, I think I may need to use params[] to get the to work the way I’m anticipating they’ll be used.  this is a part of ruby im a little hazy about.

Bryan
This Week:

  • Created a blog post to outline all the new view changes in MarkUs. Looking for feedback from other users.
  • Collaborated with Nathan regarding our next steps in connecting the views with the controller.
  • Continued to update the views for the Tags. Added in any features that do not require the controller (ie, word counter, moving tags from assigned to not assigned list, etc).

Next Week:

  • Finalize the views and start working on linking the models and controller to the views!

Irene
Last week:

  • Still trying to implement the functionality for list of assignments to change the assignment details on the right side. Spent 10+ hours on the controller/routing issue but can’t seem to pass the assignment variable. Getting the ‘undefined method `short_identifier’ for nil:NilClass’ error. David is trying to help me but I am still unable to fix the issue at the moment.

Next week:

  • Hopefully move on to implementing the logic to display the ‘current’ assignment.

Written by Victoria Verlysdonk

November 4th, 2014 at 5:34 pm