MarkUs Blog

MarkUs Developers Blog About Their Project

Custom Sort Functions for React Tables

without comments

Recently, we discovered that some columns in the new React.js Tables were not sorting the rows as expected. In particular, the GROUP NAME and REPOSITORY columns in the Submissions and Summaries views exhibited consistent but very strange behaviour, as shown in the figures below.

sort_by_group_name

sort_by_repo_name

(The COMMIT DATE column was also sorting incorrectly, but this issue was less mysterious in that it was obviously performing string comparison on the dates. In order to correctly parse the date strings into a numerically comparable value, a customized sorting method was definitely required.) I was assigned the task of refactoring the React Table class (and related classes) in order to allow customized sorting methods for columns that required special treatment.

As it was, sorting was handled by sort_by_column(). The cell values we converted to lowercase and used the dangerouslySetInnerHTML.__html property for React Components. Values of types other than string were not interpreted.

function sort_by_column(data, column, direction) {
  function makeSortable(a)
  {
    if (typeof a == 'string') {
      return a.toLowerCase().replace(' ', '');
    } else if (a.hasOwnProperty('props')) {
      // Is a react component, get innerHTML
      return a.props.dangerouslySetInnerHTML.__html.toLowerCase();
    } else {
      return a;
    }
  }
  // sorts column id
  var r = data.sort(function(a, b) {
    if (makeSortable(a[column]) > makeSortable(b[column])) {
      return 1;
    } else if (makeSortable(b[column]) > makeSortable(a[column])) {
      return -1;
    }
    return 0;
  });
  // flips order if direction descending.
  if (direction == 'desc') r.reverse();

  return r;
}

 

table.js ]

In some table columns, we use a React <span> components with dangerouslySetInnerHTML.__html set in order to properly render html contents such as links and icons in a cell. (Though not shown above, the cells in the COMMIT DATE column will contain an icon if the submission commit date is past due.)

Of course, when we attempt to sort by dangerouslySetInnerHTML.__html, we are comparing strings of html code rather than the text displayed in the table cells. Naturally, the unexpected behaviours exhibited by the GROUP_NAME and REPOSITORY columns were the result of string comparison between the hrefs for the anchor elements, rather than the hyperlink text. Since both links differed by group number, we were effectively performing an alphanumeric sort on the unformatted group numbers.

For example, group_0007 would appear below group_0011 on an ascending sort because we were comparing “11″ and “7″ rather than the repository names. In the first figure, the group names did not appear to follow any kind of logical order because the display text is completely unrelated to group number.

My solution was to leave the existing function makeComparable() intact, for the purpose of providing a default sort behaviour, while optionally allowing a custom compare function to be passed in as needed for any particular column.

function sort_by_column(data, column, direction, compare) {
  //...
  compare = compare || function(a, b) {
    if (makeComparable(a[column]) > makeComparable(b[column])) {
        return 1;
    } else if (makeComparable(a[column]) < makeComparable(b[column])) {
        return -1;
    }
    return 0;
  }

  // sort row by column id
  var sorted = data.sort(compare);
  //...
}

 

table.js ]

The compare function to use when the table rows are rendered is stored in the Table state variable sort_compare. When the selected sort column changes sort_compare is updated to the compare member of the appropriate column. Since we have the default comparison function utilising makeComparable(), we are able to leave compare undefined for any given column.

// Header col was clicked. Adjust state accordingly.
synchronizeHeaderColumn: function(sort_column, sort_direction) {
  var compare_func = this.props.columns.filter(function(col) {
      return col.id == sort_column;
  })[0].compare;
  this.setState({
    sort_column: sort_column,
    sort_direction: sort_direction,
    sort_compare: compare_func
  });
}

table.js ]

Once in place, I was able to define functions such as compare_group_names() to handle special cases, and assign the custom sort functions to column objects. In the code below, notice that the SECTION column does not define a compare member since the data for this column will not contain html code and so the default sort-by-string method will suffice.

function parse_anchor(a) {
  var open_tag_end = a.indexOf('>', a.indexOf('<a'));
  var close_tag_start = a.indexOf('<', open_tag_end + 1);
  return a.substring(open_tag_end + 1, close_tag_start);
}
function compare_group_names(a, b) {
  var a_parsed = parse_anchor(
      a['group_name'].props.dangerouslySetInnerHTML.__html.toString());
  var b_parsed = parse_anchor(
      b['group_name'].props.dangerouslySetInnerHTML.__html.toString());
  if (a_parsed > b_parsed) {
    return 1;
  } else if (a_parsed < b_parsed) {
    return -1;
  }
  return 0;
}

 

table_sorts.js ]

var SubmissionsTable = React.createClass({
  getDefaultProps: function() {
    // Defines the columns used for the table and whether they
    // are sortable searchable. The default initially sorted
    // column is the first sortable column in the array.
    return {
      columns: [
        {
          id: 'group_name',
          content: '<%= j raw I18n.t(:'browse_submissions.group_name') %>',
          sortable: true,
          compare: compare_group_names,
          searchable: true
        },
        {
          id: 'repository',
          content: '<%= j raw I18n.t(:'browse_submissions.repository') %>',
          sortable: true,
          compare: compare_repo_names,
          searchable: true
        },
        {
          id: 'section',
          content: '<%= j raw I18n.t(:'browse_submissions.section') %>',
          sortable: true,
          searchable: true
        },
        //...

 

_submissions_table.js.jsx.erb ]

Written by Victoria Verlysdonk

November 25th, 2014 at 10:51 pm

Posted in Uncategorized

Status Report – November 25

without comments

Irene

Last Week

  • Fixed Ruby development environment after it was broken from revert merge from master
  • Learning how to write tests for class methods, confused about how to create the test data. Worked on rspec tests and updating rake test for new method in Assignments model
  • Manual testing on dashboard

This Week

  • Finish rspec tests and make pull request
  • Add spreadsheets summary to dashboard

Nathan

Last Week

  • Opened PR with changes to associate grouping and tags upon tag creation (in one specific workflow), moved methods to a TagsHelper module to allow access from different controllers, set up a variable in the resultscontroller to display tags not associated to current grouping
  • Manual testing to show the association created between tags and groupings work
  • reading and research into routes.rb, still very confused!

This Week

  • continued work on the submissions table and the results view.

 

Tori

Last Week

  • new compare functions for React tables custom sorting
    • some time spent doing it the wrong way (passing extra data to the view) and then discussing/clarifying the right way (relevant commit; other work was not pushed to github before I changed it).
  • writing up work on sorting for blog

This Week

  • add custom sorts where needed in other tables (?)
  • look into filtering problems

 

Bryan

Last Week

  • Added features for the Tag view. These include edit modals, tag delete buttons, create new modal, download yml.
  • Worked on the routes file. Created routes to allow for the functions defined above. There are still issues with the routes table and certain things aren’t working properly.
  • Added some features to the controller to help assist with CSV upload and download, YML download, tag creation, etc.

Next Week

  • Bug fixes for the tagging features.
  • Start on the tagging features on the Submissions view.
  • Fixed routes.rb so that routes are defined for the tagging features.

Written by Victoria Verlysdonk

November 25th, 2014 at 1:14 pm

Status Report – November 21

without comments

Chris

This Week:

  • Continued work on the drawing system for annotations in the PDF view. Originally I was trying to use as much of the existing image annotation system as possible, unfortunately I have discovered that the way they work are too different and I am not creating a specific annotation manager for the PDF’s.
  • Read through a ton of Javascript and HTML to decipher how the current annotation drawings are done on images.

Next Week:

  • Finish the drawing, saving, and loading of annotations on the PDF.
  • Clean up as much code as possible before the final week.

 

Jakub Subczynski

This Week

  • Migrated testing of the following methods: Assignment#assigned_groups, Assignment#unassigned_groups and Assignment#add_group
  • Updated rspec matchers that checked if two arrays were equal from eq() to match_array() so that order is not considered. This caused intermittent failing when the order of the retrieved records varied with the local array. Order does not matter in the modified cases.
  • Improved readability and slightly optimized Assignment#add_group method. Both tests suites still pass with my modifications.

Next Week

  • More RSpec testing/migrations
  • Publish blog post: “RSpec tips for Markus”

Yusi

This week:

  • Updated and removed some gems
  • Upgraded rails from 4.0 to 4.1
  • Fix errors caused by the upgrade

Next week:

  • Merge master to rails4_new branch

Written by Chris Kellendonk

November 21st, 2014 at 3:58 pm

Status Report Nov 18th

without comments

Nathan

Last Week

  • Modified existing create function to handle adding a tag from tag view and from individual submission view, (individual submission view will also associate the created tag with the grouping right away)
  • Add operation to grab tags not associated with the provided grouping

Next Week

  • Test workflows and fix bugs encountered.

Irene

Last Week

  • Modified assignments model, main controller and main index view to display the most ‘recent’ assignment upon load. I mistakenly merged master into my dashboard branch and David asked me to undo the merge and make a new pull request. During that process, however, I messed up my development environment (getting errors) so I am working on fixing that.

Next Week

  • Add spreadsheets to dashboard
  • Rearrange elements based on wireframe design

 

Tori

This Week

  • Pull Request #1878 (last week’s accomplishments)
  • Opened Issue #1877
  • Pull Request #1892: removed old references to populate_repo_browser
    • still addressing some HoundCi comments on that
  • added capability for custom sort functions on React Table columns

Next Week

  • fix more tables that don’t sort correctly
  • possible write-up on sort behaviour… it wasn’t actually a hard fix and I’m not yet sure if there is enough info there for a ‘significant’ piece of writing

 

Bryan

This Week:

  • Modified the Tag controller to allow for tags to be created and for tags to be uploaded.
  • Got the React table in the Tags view to display tags that are stored in the ActiveRecord. Also allowed for users to create tags using the Create New Tag dialog.
  • Started work on the Individual Submission page by adding in code to display tags that were assigned to a particular submission. Also started working on the ability for users to tag submissions with a particular tag.

Next Week:

  • Completion of the Tag view. The React table will have the “Edit” and “Delete” links working as well as the CSV upload and download dialog boxes working.
  • The individual submissions view will be fully complete. Users will be able to assign submissions with tags, remove assignments of tags and view the top used tags in the system.

Written by Nathan Chow

November 18th, 2014 at 2:06 pm

Posted in Status Reports

Status Summary – November 14, 2014

without comments

Jakub Subczynski

This Week

  • Pull Requests:
    • Removal of unused Assignment#no_grouping_students_list method.
    • Renaming of Assignment#graded_submissions method to ‘graded_submission_results’ (as it returns a set of Results)
    • Test migration/improvement of 2 Assignment methods #groups_submitted and #add_csv_group

Next Week

  • Continue migration/improvement of Assignment model tests

Yusi

This Week

  • Added strong parameters of assignment model and update helper file
  • Updated some test cases according to the new rails API
  • Did some manual tests under admin

Next Week

  • Continue working on manual tests

Irene

This Week

  • Merge pull request #1857 (Split view on dashboard)
  • Working on the assignment summary view (according to the wireframe)

Next Week

  • Submit pull request for the revamped assignment summary view
  • Implement logic to display the most “current” assignment

Chris

This Week:

  • Continued work on integrating the annotation system.
  • Removed unnecessary files from pdf.js and optimize the viewer loading time

Roadblocks:

  • Tried to use the current annotation system for the pdf’s. I think it is incompatible with the needs for creating annotations on a pdf. I am still trying to use the existing code for selection of areas to draw annotations and then implement a annotation rendering part on top of that.

Next Week:

  • Finish javascript side of creating annotations.
  • Begin working on the server side saving of annotations.

Written by Jakub Subczynski

November 14th, 2014 at 3:51 pm

Status Report – November 11th

without comments

Bryan

This Week:

  • Coordinated with Nathan regarding the Tagging feature back-end. Started looking into linking the views with the controller.
  • Cleaned up view code and continued to add in features that didn’t require the Tagging controller.

Next Week:

  • Linking the Tagging controller with the views.
  • Have a working prototype of the Tagging feature that involves the simple creation and deletion of a tag.

 

Tori

This Week:

  • Sorting repo files table by revised-by or last-modified no longer breaks the table
  • Current path breadcrumb no longer breaks when assignment repository  root folder is selected. It was appending the assignment repository folder name repeatedly (e.g. /A1/A1 each time you selected /A1).


Next Week:

  • Sort last-modified dates as dates instead of strings
  • Fix revision selection
  • File Manager view


Issues:

  • I think I have a better understanding of the revision selection problem as described here. I spent a while investigating this. I thought perhaps there was an off-by-one error with the controller using 0-based revisions and the view using 1-based, but now I think revision 0 is just a weird case that *does* exist, but isn’t actually accessible.

Written by Bryan Muscedere

November 13th, 2014 at 11:22 am

Posted in Status Reports

Status Report – November 7th

without comments

Chris

This Week:

  • Worked on the text drawing annotations. Currently I am trying to use interface the image annotation system with the pdf viewer so that the same code can be used in both places.
  • Removed unnecessary code in the viewer and added more optimizations.
  • Fixed more display and css bugs.

Next Week:

  • Continue work on the annotation system.
  • Save annotations from the pdf to the database.

Yusi

This Week:

Fixed:
test/functional/grade_entry_forms_controller_test.rb (20 errors)
test/functional/groups_controller_test.rb (1 error)
test/functional/marks_graders_controller_test.rb (3 errors)
test/functional/notes_controller_test.rb (18 errors)
test/functional/main_controller_test.rb (12 errors)
test/functional/role_switching_test.rb (2 failures)
test/functional/groups_controller_csv_upload_test.rb (2 failures)
test/functional/submissions_controller_test.rb (1 warning)
test/functional/results_controller_test.rb (4 errors)

Next Week:

TODO:
test/functional/automated_tests_controller_test.rb (13 tests, 12 assertions, 5 failures, 2 errors, 0 skips)
test/unit/helpers/ensure_config_helper_test.rb (9 tests, 5 assertions, 0 failures, 1 errors, 0 skips)
test/unit/markus_logger_test.rb (20 tests, 20 assertions, 1 failures, 0 errors, 0 skips)
test/unit/submission_collector_test.rb (16 tests, 50 assertions, 1 failures, 0 errors, 0 skips)
spec/models/grade_entry_student_spec.rb (12 examples, 1 failure)
spec/controllers/groups_controller_spec.rb (16 examples, 3 failures)

Jakub Subczynski

This Week

  • Migrated more rspec tests. Assignment rake tests down to 800 lines
  • Identified codebase improvements consisting of removing an unused method and renaming a method
  • Faced issue creating an rspec test to test Assignment#graded_submissions. I did not realize I needed to set `version_used_submission` for Submission in order for it to be picked up as a valid submission so I kept on receiving zero submissions To solve, I removed FactoryGirl and created all of the necessary records manually, using data from development. When that worked, I started removing attributes and swapping in FactoryGirl for each record to narrow down the source of the issue.

Next Week

  • Create more tests for Assignment methods and remove rake tests that relate the to method
  • Remove invalid_override from Markus
  • Remove unused Assignment#submissions_by(user)

Written by Chris Kellendonk

November 7th, 2014 at 3:40 pm

Status Report – November 4

without comments

Tori

This Week:

  • Figured out how to add to student repos with svn
  • Added subdirectories and previous directory link to repo files table
  • Fixed current path/breadcrumbs display (was showing html instead of actual links)
  • Removed some unneeded code/files

Next Week:

  • Fix sorting in repo files table (probably need a custom sort after all; will most likely refactor table.js to make custom sorting definitions work similar to custom filers, as Lawrence Wu suggests)
  • Fix revision selection
  • Start on File Manager table conversion

Roadblocks:

  • Some time spent fixing svn ruby bindings on my desktop
  • Some time spent looking at how the db is seeded. I don’t really get it.
  • Sorting by revised_by and last_modified don’t work. I didn’t notice before modifying the repo since all the files had the same values for those columns.
  • Selecting a revision by number or with the date picker don’t appear to update the files table. With some caveman debugging, I can see that @revision_number is changed temporarily, but seems to go back to the most recent revision after the update pdfs script is used. The page says it’s displaying revision N(timestamp of N) but the table data is still still for the most recent revision every time.
  • Spent a considerable amount of time figuring out the flow of control in the existing FilterTable; it seemed to bounce around between the view and controller a lot. The new flow models the submissions table more, but I wonder if the old flow had something to do with why the revision selection and sorting aren’t updating appropriately.
  • It took some doing to get the file download links to appear, just because I’m not a ruby expert and I wasn’t sure where the links should be formed (it’s in the controller now). Also, I was using the old *_table_row.html.erb for reference and accidentally copy/pasta’d the directory row link info from file manager instead of using the repo browser action.

Nathan
Last Week

  • Open a pull request with the following changes
    • change to tags migration file to capture name and description content
    • addition of assignment_tags migration file to capture many to many association
      • this change came up looking at Bryan’s UI, may need to sync with him if we actually want this
    • when submissions table is loaded, tags for each grouping is being pulled now (not shown in UI yet)
    • Add tags functionality is the tag view and in the individual submission view should create a tag now associated with the assignment, not yet associated to the individual submission/grouping
    • a few other tag operations in tag_controller I anticipate will support other tagging features.
  • created a google doc with Bryan to go sync over what’s needed for each tagging operation

This Week

  • continue going through the list of features and implement the backend operations needed
  • also I want to revisit some of the operations i wrote this week, I think I may need to use params[] to get the to work the way I’m anticipating they’ll be used.  this is a part of ruby im a little hazy about.

Bryan
This Week:

  • Created a blog post to outline all the new view changes in MarkUs. Looking for feedback from other users.
  • Collaborated with Nathan regarding our next steps in connecting the views with the controller.
  • Continued to update the views for the Tags. Added in any features that do not require the controller (ie, word counter, moving tags from assigned to not assigned list, etc).

Next Week:

  • Finalize the views and start working on linking the models and controller to the views!

Irene
Last week:

  • Still trying to implement the functionality for list of assignments to change the assignment details on the right side. Spent 10+ hours on the controller/routing issue but can’t seem to pass the assignment variable. Getting the ‘undefined method `short_identifier’ for nil:NilClass’ error. David is trying to help me but I am still unable to fix the issue at the moment.

Next week:

  • Hopefully move on to implementing the logic to display the ‘current’ assignment.

Written by Victoria Verlysdonk

November 4th, 2014 at 5:34 pm

Status Report – October 31

without comments

Chris

This Week:

  • Completed initial prototype of PDF viewer.
  • Learned how the asset pipeline works so that all the files needed for the viewer can be compiled as one.

Next Week:

  • Fix bug where the viewer javascript is loaded for each new pdf request.
  • Begin adding annotations.

 

 Jakub Subczynski

This Week:

  • Rewrote Assignment#group_assignment?
  • Increased code coverage by creating new test
  • Migrated some rake tests over

Next Week:

  • Continue test migration

Roadblocks:

  • Spent considerable amount of time troubleshooting a failing test I wrote

Yusi

This Week:

  • Ran test one by one /test/unit directory (model tests) and fixed test errors

Next Week:

  • Fix test errors under /test/functional directory (controller)

Errors:

test/functional/automated_tests_controller_test.rb (13 tests, 12 assertions, 5 failures, 2 errors, 0 skips)
test/functional/grade_entry_forms_controller_test.rb (20 errors)
test/functional/groups_controller_csv_upload_test.rb (2 failures)
test/functional/groups_controller_test.rb (1 error)
test/functional/main_controller_test.rb (12 errors)
test/functional/marks_graders_controller_test.rb (3 errors)
test/functional/notes_controller_test.rb (18 errors)
test/functional/results_controller_test.rb (4 errors)
test/functional/role_switching_test.rb (2 failures)
test/functional/submissions_controller_test.rb (1 warning)

Written by Chris Kellendonk

October 31st, 2014 at 2:29 pm

MarkUs Tagging Feature – First Prototype

without comments

After several weeks, the first prototype of the tagging feature has been implemented. While the views for this feature are not final, this gives an idea of what the final product may look like. In the next few weeks, Nathan and I will be hooking the tagging views into the controller and model.

Feedback on the new views is very much encouraged! This is still a work in progress. Without further ado, here are some screenshots of the views.

The Tag View:

The new "Tag Creation" view for the tagging feature. Note the React table and sidebar upload and download buttons. Currently, the tag data being displayed in the table is just dummy data that is hard-coded into the view. Don't worry! This will change soon! MarkUs Tagging Feature - Tag Creation View

As shown in the previous blog post, this is the Tag view. This view is used for creating and managing specific tags for an assignment. The user is able to add, edit or delete tags and see the tag usage across assignments.

The Submissions View:

From here, we now have our Submissions view. Notice the new column in the submissions table for tags. This will hold all tags assigned to that submission.

From here, we now have our Submissions view. Notice the new column in the submissions table for tags. This will hold all tags assigned to that submission.

In the submissions table, when the user puts their mouse over the Tags column, a dropdown appears featuring a search bar to filter by tags and a list of tags that they can filter by. To make the dropdown concise, only the top 5 most used tags will be featured in the list. Others can be accessed by searching for them.

In the submissions table, when the user puts their mouse over the Tags column, a dropdown appears featuring a search bar to filter by tags and a list of tags that they can filter by. To make the dropdown concise, only the five most used tags will be featured in the list. Others can be accessed by searching for them.

Individual Submission View:

In the individual submissions (Review) view, A new tag tab has been added to the right sidebar of the screen. In it, TAs will be able to add and remove tags from a current assignment and also see most used tags. Admins will have the additional ability to add or remove tags as they see fit.

In the individual submissions (Review) view, a new tag tab has been added to the right sidebar of the screen. In it, TAs will be able to add and remove tags from a current assignment and also see most used tags. Admins will have the additional ability to add or remove tags as they see fit.

There you have it. As always, feel free to check out the tagging branch of the MarkUs repo and again, feel free to suggest any improvements that you have on the feature.

Written by Bryan Muscedere

October 30th, 2014 at 7:29 am