MarkUs Blog

MarkUs Developers Blog About Their Project

Archive for October, 2010

Looking Behind the Scenes of MarkUs Development

with one comment

MarkUs is entirely developed by undergraduate students. Since MarkUs is a web-application developed for and by undergraduate students, there are some challenges to overcome. Most notably there is high developer turn-around and development happens in a fairly distributed fashion.  We try to overcome those challenges by instrumenting established means of quality assurance (QA) in the open source world. In order to keep code in shape and well tested some quite heavy tooling is involved at markusproject.org.

MarkUs website

What started as a small project at University of Toronto is becoming more and more what one might call a classical open source organization. As FOSS (Free and Open Source Software) projects don’t have a lot of money, they use various other open source software for their development. And so is MarkUs.

MarkUs developers use a mix of method and tools for their development. Some of those tools are:

  • Git (version control)
  • GitHub (wiki, issues, pull requests)
  • ReviewBoard (code review tool)
  • Tests (functional, unit)
  • Test Coverage Analysis
  • Sandbox (MarkUs demo application)
  • Blog (blog.markusproject.org)
  • #markus IRC channel on freenode

A subset of those tools fall into the realm of QA.

Git as Source Control Management (SCM)

Git is a decentralized SCM. Every developer has its own local repository. We have chosen a decentralized workflow. Nobody pushes directly to the central repository. Instead, people push to their own public repository hosted on GitHub (fork) and notify core developers, by instrumenting GitHub’s pull requests, to get changes in to the main MarkUs repository. This ensures that only well vetted code goes into the main development repository. What’s more, every patch has to be reviewed before being applied to the main repository (see section ReviewBoard of this blog post).

Git Workflow

GitHub as a Project Management Tool

In addition to providing public repositories, GitHub allows the team to manage issues and documentation (wiki). People can interact with the wiki using the Web interface, but also using Git. Several markup languages are supported. We chose to use the ReStructuredText syntax. The issue tracker is quite simple, but has labels rather than categories, which can come in handy in several cases. On a related note, GitHub allows for issues to be closed automatically.

GitHub also displays a graph of everyone’s forks and branches.

ReviewBoard

Before code goes into our main repository, it has to be reviewed by the team. This is an excellent way to improve code quality and to quickly spot common mistakes, especially in cases of high developer turn-around, which is the case for MarkUs. Another benefit is that it helps the whole team to keep an eye on every area of the project, even the ones they don’t code on. In other words it is an excellent tool to speed up getting familiar with the code base.

Once the code has been reviewed and/or revised, developers will check the “ship it” flag for a review request indicating that they are OK with submitting this code to the main development repository. Once the person who posted the review request received several “ship it”, the original poster can open a pull request on GitHub. After another once-over of a core developer the code in question gets merged into the main repository.

Dashboard of ReviewBoard

Unit Tests and Functional Tests

A great way to increase confidence if the code is doing what it is supposed to be doing is to write unit and/or functional tests for each piece of contributed code. We think we have been doing a quite good job at this. The number of tests increase almost on a daily bases. Functional and unit tests run every few hours and results are shown on on markusproject.org.

Code Coverage: How Well is MarkUs’ Code Exercised When Tests Run?

Trustworthy tests are essential. We think it’s important to check how well our code is tested. Tests coverage analysis provides us with good hints as to where our code lacks testing. Ruby Coverage a.k.a. RCov is an excellent tool to provide visual feedback as to what is tested and what is not.

Test Coverage

Test Coverage

A MarkUs Demo: Sandbox

A current version of MarkUs in action is available to anybody on markusproject.org to play with. We try to keep the running version as up-to-date as possible. Instructions as to how to use it are described in this blog post. Note that the underlying database of this instance is flushed on a daily basis. So please don’t expect your data to persist any longer than a day or two 😉

MarkUs Blog

On our blog we describe progress, propose new features, announce new releases, post individual experiences related to MarkUs development (mostly including solutions too) and various other random things. Following our blog allows you to get a feel of the current state of the project, planned features and many more. For developers it is recommended to keep an eye on the blog every once in a while. You might find an elegant solution a problem you might have encountered. Keep reading it 🙂

Blog

#markus IRC channel

We have our IRC channel, #markus, on irc.freenode.net. That’s the place where you can meet developers and can get almost instant help for a MarkUs related problem you might have. You can also just drop by and say hello. It’s a friendly small community hanging out there. In short, it is a good place to chat with current and former developers or other lurkers, who tend to hang out on IRC. See you there! 🙂

Written by Benjamin Vialle

October 16th, 2010 at 1:11 pm

Posted in Uncategorized

Punchlines for Oct 17 (Centrale Nantes)

without comments

Benjamin Vialle

Status:

  • Installed and maintaining MarkUs installation on Nantes campus. All second and third year (C.S. only) students must have access to MarkUs.
  • Wrote a blog post on MarkUs Quality Assurance with Clément
  • Migrated svn based workflow of MarkUs to a git one (with Nelle and Clément)
  • Converted a maximum of old reviews into git ones

Next Steps:

  • Write a specification report for creating annotations with Tablet PCs
  • Create some mock-ups of the new interface
  • Have a meeting with Vincent Tourre, UI expert in Centrale Nantes
  • Transfer translated end-user documentation onto MarkUs’ wiki
  • Upgrade Review Board to the latest version

Roadblocks

  • Review Board alters patches produced by Git. Still not corrected

Clément Delafargue

Status:

  • Wrote a blog post on MarkUs Quality Assurance with Clément
  • Migrated svn based worflow of MarkUs to a git one (with Nelle and Benjamin)

Next Steps:

  • Find what Javascript framework would be the best for our work, and write a blog post with findings
  • Start to work with Jérôme and Clément S. on database tables (a way to record lines from the Tablet PC)

Roadblocks

  • None

Valentin Roger

Status:

  • Installed Ubuntu and MarkUs
  • Translated Admin end-user documentation written by Morgan

Next Steps:

  • Work with Noé and Benjamin on the specification report
  • Draw some mock-ups of the new interface
  • Have a meeting with Vincent Tourre, UI expert in Centrale Nantes

Roadblocks

  • Did a chmod 700 on /etc and broke his Ubuntu 🙂 Everything is fine now !

Jérôme Gazel

Status:

  • Installed Ubuntu and MarkUs
  • Translated Grader end-user documentation written by Morgan

Next Steps:

  • Work with Clément S. on interpolating lines
  • Start to work with Clément D. and Clément S. on database tables (a way to record lines from the Tablet PC)
  • Have a meeting with Vincent Tourre, UI expert in Centrale Nantes
  • Write a blog post with findings

Roadblocks

  • None

Clément Schiano

Status:

  • Installed Ubuntu and MarkUs
  • Translated Student end-user documentation written by Morgan

Next Steps:

  • Work with Jérôme on interpolating lines
  • Start to work with Clément D. and Jérôme on database tables (a way to record lines from the Tablet PC)
  • Have a meeting with Vincent Tourre, UI expert in Centrale Nantes
  • Write a blog post on findings

Roadblocks

  • Broke his Ubuntu. Everything is fine now !

Noé Bedetti

Status:

  • Installed Ubuntu and MarkUs
  • Created English screenshots for end-user documentation written by Morgan

Next Steps:

  • Work with Valentin and Benjamin on the specification report
  • Draw some mock-ups of the new interface
  • Have a meeting with Vincent Tourre, UI expert in Centrale Nantes

Roadblocks

  • None

Written by Benjamin Vialle

October 16th, 2010 at 11:27 am

Posted in Uncategorized

Closing GitHub Issues

without comments

We’ve just moved to GitHub, and one of the main differences from Dr. Project is that we only allow a few core members to push to the official MarkUs repository.

A side-effect:  only users with push permissions on the MarkUs repository can close issues.

So how do developers close issues if they don’t have permission to do it?  GitHub actually gives us a neat mechanism to do this.  Here’s the workflow:

  1. Find an an issue on GitHub, and work on it on your fork.
  2. Once your work is done, commit to your fork with the commit message containing “closes #<issue number>”.  Without the <> symbols.
  3. Make a pull request
  4. Once the MarkUs core developers merge your patch in, GitHub will automatically close the issue.

Easy as pie.

More information here.

Written by m_conley

October 15th, 2010 at 2:59 pm

Posted in Uncategorized

Punchlines for Oct 15

without comments

Vivien Suen

Status:

  • Met with Karen and Misa to discuss some possible features/interfaces
  • Looked at Markus Demo and just started looking at the code
  • Had a bit of git issues but they have been solved

Next Steps:

  • Figure out what is possible and what isn’t
  • Meet with Misa to discuss the possibilities and draft up prototypes

Roadblocks

  • None

Hora Halmaghi

Status:

  • experimenting with different graphing libraries
  • figuring out the best ways to collect and display data in the dashboard
  • wrote a blog post outlining what Kurtis and I will be working on this semester

Next Steps:

  • write an outline of the graphing libraries Kurtis and I are considering
  • once we’re set on which library to use, get working on the dashboard!
  • finish fixing a couple more bugs I started working on

Roadblocks:

  • none

Evan Browning

Status:

  • Fixed issue with image annotation reuse and made changes suggested on ReviewBoard
  • Met with Test Framework team to discuss our plans
  • Installed MarkUs on Linux as I think it will be necessary for the test framework stuff
  • Tried (without complete success so far) to get the existing test framework working smoothly

Next Steps:

  • Get the existing test framework working
  • Fix issues with it that are currently identified on ReviewBoard

Roadblocks:

  • Having problems getting the test framework working… in particular when running tests I get this error: “RuntimeError (unknown file type: /home/user/markus/testdrive/bin/rev_req_750/test_framework_repos/A2/parse)”

Jiahui Xu

Status:

  • Have been sick since last Friday and is still trying to recover
  • Had meeting with Victor and Evan and talked about our plan for the Text Framework

Next Steps:

  • Start working on the Test Framework

Roadblocks:

  • none

Misato Sakamoto

Status:

  • Met with Karen and Vivien to discuss some possible features/interfaces
  • Started looking at (or rather, searching for) the relevant parts of code
  • Wrote a blog post about the status of the re-mark request feature

Next Steps:

  • Figure out what is possible and what isn’t
  • Meet up possibly in person with Vivien to discuss the possibilities and draft up prototypes

Roadblocks

  • None

Victor Ivri

Status:

  • Looking at existing auto-test framework

Next Steps:

  • Finish figuring out what’s already there, and what needs improvement
  • Get to work on the area of the test framework that’ll be assigned to me.

Roadblocks

  • The git workflow. I’ve been working hard to (re)familiarize myself with the system, but I still get ‘surprised’ on occasion.
  • RoR and existing codebase still present a productivity kill, but I’m getting better.

Kurtis Schmidt

Status:

  • Researched prospective graphing solutions
  • Held a meeting with Hora to determine development plans
  • Worked out a set of test graphs that we’ll use when experimenting with different frameworks

Next Steps:

  • Implement test graphs in chosen frameworks
  • Cut down list of choices to a couple of great solutions
  • Put up choices on the blog and get developer feedback
  • Start implementing reporting solution

Roadblocks

  • Time

Written by misa

October 14th, 2010 at 11:45 pm

Posted in Uncategorized

Dashboard Data Display and Graphing (Horatiu and Kurtis)

with one comment

While MarkUs is already a great tool, it is missing easy access to useful data. Our goal for this semester is to present this data in a friendly and easy-to-understand manner in the dashboard, both graphically and textually. Professors will have access to all the data, and will have the option of allowing TAs and students to see certain information, such as:

Students may be displayed the number of submission to date for a given assignment, the mode, median and average for an assignment, and the distribution of the grades.

TAs may see a progress report that shows them how many assignments they have left to mark, the number of annotations they’ve made per student and how many unique annotations they’ve made, as well as a distribution of the grades.

Professors will have access to all the data above, as well as being able to compare the TAs, by viewing their progress, annotations and their grade distributions.

At the moment we are figuring out the best ways to gather and display the data, as well as experimenting with different graphing libraries. Stay tuned for a post outlining the pros and cons of the ones we’re considering!

Written by horatiu

October 13th, 2010 at 11:34 pm

Posted in Uncategorized

Re-mark Request Feature for MarkUs (Vivien & Misa)

without comments

We met up with Karen last week to discuss what she had in mind, and iron out some issues such as where what goes and how.  We want the students to be able to request for re-marking but not make it too easy to request.  We also want them to be able to edit their request until their assignment is actually in the re-marking stage by the prof/TA.  We want profs to be able to comment on the re-mark request, as well as save original marks.

Our next steps over the course of the term are:

1) Figure out what is and isn’t possible to do GUI/feature-wise (i.e. familiarize ourselves with the existing framework and GUI)

2) Create a paper prototype

3) Have prototype tested by other profs and students

4) Possibly repeat steps 2) and 3)

5) Implement!

Written by misa

October 13th, 2010 at 7:57 pm

Posted in Uncategorized

Auto-test framework for MarkUs (Jiahui, Evan & Victor)

without comments

So far, we have been busy familiarizing ourselves with the existing structure and functionality of the auto-test framework Benjamin and Diane had been working on. We have identified several directions for work:

1) Integrate existing code, and get the basic functionality running.

2) Create the option to automatically assign marks based on test results.

3) Automatic generation of test-scripts from specs, to make the instructor’s job easier.

4) Investigate / implement security aspects to running tests.

We have decided to first familiarize ourselves with the existing framework, and only then split the work amongst ourselves.

Written by victor

October 13th, 2010 at 2:11 pm

Posted in Uncategorized

Meeting Minutes: October 8, 2010

without comments

The IRC log for this meeting can be found be find here.

Agenda:

  • Status report
  • Make sure everyone can move forward with their projects
  • Set up individual project meetings

Status report:

  • Some people have continued working on bug fixes
  • Evan hit some problems with the urgent bug he’s working on
  • Jiahui is having problems with the testing of assignments
  • Karen found a java sample and python version of some exercises that were tested automatically through a cron job (for Victor and Jiahui)
  • issue with applying patches with new files still being worked on

Projects:

  • Every team will post a blog post (a couple paragraphs) about where you are, what is the plan, etc. by Wed. Oct 13, 2010
  • Set up meetings per project via skype/google/in person

Written by misa

October 8th, 2010 at 1:56 pm

Posted in Uncategorized

Punchlines

without comments

Misa Sakamoto

Status:

  • continue with bug-fixing
  • currently waiting on a pull-request to go through

Next Steps:

  • schedule a meeting with Karen and Vivien regarding the ‘remarks request’ feature
  • continue work on bugs/issues

Roadblocks:

  • none

Evan Browning

Status:

  • Worked on bugfixes
  • Helped others get up to speed with MarkUs

Next Steps:

  • A fairly urgent bug fix
  • Coordinate with team members about testing framework
  • Go over and test code from Benjaming & Diane, fix issues that have already been identified, figure out what still needs to be done

Roadblocks:

  • none

Jiahui Xu

Status:

  • Got familiar with the test framework
  • Got faimiliar with git and the workflow

Next Steps:

  • Keep fixing some bugs in the project
  • Start working on the test framework

Roadblocks:

  • none

Vivien Suen

Status:

  • Worked on a few bugs during the code sprint
  • Found and started on a bug with the grader view and the subtotal mark not updating when adding/deducting marks via Bonus marks/deductions
  • Played with UI for remark request GUI ideas
  • Finished a midterm today!

Next Steps:

  • Look at more bugs
  • Find out more information regarding the remark request feature with Karen and Misa

Roadblocks

  • None

Hora Halmaghi

Status:

  • recovered from the weekend by sleeping in
  • thought back on how fun the weekend was, and how excited I am to work on MarkUs for the rest of the semester with everyone
  • caught up on other homework.. I promise next week you’ll start seeing review requests!

Roadblocks:

  • none

Kurtis Schmidt

Status:

  • Wrote deletion of assignments
  • Recovered from the code sprint

Next Steps:

  • Write unit test for assignment deletions
  • Research graphing solutions for MarkUs with Hora (and others?)

Roadblocks

  • none

Written by kurtis

October 8th, 2010 at 10:36 am

Posted in Uncategorized

Git Workflow

with one comment

IMPORTANT:  we highly recommend following the Git workflow instructions on the Wiki, as opposed to this post.  The Wiki workflow is more up to date, and the instructions in this post are deprecated.

Since I started writing this, Nelle wrote up a Git workflow on the Github wiki, but it shouldn’t hurt to have it here too!

The MarkUs project was just recently migrated from SVN to Git (thanks Nelle and Benjamin!) which makes us all excited! This does mean, however, that our workflow has changed a little bit. Here’s a guide for working with the new version control system:

  1. Create a Github account, and fork the MarkUs project.
  2. Add your machine’s public SSH key to your Github profile, then clone your forked repository to your machine: git clone <ssh url to your repsitory>
  3. After you find what you want to work on, create a new branch on your system: git checkout -b branch_name
  4. Code away!
  5. Add your modified files to the staging area: git add file1 file2
  6. To check the status of your files, use git status
  7. Commit your changes: git commit -m “Your commit message.”
  8. Print out the diff for Review Board: git diff master branch_name
  9. Post your diff to Review Board. If you need to fix your code, you may consider just amending your previous commit: git commit –amend
  10. Once your code has passed review, merge your branch into your master: git checkout master, then git merge branch_name
  11. Push your code to your Github: git push origin master
  12. Ask for a pull request on Github (github.com/your_username/Markus), then wait for an admin to merge your code!
One final note: make sure you’re working with the latest MarkUs code. To do so, add a second remote repository: git remote add upstream git://github.com/MarkUsProject/Markus.git and then pull it into your repository: git pull –rebase upstream master

Written by horatiu

October 2nd, 2010 at 11:14 am

Posted in Uncategorized