Archive for June, 2010
I noticed that loading MarkUs from localhost with Firefox was taking much much longer than it took with Chrome or even IE. I thought this was just due to my many development add-ons but when I looked at the page loading with firebug I noticed that most of the time was spent on DNS lookup. From searching online, it seems that this is a known issue that occurs because Firefox is trying to find an IPV6 address for localhost but only receiving an IPV4 address.
To make things faster, go to about:config and set network.dns.ipv4OnlyDomains to the value “localhost”. Now Firefox only tries to find an IPV4 address 🙂
As some of you probably already know, I’ve recently finished implementing the image annotation feature for MarkUs and already committed the work. MarkUs now supports the standard web image formats – jpg, gif and png. Currently I am in the process of adding the ability to annotate PDFs.
My approach to this task is to convert the PDF files to JPG format via the ImageMagick software and the RMagick gem designed for ruby and then display it in the browser. This seemed like a reasonable approach, as we already have the ability to annotate JPG’s. Currently, I successfully managed to integrate the PDFs and I am at a point where they are converted and displayed in the browser and the user is able to annotate them.
Great! So why not commit and ship these features right away you say? Right now, the main issue is the fact that PDF-JPG conversion is a very expensive operation on the order of ~2-3 seconds per page and we obviously need to accommodate for multiple-page submissions. For example, it takes a good 25 seconds to convert a 900kb 11-paged PDF file. Class sizes can easily range in the hundreds, so a 300 person class would require 7500 seconds – just over 2 hours to convert all the PDFs.
The length of the conversion is due to the need of preserving the file-content quality. On its default settings, an ImageMagick conversion gives a poor image quality, rendering standard-sized text unreadable and thus the submission is effectively useless. To combat this, I use supersampling – increasing the image resolution while decreasing the image size. This significantly lengthens the conversion process.
In light of this problem, we clearly want as little conversions happening as possible – 1 conversion per file is the best we can do. Thus I have decided to store the binary data of the images (post-conversion) in the database, for quick access. The alternative would be to store the converted file in the student repositories, but this is very undesirable for multiple reasons.
Thus we are faced with several new questions. When do we convert and how many files at a time do we convert? After speaking with Karen, we decided to kick-off a process to collect all submissions and convert them as soon as the first user logs in after an assignments submission date and grace periods have passed. I will create a queue-like object to handle this process. It will be responsible for accommodating any graders that try to mark a submission with a PDF in it before it has been converted, by bumping their job ahead of schedule. Thus waits for conversion of individual submissions may still occur, but they will happen less often. Colour coding the list of submissions may also prevent waits on the conversion process.
I met with Dan today to discuss his experiences using MarkUs and what suggestions he would have for improving its accessibility. His first comment was that MarkUs is a mess, enough so that he ended up hacking the databases instead of using the web interface for most things. However, it’s encouraging that MarkUs isn’t the grading system that he finds the most frustrating 😉
Here are the most important issues that he would like dealt with:
- Currently, in order to read annotations, you have to mouse over the corresponding line of code. It would be great to have an alternate format, perhaps available for download by instructors, that inserts the annotations as comments into the code files themselves. This would make it easy for blind users to review annotations in context.
- There is a problem with the groups and graders tab that prevents him from adding, removing or moving students between groups. Since we are planning to redo this section of MarkUs anyway, that is a perfect opportunity to make sure the new groups page is accessible! It would also be good to be able to download and upload group lists in a more palatable format, something that begins by listing the students in the group for example. The format (this may not be the same as the new csv format) that Dan was using had the “group_####” name followed by a large list of TAs, before finally eventually getting to the students that are actually in the group, so Dan found it difficult to navigate.
- It would be great to give instructors a link to the SVN repositories of groups, and maybe a way to download a file containing all the repositories of every student. Dan would be much more able to commit a file with comments it to a student’s repository than he would be able to use our visual annotation tools.
It might not be feasible to do everything that would help with accessibility, but if we can do some of these things then maybe we can stop Dan from editing databases! He is very interested in helping us out and will set up a development environment on his machine so that he can keep track of our progress and make suggestions.
It was good to hear that some issues have already been fixed. Dan wanted a way to re-upload rubrics after he had downloaded them, and the latest MarkUs supports that. He also very much wanted to be able to reorder criteria without using a mouse, and that fix is currently on ReviewBoard. I’m really happy that someone will be able to immediately benefit from my code!
Talks concerning MarkUs Automated Testing Framework
We defined some specifications for the Automated Testing Framework we wanted to implement. First, the framework must add as few dependencies as possible, and not disturb the core of MarkUs (MarkUs must not be overcrowded by other programs for testing student’s code). The Automated Testing Framework must also support as many languages as possible, allowing users to test any language regardless of the languages MarkUs has been implemented in.
Today, C, Java, Python and Scheme are used in the Universities where MarkUs is deployed (Department of Computer Science, University of Toronto.School of Computer Science, University of Waterloo.Ecole Centrale de Nantes).
C++ should be easy to implement according to the work done with C.
Diane and I are going to build our framework by using examples from these languages. When we speak of an Automated Testing Framework, the idea is not to build a new Unit Testing Framework. We aim to build an “abstraction layer” between MarkUs and most used Unit Testing Frameworks for most used languages in University.
A picture is far better than words :
MarkUs will call an Ant script written by the TA. This script, and all necessary files for the test environment (student code, data and tests) will be sent to a “working environment”. Later, the working environment could be moved to a dedicated server.
MarkUs will ask the Ant script to do many tasks (see Ant tasks), in an asynchronous way. Next, the Ant script, customizable by each instructor, will compile, test and run student’s code. The Ant script will produce an xml or text output for tests and build. Ant’s output will be saved in MarkUs and will be filtered for instructors and students.
The student will be able to see the result of the compilation and some test results will be made available by the instructor.
I have been working on making MarkUs more accessible, particularly to blind users. To help me with this goal I installed a free open source screen reader, NVDA. I plan on using other screen readers and technology in the future as well as working with blind users.
To begin with, the screen reader is really fun, but after navigating for sometime you start to get frustrated with the length of time that it takes to do some things, and the voice gets annoying too. I can see why people who use screen readers all the time make the voice go so fast, you want to be able to listen to everything as fast as you could read it visually.
After working with MarkUs for a few days in this format, here are some key features for accessibility that I need to work on, and which other developers should keep in mind when working on user interfaces:
- Everything has to be accessible using the keyboard only! This is the only input method that we can assume everyone will be able to use. The biggest problem with this in MarkUs is the accordion menus, they are only accessible through the mouse. We will have to either get rid of them or make it so that they can be navigated to using the keyboard. Another problem is dragging and dropping for reordering things, this is really convenient with the mouse and should not be removed, but we should add alternative buttons for keyboard access.
- Every form input element should have a label! We are pretty good at doing this so far, but there are places here and there where there is text that describes the form element, but is not formatted as a label. It’s pretty easy to change this text into a label and it really helps to identify what each element does, otherwise you just hear something like “checkbox unchecked” and have to go hunt for the text that labels it. There are some places where having a label does not make sense for sighted users (like in a table of students, we don’t want to have a label beside each checkbox) but we can make hidden labels in this case for the benefit of people who don’t see the visual association.
Overall we have already done a good job of making things accessible just by following web conventions well, and if we deal with the points I mentioned above then I think MarkUs will be doing well accessibility-wise. The next step is to work with someone who actually requires the use of a screen reader, and I’m sure I will learn a lot more about what we could do to make things better!
I am using the wiki page as a way to keep track of progress for myself, look there for more specific details.