MarkUs Blog

MarkUs Developers Blog About Their Project

Archive for October, 2012

Punchline – UCOSP Group (Friday Meeting) – 26/10/2012

without comments

Andrey
Status:
Fixed Issue 874, created pull request.
Done with Midterms

Next Step:
Work on Issue 860 and Issue 868

Road Blocks:
Need better understanding of ruby on rails.

Kira
Status
– Same as last week. Unfortunately, I was really sick over the weekend and into the week, and have been trying to catch up on my homework and project all this week.

Next Step
– Take on the sorting issue with help from last weeks meeting notes.

Roadblocks
– None, just need a day to sit down and work on mark us (Saturday!)

Joey
Status:
– Got my previous pull requests merged (yay!) and finished a small fix to some non-escaped markup and now am into removing all of the prototype-helper code and switching it over to unobtrusive js.

Next Step:
I have most of the prototype-helper stuff removed, but it’s being used in some views that I can’t figure out how to access (which role, which function). I’ll probably bring this up in the meeting tomorrow if I make it in time to see if anyone is familiar with these particular routes.

I have a few ideas I’d like to take a look at next, the main one being optimizing a lot of ajax calls around the site. Looks like the main culprit is the FilterTable, and the way it expects data from the ajax calls, and how the data needs to be generated on the server. Basically, for each row in the table, the server has to create a hash with the values of the row, and then render a view that puts these values into a string of html which is also a hash key, and then has to send all of this data (the hash including the rendered row) to the client to be inserted in the table. In the log, I can see that the
view rendering sometimes takes up to 200ms per row, on top of 40-100 rows, which causes a long delay on some requests. I figure with some changes, the FilterTable should be able to accept just the jsonized
hash of values, and generate the mark-up for the table on it’s own, using a client side template of some sort, or just render the table directly into the view initially, and have the FilterTable manipulate
the table. Seeing as though there are so many pages that depend on the FilterTable, it seems like something worth putting time into and optimizing.

Roadblocks:
– I’ll be running a little short on time coming week, with 3 assignments due, and a midterm, but I’ll do what I can!

Emeric
Status:
– Finished understanding the current implementation of the user preferences
– Fixed some bugs in the Graders page (with user preferences)
– Fixed Issue 900(based on Aimen’s work)
– Made the submissions user preferences (hide/show column) save correctly

Next Step:
– Save user preferences in the cookies instead of the session
– Hide/Show columns popup alignment on both Firefox and Chrome
– Some work still needs to be done on the Graders page to make user preferences work
– There are also some alignment issues with the graders table (since new columns were added to it)
– Add sorting order to the user preferences (make sure it’s saved on all pages)

Roadblocks:
– None

Joel
Status:
– Fixed Issue 851 with help from Michael Ing, pull request now merged.
– Closed Issue 896. I was working on older code and someone had fixed it already.
– Documented two more issues for the Remark Requests (#903, #904)

Next Step:
– I feel like writing a blog soon!
– Fixing and closing all of the documented Remark Request issues.
– Also, I didn’t notice anything wrong with csv outputs after remark requests. I’ll peek at it again, but no bugs found yet.

Roadblocks:
– Still getting familiar with Ruby on Rails… It took me a wee bit of time to follow the Ajax requests and understand calling controller methods from the view. Baby steps!

Michael Ing
Status:
– Been busy the whole week so I am pretty much in the same spot as last week.
– Setting up production test environment. Ran into some issue about not setting up correctly with the master. May have to re install again and document where I went wrong so it may improve installation.

Next Step:
– Get production environment setup so I can update documentation based on it.

Roadblocks:
– To busy to come up with an interesting roadblock about how busy I am.

Ante
Status:
-Didn’t do much this week.

Next Step:
Issue 872(disable button). Must get it done this weekend.

Roadblocks:
-Assignments from other courses are so hard!

Oussama
Status:
– At the same point as last week, was not able to do much because of a hectic week with multiple assignments/midterms

Next Step:
– Will be be to allocate more time for Markus from this point onwards
– With the help of last weeks meeting notes, will proceed with Karen’s suggestions regarding making improvements to the marks spreadsheet

Roadblocks:
– Will have to miss this weeks meeting, unfortunately, due to a religious celebration I need to attend

Written by mikeing2001

October 26th, 2012 at 10:01 am

Posted in Uncategorized

Punchlines ECN – 24th October 2012

without comments

Here are the punchlines from our group (students from Centrale Nantes):

Amandine:

Status :

  • Final validation of the requirement document
  • Study of the code of MarkUs
  • Quick review of Ruby On Rails and of RailRoady (UML diagram for Rails)

Next steps :

  • Development and development
  • Implementing a quality phase

Alexy:

Status :

  • Final modifications of the views to match the requirement document
  • Final version of the requirement document
  • Studying of the existing code about markus

Next steps :

  • Elaborating the different teams and which tasks they are going to work on
  • Finding someone who is familiar with MarkUs

Nicolas:

Status :

  • Worked on the final version of the requirements doc
  • Learning Ruby on Rails (for Zombies)

Next steps :

  • Look at MarkUs’s code

Sebastien:

Status :

  • Final modification of planning
  • Final version of requirements

Next steps :

  • Dig in MarkUs’s code

Nora:

Status :

  • Final validation of the requirement document
  • Study of the code of MarkUs
  • Get familiar with git

Next steps :

  • Trying to implement quality methods for the development
  • Continue studying MarkUs code

Antoine

Status :

  • Final validation of the requirement document
  • Study RoR and Railroad
  • Look at the code of MarkUs

Next steps :

  • Start development

Written by amandine.lavergne

October 24th, 2012 at 9:04 am

Posted in Uncategorized

Ideas for passing data to the Test Runner

with 10 comments

The test runner is the program that would be called to run the instructor-provided test scripts, and would return the data to MarkUs, which would then parse the results.

The test runner is supposed to return the following data as XML:

–          Test id – The name of the test

–          Input – The input used for the test

–          Expected Output – The correct output

–          Actual Output – The actual output (from the submitted code)

–          Result – Pass or Fail, depending on the output, or Error if there was an error during testing

–          Marks – The marks earned by the student

The input for the test runner is currently a file (or input on stdin) that contains the name of each test followed by a flag (to determine whether the program should halt if that test fails), followed by another test, etc.

The issue that arises from this is that there is more data returned than is sent to the test runner in the input file, and the test runner needs some way of receiving that data.

As a result, we must find another way to send that data to the test runner.

 

There are 3 solutions that I can see, and I’d like to ask for feedback.

1.       Return less data

This is the simplest solution, although arguably the worst (I’m just including it here in the off chance that it’s actually a viable solution).

Instead of returning all of the above information, the test runner could return a subset of it; specifically, it could return the test id, the actual output, and the exit status of the test script.

The advantage of this approach is that the input for the test runner would remain the same.

The disadvantage is that MarkUs would need to compare the results and determine the marks received. This means that the instructor would still need to submit the input and correct answer to MarkUs, which raises the question of why that information wouldn’t be sent to the test runner.

 

2.       Include more data in the input file

Instead of simply passing a file where each line is “test_name, halt_on_fail”, more data could be included. For instance, “test_name, halt_on_fail, input, expected_output, marks” could be passed instead.

By passing this data, the test runner could compare the actual and expected output, and return the appropriate status, and determine the number of marks to be awarded. The “input” field could likely be omitted since the test runner doesn’t use it, so the only real changes would be the inclusion of the “expected output” and “marks” fields.

The main advantage to this approach is that the input would remain almost the same.

The disadvantage though is that the testing interface would need to be updated to allow the instructor to specify the input, target output, and number of marks, which can be quite tedious to enter if there are a large number of tests.

 

3.       Have the test script output the data

Rather than change the data that is passed to the test runner, or change the testing interface, we can simply require that the first 3 lines output by each testing script are the input, target output, and number of marks that test is worth.

This seems to be the simplest of the 3 approaches.

Without this approach, the test script would already need to define the input for the student code, as well as the target output. It would be trivial to include a print statement to print that to stdout. The only real addition to the script would be to add a line printing the number of marks that test is worth.

The main advantage of this approach is that the interface does not need to be changed, and the instructor will not be required to submit additional information; they can simply put the information in the test script. As well, no extra input is needed for the test runner.

The main disadvantage though, is that the instructor must output the data in every script, and in the correct order. If the print statements are ordered incorrectly, then the results for that test will be incorrect.

I believe that this may be the best solution to this problem.

 

Feedback would be appreciated.

-Mike

Written by mmargel

October 20th, 2012 at 4:18 am

Posted in Uncategorized

Punchline – UCOSP Group (Friday Meeting) – 19/10/2012

without comments

Joey
Status:
Just submitted 2 pull request for issues that I was working on. One pretty simple fix to warnings/errors in tests, but at the same time there were 2 more functional test errors pushed to master so… The other one was some fairly significant style changes, and I hope everyone is okay with them.

Next Steps:
There are a few issues I’d like to pick up, but I’ll start with one that deals with legacy JS code, so that I can move into trying to optimize the ajax requests, and then maybe take on some more UI tweaking and design because I’ve found lots of places where it could be fixed/improved.

Roadblocks:
None at this time.

Kira
Status:
– wrote the blog post about improving the marks spreadsheet
– re-factored the warnings for global actions to use a shared partial.
– added the section column to the marks spreadsheet
– started to look into sorting on the column names. This has been done with a FilterTable in the Users page, I think the same thing could be applied to the marks spreadsheet. Read more documentation around FilterTable written by Mike Conely. Had a first go around with it.

Next Steps:
I could get the FilterTable showing up but it needs the correct headers, and I’m not sure how exactly it deals with columns that you can input stuff into. So the next step is to investigate whether FilterTable is the correct approach and implement it (lots of work here)

Road Blocks:
Don’t know much javascript so making filter tables work may be quite the undertaking for me but I’m for a challenge

Emeric
Status:
– Pulled Aimen’s user preferences branch (issue689)
– Merged with master and fixed conflicts
– Started testing and working on the user preferences (making sure everything is still working after the merge)

Next Step:
– Finish saving user preferences on reload (cookies, etc.)
– Finalize work on user preferences in general (if I find glitches of any sort)
– Create some unit/functional tests to test the user preferences

Road block:
– None

Ante
Status
– Made a pull request for issue871(disable ‘save’ button after press )

Next Step
– Work on issue872(similar to 871)

Road Blocks
– Midterm tomorrow morning so I may have to miss the meeting again(Sorry!). Will try to come for the last 10 minutes and go through the meeting log
– Need to figure out how to update markus wiki on github. I forked it into my github account and updated it but how do I generate a pull request?

Andrey
Status:
– Closed pull request for issue 876
– Made the fix for issue 874, need to make a pull request

Next Step:
– Do tutorials on ruby and rails.
– Work on issue-868

Road Blocks:
– Midterms

Michael Ing
Status:
– Setting up production test environment. Ran into some issue about not setting up correctly with the master. May have to re install again and document where I went wrong so it may improve installation.
– Wrote blog post about git pull a branch

Next Step:
– Get production environment setup so I can update documentation based on it.

Road Blocks:
– If time is money and I pay very high tuition, shouldn’t I have more time?

Joel
Status:
– Spent time trying to replicate a few issues I found. Also, with Michael Ing’s help I’m just about done with issue #851.

– While trying to replicate the remark request problems I had another error pop up on me. As administer I release grades for an assignment, then log in as that student and get an error.

ActionController::RoutingError in Assignments#index
Showing /Markus/app/views/assignments/_list.html.erb where line #58 raised:
No route matches {:action=>”view_marks”, :controller=>”results”, :id=>1}

Next Step:
– Figure out why this routing error pops up. Has anyone had problems with releasing grades before?

Road Blocks:
– Getting through bugs and finally working on the remark requests.

Oussama
Status:
– Learning about the marks spreadsheet and trying to find room for UI improvement

Next step:
– Will read more documentation and Kira’s blogpost about the this component of the system
– Will coordinate any planned fixes/features with Kira

Road blocks:
None

Written by mikeing2001

October 19th, 2012 at 9:37 am

Posted in Uncategorized

Punchlines ECN – 17th October 2012

without comments

Here are the punchlines from our group (students from Centrale Nantes):

Amandine:

Status :

  • Made the planning of the project
  • Reflexion about the views according to the scenarios

Next steps :

  • Validation of the requirement document by Morgan Magnin with the different views
  • Focus on which points to develop
  • Point with all the team for the technical aspects

Road block :

  • Issues with the planning still on the part “development” which won’t be clear until the requirement document is not validated

Alexy:

Status :

  • First elaboration of the development part of our project
  • Creation of some of the views to illustrate our project according to the scenarios
  • Learning of Ruby
  • Precision of the specifications

Next steps :

  • Validation of the specifications
  • Finish the development part of the planning
  • Start the sharing of the tasks (development part)

Nicolas:

Status :

  • Summarized and synthesized the poll’s results
  • Worked on the requirements document (mainly translation)
  • Learning Ruby

Next steps :

  • Finish the development part of the planning
  • Validation of the requirements document by Mr. Magnin
  • Become more familiar with Markus
  • Ruby on Rails

Sebastien:

Status :

  • Reflexion on the planning
  • Analysed and summarized the poll’s results
  • Ruby

Next steps :

  • Finish the development part of the planning
  • Validation of the requirement document by Morgan Magnin

Nora:

Status :

  • Working on the requirement document (adding sequence diagrammes)
  • Correcting the requirement document

Next steps :

  • Learning more about Ruby
  • Finishing the requirement document

Antoine

Status :

  • Creation of views to illustrate our project according to the scenarios
  • Start learning Ruby
  • Correcting the requirement document

Next steps :

  • Validation of the requirement document
  • Keep on learning Ruby

Road block :

  • None

Written by amandine.lavergne

October 17th, 2012 at 8:56 am

Posted in Uncategorized

Ideas on how to upgrade the marks spreadsheet

without comments

1. Add a column for Section (Issue 891)

2. The user should really be able to sort by User Name, Last Name, First Name, Section. Follow the same style as the submissions page (Issue 894)

2.a Have the Jump to box correspond to column which the sort is applied to

3. Be able to add groups for a grader (just like for assignments).

4. When marks are released there should be a prevention of accidental changes.

5. When the marks are entered, there should be a validation that checks if the number is a number (not a letter or symbol). Should not allow invalid characters.

6. If you type a number outside of the grading range then a warning should be shown. I’m thinking what happens when the mark 9.5 wants to be entered and the decimal is forgotten. Then 95 is shown, a warning pops up and the system still excepts it. Same goes for negative numbers.

7 The message field in properties. This field is ONLY viewable to admins. If this was the intention (which I don’t believe it was) it should really be documented somewhere. Otherwise, it should be somehow made view able to the appropriate parties.

8. The date field in properties is more confusing than useful. Is it for when the marks are being release, when the marks were collected, or something else. What I think is needed is some brief documentation (I can’t find any, there was only a screen cast) just about an overview of the mark spreadsheet.

9. The student interface for the marks could use a little update to make it look a bit better.

Please keep adding to this post if you have more ideas on how we can improve the marks spreedsheet

Written by kiramccoan

October 16th, 2012 at 8:19 pm

Posted in Uncategorized

Git tips on How to Pull a Branch

without comments

This blog post takes the assumption that you setup your folk using How to use git with MarkUs blog post.

From the previous blog, to update our local master branch we used:

$ git checkout master
$ git pull markus-upstream master

What this does is pull and merge the changes from the location markus-upstream and from the master branch into your current local branch. Thus following this format.

$ git pull {repo} {remotebranchname}

So if we want to pull someones branch there are 2 ways it can be done

1) Pull to a local branch that already exists

  • This method will be very similar to above. First do a git checkout to the branch
    $ git checkout branch123
  • Second do a git pull on what you want to pull from
    $ git pull {repo} {remotebranchname}
  • Example
    Say someone want to pull my branch123 and place it in there local branch Michael-123fix

    $ git checkout Michael-123fix
    $ git pull git://github.com/mikeing2001/Markus.git branch123

2) Pull to a new branch that hasn’t been created yet

  • If the branch has not been created then the steps can be combine into one.
    $ git pull {repo} {remotebranchname}:{localbranchname}
  • Example
    Say someone want to pull my branch123 and place it in there local branch Michael-123fix

    $ git pull git://github.com/mikeing2001/Markus.git branch123:Michael-123fix

With the above 2 methods you can also replace the git url with a remote like we did with markus-upstream
$ git remote add mikeing2001-markus-branch git://github.com/mikeing2001/Markus.git

Written by mikeing2001

October 13th, 2012 at 2:35 pm

Posted in Uncategorized

Punchline – UCOSP Group (Friday Meeting) – 12/10/2012

without comments

Emeric
Status:
– Done with issue #847 (notes dialog alignment)
– Fixed the problem Mike told me about as well

Next Step:
– Continue Aimen’s work on user preferences (get his branch)

Road block:
– I might need some advices on the best way to get Aimen’s branch to continue his work. I’ll look into it a lot more next week.

Kira
Status:
– Made a fixes for issue271 to have less code duplication. Created my first partial (yay)
– Made a pull request for issue883 (more warnings for graders)
– Made a pull request for issue879 (pagination and jump to fix)

Next Step:
– Have list compiled of things I would like to improve on the marks spreadsheet so just need to decide on one and start working on it.

Road Blocks:
– Have a midterm and project design review next week so may be a bit slow in the beginning of the week. Other then that nothing

Andrey
Status:
– Worked on issue 874.
– Made a pull request.

Next Step:
– Make sure issue 874 is completely finished.
– Try to work on issue 868 (Will need help!!!)

Road Blocks:
– Things due everyday, and I’m a last day worker.
– I slowly learn more about web development. I’m taking another class about web development. So far I have covered HTML and CSS, JavaScript is next.
– It will take sometime before I can tackle something bigger then just layout issues.

Oussama
Status:
– Reverted previous changes pertaining to issue 840. Checked in new changes and pull request was updated automatically.

Next step:
– Looking into user preferences. Also searching for any other issues that need attention.

Road blocks:
– None

Joey
Status:
– Just wrapped up an issue 866 I was working on that fixes any warnings or errors in the unit/functional tests. Will hopefully get a pull request out tomorrow.

Next Steps:
– Will be starting work on an issue dealing with some changes to the stylesheets to fix deprecated styles in Firefox, as well as another for modernizing some old prototype/javascript code.

Road blocks:
– None at the moment.

Michael Ing
Status:
– Currently reading production documentation to look for places where it needs to be changed
– Performing code reviews as well as trying to find fault in the code

Next Step:
– Test out the documentation with the latest version of software (Apache, PostgreSQL…) as MarkUs last release was about a year ago and document any changes needed
– Continue finding bugs, code review
– Possibly finish removing the remaining legacy code (time permitting if no one has done so yet)

Road blocks:
– Nothing just busy

Joel
Status:
– Went through the request remark process and have a good grasp of how to duplicate the odd problems we were getting. Data doesn’t seem to be lost though, I think its just a visibility thing. I also have a list of other things to fix on those pages.

Next Steps:
– Make issues for any newly discovered problems, and eventually fix them.
– Dig into why the old marks don’t always appear.

Roadblocks:
– This bug: issue 851 It takes me several tries to get a grade to stick, it just makes it slower when manually creating test cases.
– Finding the error

Ante
Status:
Next Steps:
Roadblocks:

Written by mikeing2001

October 12th, 2012 at 12:11 am

Posted in Uncategorized

Punchlines ECN – 10th October 2012

without comments

Here are the punchlines from our group (students from Centrale Nantes):

Amandine:

Status :

  • Meeting with Anne-Céline Grolleau of Centrale who has an expertise in the skills’ approach.
  • Write scenarios and UML diagrams of activity for the requirements of MarkUs.
  • Sent a poll to the students of Centrale to identify their needs for MarkUs.

Next steps :

  • Last consolidation and validation of the requirement.
  • Planning, management of the tasks.
  • Do a dashboard of the results of the poll.

Road block :

  • None

Alexy:

Status :

  • Meeting with Anne-Céline Grolleau of Centrale
  • Participation in the writing of scenarios and UML diagrams of activity for the requirements of MarkUs.

Next steps :

  • Last consolidation and validation of the requirement.
  • Finish the requirement document.
  • Continue learning Ruby on rails

Road block :

  • None

Nicolas:

Status :

  • Meeting with Anne-Céline Grolleau.
  • Participation in the writing of scenarios and diagrams of activity for the requirements.

Next steps :

  • finish the work on the requirements
  • analyse poll results

Road block :

  • none

Sebastien:

Status :

  • Meeting with Anne-Céline Grolleau.
  • Wrote scenarios and their diagrams for the requirements document.
  • Began to collect results of the poll.

Next steps :

  • Analyse the poll’s results.
  • Finish the requirements document.
  • Continue learning ruby on rails.

Nora:

Status :

  • Meeting with Anne-Céline Grolleau of Centrale who has an expertise in the skills’ approach.
  • Write scenarios and UML diagrams of activity for the requirements of MarkUs.

Next steps :

  • Last consolidation and validation of the requirement.
  • Writing comments for the requirement document.
  • Continue learning Ruby on rails

Road block :

  • None

Antoine

Status :

  • Wrote scenarios and activity diagrams for the requirement document

Next steps :

  • Finish the requirement document.
  • Focus on Ruby on Rails

Road block :

  • None

Written by amandine.lavergne

October 10th, 2012 at 8:27 am

Posted in Uncategorized

New database schema for storing test configuration and results

with one comment

I have taken Byron’s schema and thought about it a little more, and here is what I came up with. I am also trying to figure out how to save the test configuration in a way that makes it easy for us to send it to the test runner to start the test. Here is what I have so far:

Test_suite
  test_suite_id           // identify this test
  assignment_id           // the assignment this test belongs to
  name                    // test name
  description             // description of the test suite, if any
  halt_on_error           // flag whether to halt or continue on error
  pretest_scripts         // ordered space separated list of pretest scripts
  build_scripts           // ordered space separated list of build scripts
  test_scripts            // ordered space seperated list of test scripts

Test_run
  run_id                  // sequential number identifying this run
  test_suite_id           // the test suite that was ran
  group_id                // group that ran the test
  assignment_id           // assignment being tested
  cumulative_mark         // sum of marks from all the test files
  date                    // date & time run was requested

Test_file_result
  run_id                  // link to the Automated_test_run it belongs to
  test_suite_id           // the specific test that was run
  name                    // name of the specific test file that was ran
  kind                    // one of {pretest, build, test}
  status                  // one of {passed, failed, error}
  marks                   // marks awarded for this test file
  input                   // input passed to the student file
  expected_output         // expected output from the student file
  actual_output           // actual output from the student

 

The test suite contains information we get from the instructor when he/she first sets up the testing. The most interesting fields are pretest_scripts, build_scripts, and test_scripts. The three fields are simply space separated strings of the names of respective scripts the instructor wants to run for each test.

The test run table contains an record of an individual run of a test suite

The test_file_result table contains the results of running each individual test file in the test suite.

One thing to note. For this schema, I assume that the instructor will go through the trouble of creating pretest, build, and test scripts that don’t need any flags or input files to run, and hence all we need to run them is the name of the script, which the only thing this this schema stores about the test files the instructor uploads. Is that what we are aiming for? If not, then I think we will have to add another table dedicated to the test files, and information about them (their input files and the flags they run with).

Written by mina

October 7th, 2012 at 9:10 pm

Posted in Uncategorized