MarkUs Blog

MarkUs Developers Blog About Their Project

Archive for August, 2008

Switching Rails to PostgreSQL

with one comment

Since most of the web applications we develop here uses PostgreSQL, we’ve decided to port our environment to use postgres instead. Here I outline how I managed to get rails and PostgreSQL 8.3 up and running on Ubuntu 8.04.

1. Install PostgreSQL

If you haven’t already, try typing:

sudo apt-get install postgresql postgresql-client postgresql-contrib libpq-dev

Make sure that the installation completed successfully by typing psql --version. Current version is 8.3.3. libpq-dev is important since the postgres gem needs it, so make sure that we have that installed even if you have postgresql installed already.

2. Install PostgreSQL adapter for ruby

Update your rubygem first by typing sudo gem update --include-dependencies. You then need to install ruby-pg, which superseded the now-obsolete ruby-postgres gem (which can’t even compile postgres 8.3 due to a compilation bug). To install postgres gem, type:

sudo gem install postgres

3. Create a PostgreSQL user

By default, the user in the rails DB config file is the project name. Thus we can create this user by going into the postgres console

sudo su postgres -c psql

Once in the psql console, type (including the quotes)

create user "<user_name>" with [superuser] password '<password>';

Include the superuser option if needed, usually easier for development when working on your own computer. Verify that the CREATE ROLE message was displayed then exit the console.

4. Change PostgreSQL Authentication

When executing some of the database commands, you might encounter a FATAL: ident authentication failed error. To avoid this, you need to edit your pg_hba.conf (found in /etc/postgresql/8.3/main in Ubuntu 8.04) and change the authentication scheme. Open the file with root privilege and change

# "local" is for Unix domain socket connections only
local all all ident sameuser

to

# "local" is for Unix domain socket connections only
local all all md5

Then restart your postgres server by typing sudo /etc/init.d/postgresql-8.3 restart

5. Create the databases

to create the database we can use the createdb shell command, by typing on prompt:

createdb <project_name>_development -U <user_name> -W

or, if user does not have superuser account, from the psql console (by sudo’ing as postgres from before) typing:

create database <project_name>_development owner <user_name>;

and give privilege to the user by typing:

grant all privileges on database <database_name> to <user_name>;

Do the same thing for creating <project_name>_test for the test database.

5.5 Install phppgadmin (optional)

To make your life easier, there’s a php console for developing with postgres called phppgadmin, derived from its MySQL counterpart, phpmyadmin. phpmyadmin is still a lot better interface, but phppgadmin is better than nothing at all. Assuming you have PHP and apache2 installed, install phppgadmin by typing:

sudo apt-get install phppgadmin

and create a symlink for apache:

sudo ln -s /etc/phppgadmin/apache.conf /etc/apache2/conf.d/phppgadmin.conf

Reload apache by typing sudo /etc/init.d/apache2 reload and go to http://localhost/phppgadmin/ to play around with your new tool.

6. Generate a database config file for PostgreSQL

If you’re starting from scratch, the easiest way is to create your application using

rails <project_name> -d postgresql

However, if we’re migrating to postgres from a current rails application, then the easiest way is to just create another rails application of the same name but on a different directory. Go to a directory where your current rails application is not located and create a rails application of the same name using above command. Once that’s created, just copy the database config file to your old rails application by copying:

mv <original_project_path>/config/database.yml <original_project_path>/config/database.yml.old

cp <project_name>/config/database.yml <original_project_path>/config

Once copied, open database.yml on a text editor and fill in the database details we just created.

Epilogue

One of the problems I encountered was figuring out which postgresql adapter to use. ruby-postgres wasn’t working with postgres 8.3 so I regressed back to 8.2, only to find out postgres adapter exists. Not much documentation exists since it’s fairly new and a lot of people I guess is still in psql-8.2. The other one is the dreaded FATAL: ident authentication failed error you get when starting to use psql on Ubuntu. Again, thanks to Jeff and the Dr. Project documentation, we finally got it working.

And finally, psql-8.3 seems to break the implicit cast being done from the previous 8.2 before.  So now, when I try to find matches where numbers are stored as string, I get the following error:

ERROR:  operator does not exist: character varying = integer

The column being declared as string does not match the passed integer argument, which should be converted to String in the first place.  Thus I had to explicitly cast it to a String before using.

For more information regarding PostgreSQL and Ruby on Rails, see the official wiki.

Written by Geofrey

August 12th, 2008 at 10:22 am

Posted in Uncategorized

Migrate Your Tests

without comments

Rails has a nice way of maintaining versions of your database schema through ActiveRecord:Migration.  This lets you modify your existing schema without the hassle of manually copying the same schema to all the different deployed instances that you have, and does this automatically for you.

When I usually change the schema, I usually drop all the tables and recreate them using:

rake db:migrate VERSION=0
rake db:migrate

and then repopulate my development DB environment using a script I use.  However it turns out that this doesn’t migrate your test environment automatically with it. I tried creating a simple test of making sure that my ActiveRecord validation works.  However, I ended up getting this error:

test_numericality_group_limit(AssignmentTest):
NoMethodError: undefined method `group_limit=' for #<Assignment:0xb7242018>
/var/lib/gems/1.8/gems/activerecord-2.1.0/lib/active_record/attribute_methods.rb:251:in `method_missing'
test/unit/assignment_test.rb:11:in `test_numericality_group_limit'
/var/lib/gems/1.8/gems/activesupport-2.1.0/lib/active_support/testing/setup_and_teardown.rb:33:in `__send__'
/var/lib/gems/1.8/gems/activesupport-2.1.0/lib/active_support/testing/setup_and_teardown.rb:33:in `run'

After hours of finding the bug, I found out that you have to migrate your test DB environment as well, by executing the following:

rake db:schema:dump
rake db:test:prepare

This will copy the schema that you have right now in your development environment, and copy it to your test environment.

Written by Geofrey

August 6th, 2008 at 10:49 am

Posted in Uncategorized

Phase 1: Complete

without comments

This week, I’ve finally managed to wrap up the student management module in rails. I had some leftover work done from implementing the authentication and session timeouts, as well as writing several tests. Since there were several complaints in OLM before from TAs being timed out, we’ve decided to set different timeout lengths for students and faculty, giving longer timeouts for TAs to finish their work (although it wouldn’t hurt to save once in a while too).

Next, I moved on to implementing the page for managing the course classlist. This involved doing some parsing work for the uploaded CSV file. This took me a while since I was under the assumption that the given classlist would be separated by spaces. I’ve started looking for CSV parsers where you can change the default delimiter (you’d think this would exist). Since the search was in vain, I just implemented my own parser only to find out that there was no way to read the uploaded file line-by-line unless I write it first to a tempfile. By this time, Karen already sent me a sample classlist only to find out that it was indeed a normal CSV file (whew!). So I just used the rails CSV parser and everything works as planned.

The problem I have now is how to test uploading the file automatically. Fortunately, rails has something called a fixture_file_upload where you can just specify a sample upload file and call a post method on your action. I haven’t successfully tried to use this though, since I’m still getting some errors when running it. I’m running out of ideas why my tests are failing. I also ran into some problems dealing with error handling when trying to enter invalid inputs on creating and editing students. Turns out that the symbol declared in the form expects a global variable with the same name in order to hook them up together (who knew?). Things that are automatically done like this without telling you is where sometimes rails becomes a pain. Sometimes, declaring an innocent variable might be something that would override the default rails way of doing things. But all that hard work seemed to pay off once you get everything working (except for the upload tests).

Now, I was suppose to get the student file submission done by the end of the week, but I’m sort of glad that I didn’t. My initial design draft focused only on the uploading the file draft without even considering the next step: student groups. As it turns out, I might need to implement them at the same time to have a cleaner design with less hassle. The plan is to get a database schema that I can present to Karen and Jennifer by next week, together with some user stories on how partnerships will be done.

For now, I think I’ll continue reading up on my new Rails book, as well as watch Randy Pausch’s last lecture, which Luke hails as “simply amazing and inspiring”.

Written by Geofrey

August 1st, 2008 at 10:20 pm

Posted in Uncategorized