Liquid Media's Apps

Ruby 1.9, Rails, and unicode nightmares

Although they tell you that Rails (2.3) is ready for Ruby 1.9, the reality is that few of the gems are. On a particular project where we really wanted to use Ruby 1.9's new M17N and I18N functionality, I didn't actually run into too many problems with most of the gems I use, but for one, VERY painful one: the postgres(/pg/ruby-pg) gem.

Interestingly enough, the gems work just fine until you pull some UTF-8 content from the database. The existing database adapters (and mysql is included in this) return that content as encoding ASCII-8BIT (aka BINARY). As soon as you try to show that content on a page that is otherwise encoded UTF-8, you get incompatible character encodings: ASCII-8BIT and UTF-8.

This is definitely a problem with the database drivers, which should be returning the same encoding as the database. This problem hasn't officially been fixed yet, but there has been some effort to that end. For postgres, you'll need to use ruby-pg (rather than postgres or pg, even though they're somehow the same, yet different... if anyone cares to explain...?) along with a patch that's referenced in an excellent ticket on the topic. I know this works on Mac OS X and on Fedora 8. For Windows, YMMV:

# do everything as root (it's just easier this way)
sudo bash

# get rid of the pg if you have it already; you might want to do this for ruby-pg as well
/usr/local/bin/gem uninstall pg

# change to some convenient working directory
cd /usr/local/src
svn checkout
cd svn/ruby-pg/trunk/ext

# download and install the patch
curl -O
patch < pg-m17n3.patch
# should respond indicating that all patches successfully applied

# set up an important environment variable
export ARCHFLAGS='-arch i386'

/usr/local/bin/ruby extconf.rb
make install

Worked for me. Let me know in the comments if I need to make a change to the instructions above.


It strikes me that if you run into any troubles applying the patch, then these instructions may be out-of-date, and you can probably just go ahead and install the latest gem, assuming it's something higher than 0.8.0.

Tagged postgres, rails, ruby19, and unicode.
blog comments powered by Disqus