Showing posts with label rails. Show all posts
Showing posts with label rails. Show all posts

Wednesday, September 1, 2010

Sharing is caring: Ruby, Perl, Memcached and MsgPack

Do you need to share data structures between ruby and perl ... FAST?

I recently saw MsgPack bubble through my RSS feeds and tagged it to go have another look. It provides very fast multi-language bindings for serialisation/de-serialisation.

A few quick experiments with the data I share (via memcached) between ruby and perl showed a write (serialisation + write to memcached) speed increase from 20s to 1.8s. Read (read from memcached + de-serialisation) performance showed similar performance increases.

All initial testing was done ruby -> memcached -> ruby but as soon as I switched to reading from memcached via perl I started getting 'extra bytes' errors from the perl side. I then tried perl -> memcached -> perl and everything was fine.

Weird.

A closer look at the data written to memcached and then read from perl showed that the data serialised with MsgPack on the ruby end was not the same as the data read by perl from memcached (validating the 'extra bytes' error).

Testing the write -> read process from perl to ruby yielded the following error:

/opt/local/lib/ruby/gems/1.8/gems/memcached-0.19.5/lib/memcached/memcached.rb:514:in `load': incompatible marshal file format (can't be read) (TypeError)
format version 4.8 required; 147.1 given
from /opt/local/lib/ruby/gems/1.8/gems/memcached-0.19.5/lib/memcached/memcached.rb:514:in `get'
from ./t_msgpack.rb:35:in `read_test'
from ./t_msgpack.rb:49

Now why on earth would I be getting a 'incompatible marshal file format' error as I am not using the ruby marshalling lib at all?

Turns out the memcached lib I use turns marshalling of ruby data on by default when you write to/read from memcached. This is most likely the best option for most cases where you don't want to use some other form of serialisation/de-serialisation but was really biting me here.

The solution is to simply stop the default behaviour of the memcached lib by using the following forms of get and set that turns on the 'raw' data handling switch for the memcached lib:

get KEY, false
set KEY, VALUE, TTL, false

The 'false' parameter at the end of those overrides the default behaviour turning default serialisation/de-serialisation via Marshall off.

Reality restored.

Sunday, July 5, 2009

Installing the MySQL gem on OS X

If you manage your system packages via the MacPorts system you may run into some problems when trying to install the native MySQL driver gem on OS X.

Typically, you'd see output like this when trying to install the gem:

$ sudo gem install mysql
Building native extensions. This could take a while...
ERROR: Error installing mysql:
ERROR: Failed to build gem native extension.

/opt/local/bin/ruby extconf.rb
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lm... yes
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lz... yes
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lsocket... no
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lnsl... no
checking for mysql_query() in -lmysqlclient... no
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers. Check the mkmf.log file for more
details. You may need configuration options.

Provided configuration options:
--with-opt-dir
--without-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/opt/local/bin/ruby
--with-mysql-config
--without-mysql-config
--with-mysql-dir
--without-mysql-dir
--with-mysql-include
--without-mysql-include=${mysql-dir}/include
--with-mysql-lib
--without-mysql-lib=${mysql-dir}/lib
--with-mysqlclientlib
--without-mysqlclientlib
--with-mlib
--without-mlib
--with-mysqlclientlib
--without-mysqlclientlib
--with-zlib
--without-zlib
--with-mysqlclientlib
--without-mysqlclientlib
--with-socketlib
--without-socketlib
--with-mysqlclientlib
--without-mysqlclientlib
--with-nsllib
--without-nsllib
--with-mysqlclientlib
--without-mysqlclientlib


Gem files will remain installed in /opt/local/lib/ruby/gems/1.8/gems/mysql-2.7 for inspection.
Results logged to /opt/local/lib/ruby/gems/1.8/gems/mysql-2.7/gem_make.out

Based on those errors the configure process seems to be failing when looking for MySQL libs. The default path to the libs based on extconf.rb seems to be /usr/local which is not where the ports system installs MySQL.

All you need to do is point to the correct mysql_config and let the gem install command line know of this to get things going:

$ sudo gem install mysql -- --with-mysql-config=/opt/local/bin/mysql_config5
Building native extensions. This could take a while...
Successfully installed mysql-2.7
1 gem installed

All's well that ends well.

Wednesday, July 1, 2009

Home made nginx + phusion passenger OS X port

The instructions to do this are similar to those in my previous article.

Install passenger gem
The first step is to install the Phusion Passenger gem:
$ sudo gem install -V -r passenger
[...]
Installing RDoc documentation for passenger-2.2.4...

Take note of where the passenger gem installed the nginx module code (ext/nginx/*) that will be required further down. In this case the install directory will be /opt/local/lib/ruby/gems/1.8/gems/passenger-2.2.4/ext/.

Compile nginx passenger module
You also need to compile the passenger nginx module if it was not already compiled:

$ cd /opt/local/lib/ruby/gems/1.8/gems/passenger-2.2.4/ext/
$ sudo rake nginx
[...]
g++ ext/nginx/HelperServer.cpp -o ext/nginx/HelperServer -Iext -Iext/common -D_REENTRANT -I/usr/local/include -Wall -g -DPASSENGER_DEBUG -DBOOST_DISABLE_ASSERTS ext/nginx/libpassenger_common.a ext/nginx/libboost_oxt.a -lpthread


Modify the nginx Portfile
Ensure you have nginx port installed. Your Portfile will most likely be in /opt/local/var/macports/sources/rsync.macports.org/release/ports/www/nginx/Portfile.

Add the following variant block to the end of the file:

variant passenger description {Enable Phusion Passenger (mod_rails) support} {
configure.args-append --add-module=/opt/local/lib/ruby/gems/1.8/gems/passenger-2.2.4/ext/nginx
}


Rebuild/install nginx
When you try and reinstall nginx now it will use the updated Portfile and add the passenger module to the mix:

$ sudo port uninstall nginx && sudo port clean nginx
$ sudo port -vvvvv install nginx +passenger


Nginx passenger configuration
To enable and take advantage of the passenger nginx module you need to edit the /opt/local/etc/nginx/nginx.conf config. Here's an example:

#user nobody;
worker_processes 1;

#error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;

#pid logs/nginx.pid;


events {
worker_connections 1024;
}


http {
include mime.types;
default_type application/octet-stream;

#log_format main '$remote_addr - $remote_user [$time_local] $request '
# '"$status" $body_bytes_sent "$http_referer" '
# '"$http_user_agent" "$http_x_forwarded_for"';

#access_log logs/access.log main;

sendfile on;
#tcp_nopush on;

#keepalive_timeout 0;
keepalive_timeout 65;

gzip on;

# Enable Phusion Passenger nginx module
passenger_root /opt/local/lib/ruby/gems/1.8/gems/passenger-2.2.4;
passenger_ruby /opt/local/bin/ruby;

server {
listen 80;
server_name foo.example.com;
root /var/www/foo/public; # <--- be sure to point to 'public'!
passenger_enabled on;
}
}

Ensure that the port, server_name and root variables have valid values for your environment.

If you want to use a non-production rails environment you can add the following to the block after passenger_enabled:

rails_env development;

Once done you can restart nginx and you should see the following processes that confirm things are running:

$ ps axw | egrep "nginx|passenger"
55935 ?? Ss 0:00.00 nginx: master process nginx
55936 ?? S 0:00.00 nginx: worker process
55925 s001 S 0:00.01 PassengerNginxHelperServer /opt/local/lib/ruby/gems/1.8/gems/passenger-2.2.4 /opt/local/bin/ruby 3 4 0 6 0 300 1 nobody 4294967294 4294967294 /tmp/passenger.55923
55938 s001 R+ 0:00.00 egrep nginx|passenger

The final test to see if everything is working is to open a browser and to browse to the server_name and port defined above. Errors will be reported in /opt/local/var/log/nginx/error.log as well as the development/production logs of the rails project you're running.


Home made nginx + phusion passenger Debian/Ubuntu dep

If you are using Ubuntu Hardy then you could try using Brightbox's repo instead of building your own deb.

If you are however running some other distro then the following (non-definitive) instructions are for you.

Warning: it may be safer to install nginx + passenger via the passenger-install-nginx-module script as it will match passenger with a tested version of nginx.

The instructions below simply add passenger support to the latest nginx deb in your distro of choice.

Install passenger gem
The first step is to install the Phusion Passenger gem:
$ sudo gem install -V -r passenger
[...]
Installing RDoc documentation for passenger-2.2.4...

Take note of where the passenger gem installed the nginx module code that will be required further down:
$ gem contents passenger | egrep '/ext/nginx/' | head -n 1
/var/lib/gems/1.8/gems/passenger-2.2.4/ext/nginx/Configuration.c

In this case the install directory will be /var/lib/gems/1.8/gems/passenger-2.2.4/ext/nginx.

Compile nginx passenger module
You also need to compile the passenger nginx module if it was not already compiled:

$ cd /var/lib/gems/1.8/gems/passenger-2.2.4/ext/nginx
$ sudo rake nginx

(in /var/lib/gems/1.8/gems/passenger-2.2.4)
mkdir -p ext/nginx/libboost_oxt/boost
g++ -Iext -D_REENTRANT -I/usr/local/include -Wall -g -DPASSENGER_DEBUG -DBOOST_DISABLE_ASSERTS -o ext/nginx/libboost_oxt/boost/exceptions.o -c ext/boost/src/pthread/exceptions.cpp
[...]
g++ ext/nginx/HelperServer.cpp -o ext/nginx/HelperServer -Iext -Iext/common -D_REENTRANT -I/usr/local/include -Wall -g -DPASSENGER_DEBUG -DBOOST_DISABLE_ASSERTS ext/nginx/libpassenger_common.a ext/nginx/libboost_oxt.a -lpthread


Install build packages

$ sudo aptitude install fakeroot debhelper dpkg-dev autotools-dev libpcre3-dev libssl-dev


Modify APT sources.list
Open /etc/apt/sources.list and ensure you add deb-src lines for each of the universe repos. The easiest way to do this is to simply copy the universe repo lines that exist and change the starting deb to deb-src:

deb http://au.archive.ubuntu.com/ubuntu/ jaunty universe
deb-src http://au.archive.ubuntu.com/ubuntu/ jaunty universe
deb http://au.archive.ubuntu.com/ubuntu/ jaunty-updates universe
deb-src http://au.archive.ubuntu.com/ubuntu/ jaunty-updates universe

Replace 'au.archive.ubuntu.com' with the repo closest to you and also replace 'jaunty' with your distro. Ensure you do a 'sudo aptitude update' to load the latest package indexes.

Set up a packaging environment

$ mkdir -p /var/tmp/nginx-passenger && cd /var/tmp/nginx-passenger
$ apt-get source nginx
Reading package lists... Done
Building dependency tree
Reading state information... Done
NOTICE: 'nginx' packaging is maintained in the 'Svn' version control system at:
svn://svn.debian.org/svn/collab-maint/deb-maint/nginx/trunk
Need to get 537kB of source archives.
Get:1 http://au.archive.ubuntu.com jaunty/universe nginx 0.6.35-0ubuntu1 (dsc) [1305B]
Get:2 http://au.archive.ubuntu.com jaunty/universe nginx 0.6.35-0ubuntu1 (tar) [525kB]
Get:3 http://au.archive.ubuntu.com jaunty/universe nginx 0.6.35-0ubuntu1 (diff) [11.0kB]
Fetched 537kB in 0s (1504kB/s)
gpg: Signature made Sat 21 Feb 2009 07:02:34 AM EST using DSA key ID 3FE63E00
gpg: Can't check signature: public key not found
dpkg-source: extracting nginx in nginx-0.6.35
dpkg-source: info: unpacking nginx_0.6.35.orig.tar.gz
dpkg-source: info: applying nginx_0.6.35-0ubuntu1.diff.gz
$ ls -la
total 548
drwxr-xr-x 3 root root 4096 2009-06-29 08:55 .
drwxrwxrwt 3 root root 4096 2009-06-29 08:39 ..
drwxr-xr-x 8 root root 4096 2009-06-29 08:55 nginx-0.6.35
-rw-r--r-- 1 root root 11018 2009-02-21 08:04 nginx_0.6.35-0ubuntu1.diff.gz
-rw-r--r-- 1 root root 1305 2009-02-21 08:04 nginx_0.6.35-0ubuntu1.dsc
-rw-r--r-- 1 root root 524987 2009-02-21 08:04 nginx_0.6.35.orig.tar.gz
$ rm -f nginx_*
$ mv nginx-0.6.35 nginx-passenger-0.6.35


Package customizations
In order to build our nginx-passenger custom package we need to modify the following packaging resources:

  1. control file

  2. changelog file

  3. rules file

  4. nginx.install file

  5. nginx.logrotate file

  6. passenger.conf file

  7. postinst file



Control file

$ cd nginx-passenger-0.6.35/debian

Edit the control file and change the Source, Maintainer, Uploaders, Package, Provides, Conflicts and Description fields:

Source: nginx-passenger
Maintainer: Charl Matthee
Uploaders: Charl Matthee
Package: nginx-passenger
Provides: httpd, nginx
Conflicts: nginx
Description: small, but very powerful and efficient web server
Nginx (engine x) is a web server created by Igor Sysoev and kindly provided
to the open-source community. This server can be used as standalone HTTP
server and as a reverse proxy server before some Apache or another big
server to reduce load to backend servers by many concurrent HTTP-sessions.
.
This is a custom nginx package that includes Phusion Passenger
support for nginx.

Note: spacing in the control file is significant.

More information on the control file internals can be found here.

Changelog
Add a new changelog entry in the changelog file:

nginx-passenger (0.6.35-0custom1) jaunty; urgency=low

* New custom Passenger Phusion nginx module addition

-- Charl Matthee Mon, 29 Jun 2009 09:08:36 +0100

nginx (0.6.35-0ubuntu1) jaunty; urgency=low

* New upstream bugfix release

-- Daniel Hahler Fri, 20 Feb 2009 21:01:23 +0100
[...]

More information on the changelog file internals can be found here.

Rules
Add the passenger module inclusion to the configure script options when nginx is built by modifying the './configure $(CONFIGURE_OPTS)' block in the rules file like this:

./configure $(CONFIGURE_OPTS) --conf-path=/etc/nginx/nginx.conf \
--error-log-path=/var/log/nginx/error.log --pid-path=/var/run/nginx.pid \
--lock-path=/var/lock/nginx.lock --http-log-path=/var/log/nginx/access.log \
--http-client-body-temp-path=/var/lib/nginx/body --http-proxy-temp-path=/var/lib/nginx/proxy \
--http-fastcgi-temp-path=/var/lib/nginx/fastcgi --with-http_stub_status_module \
--with-http_flv_module --with-http_ssl_module --with-http_dav_module \
--with-http_realip_module --add-module=PASSENGER_LIB_DIR

Where PASSENGER_LIB_DIR is the path you noted above when you installed the passenger gem. So, the updated block looks like this:

./configure $(CONFIGURE_OPTS) --conf-path=/etc/nginx/nginx.conf \
--error-log-path=/var/log/nginx/error.log --pid-path=/var/run/nginx.pid \
--lock-path=/var/lock/nginx.lock --http-log-path=/var/log/nginx/access.log \
--http-client-body-temp-path=/var/lib/nginx/body --http-proxy-temp-path=/var/lib/nginx/proxy \
--http-fastcgi-temp-path=/var/lib/nginx/fastcgi --with-http_stub_status_module \
--with-http_flv_module --with-http_ssl_module --with-http_dav_module \
--with-http_realip_module --add-module=/var/lib/gems/1.8/gems/passenger-2.2.4/ext/nginx/

More infomration on the changelog file internals can be found here.

nginx.install
Move this file to the correct name based on the package name and add the installation instructions for the default nginx configuration that is required to turn passenger support on:

$ mv nginx.install nginx-passenger.install
$ cat nginx-passenger.install
objs/nginx usr/sbin
html/* var/www/nginx-default
debian/conf/* etc/nginx
debian/passenger.conf etc/nginx/conf.d/
debian/NEWS.Debian usr/share/doc/nginx


nginx.logrotate
Move this file to the correct name based on the package name:

$ mv nginx.logrotate nginx-passenger.logrotate


passenger.conf
Create the following file in the debian directory:

passenger_root /var/lib/gems/1.8/gems/passenger-2.2.4;
passenger_ruby /usr/bin/ruby1.8;

Ensure those paths point to the correct passenger isntall and ruby directories.

postinst
Add the following steps to the postinst script after the '#DEBHELPER#' token:

# Phusion Passenger module settings for nginx
echo
echo "***********************************************************************"
echo "For each virtual host you want to add Phusion Passenger support for you"
echo "need to create a config like this in /etc/nginx/site-available:"
echo
echo " server {
listen 80;
server_name DOMAIN_NAME;
root /var/www/RAILS_PROJECT/public; #<--- be sure to point to 'public'!
passenger_enabled on;
}"
echo
echo "Don't forget to add a symlink from this file(s) to"
echo "/etc/nginx/site-enabled to enable the configuration in nginx."
echo "***********************************************************************"
echo

This will warn administrators that install this package that they still need to do some manual work to get a virtual host set up to take advantage of the passenger module.

Package rebuild
The package can then be rebuilt just like any other custom deb:

$ cd /var/tmp/nginx-passenger/nginx-passenger-0.6.35
$ dpkg-buildpackage -rfakeroot
dpkg-buildpackage: set CFLAGS to default value: -g -O2
[...]
dpkg-deb: building package `nginx-passenger' in `../nginx-passenger_0.6.35-0custom1_i386.deb'.
dpkg-deb: ignoring 1 warnings about the control file(s)
signfile nginx-passenger_0.6.35-0custom1.dsc
gpg: skipped "Charl Matthee ": secret key not available
gpg: [stdin]: clearsign failed: secret key not available

dpkg-genchanges >../nginx-passenger_0.6.35-0custom1_i386.changes
dpkg-genchanges: warning: the current version (0.6.35-0custom1) is smaller than the previous one (0.6.35-0ubuntu1)
dpkg-genchanges: including full source code in upload
dpkg-buildpackage: full upload; Debian-native package (full source is included)
dpkg-buildpackage: warning: Failed to sign .dsc and .changes file

Those signing errors were triggered because I have did not have the required key infrastructure installed. They can be ignored for the moment.

Install the new package:

$ sudo dpkg -i ../nginx-passenger_0.6.35-0custom1_i386.deb
[...]
Setting up nginx-passenger (0.6.35-0custom1) ...

***********************************************************************
Create a config like this for each virtual host you want to add Phusion
Passenger support to. They should be created in
/etc/nginx/site-available:

server {
listen 80;
server_name DOMAIN_NAME;
root /var/www/RAILS/public; # <--- be sure to point to 'public'!
passenger_enabled on;
}

Don't forget to add a symlink from this file(s) to
/etc/nginx/site-enabled to enable the configuration in nginx.
***********************************************************************
[...]


Nginx passenger configuration
To enable and take advantage of the passenger nginx module you need to create a /etc/nginx/conf.d/passenger.conf config:

passenger_root /var/lib/gems/1.8/gems/passenger-2.2.4;
passenger_ruby /usr/bin/ruby1.8;

This will of course have been done when we installed our custom package.

Next, add a virtual host for each of the rails applications you want to support by dropping a file like 'foo' into /etc/nginx/sites-available/:

server {
listen 80;
server_name foo.example.com;
root /var/www/foo/public; # <--- be sure to point to 'public'!
passenger_enabled on;
}

Ensure that the port, server_name and root variables have valid values for your environment.

If you want to use a non-production rails environment you can add the following to the block after passenger_enabled:

rails_env development;


To enable the virtual host(s) you need to ensure that there's a symlink linking the virtual host in sites-available to sites-enabled:

$ sudo ln -s /etc/nginx/sites-available/foo /etc/nginx/sites-enabled/foo
$ ls -la /etc/nginx/sites-enabled
total 8
drwxr-xr-x 2 root root 4096 2009-06-29 12:33 .
drwxr-xr-x 5 root root 4096 2009-06-29 10:51 ..
lrwxrwxrwx 1 root root 34 2009-06-02 07:25 default -> /etc/nginx/sites-available/default
lrwxrwxrwx 1 root root 30 2009-06-29 12:33 foo -> /etc/nginx/sites-available/foo

Once done you can restart nginx and you should see the following processes that confirm things are running:

$ ps axf | egrep "nginx|passenger"
28608 pts/1 S+ 0:00 \_ egrep nginx|passenger
28560 pts/1 Sl 0:00 PassengerNginxHelperServer /var/lib/gems/1.8/gems/passenger-2.2.4 /usr/bin/ruby1.8 3 4 0 6 0 300 1 nobody 33 33 /tmp/passenger.28557
28580 ? Ss 0:00 nginx: master process /usr/sbin/nginx
28581 ? S 0:00 \_ nginx: worker process

The final test to see if everything is working is to open a browser and to browse to the server_name and port defined above. Errors will be reported in /var/log/nginx/error.log as well as the development/production logs of the rails project you're running.


Sunday, February 22, 2009

Silly Rails Environment Optimisation

Do you find supplying the ./script base directory for any of the commands you want to run in it annoying?

Why not practice some Convention over Configuration and assume these commands will always be run from within a valid rails project directory.

Based on this assumption you can drop the following into your .bash_profile (OS X) or some system-wide profile to facilitate some laziness:
# Rails aliases
alias about="./script/about"
alias console="./script/console"
alias dbconsole="./script/dbconsole"
alias destroy="./script/destroy"
alias generate="./script/generate"
alias performance="./script/performance"
alias plugin="./script/plugin"
alias process="./script/process"
alias runner="./script/runner"
alias server="./script/server"
Don't forget to load it into you current shell with:
$ . ~/.bash_profile
Now you can simply run 'console', 'dbconsole', 'server', etc. when you're in a valid rails project directory without having to append that annoying base directory.

Caveat: If similar commands exist on your system or are introduced by a non-rails package a conflict will arise and the alias version of the command will win. Also, TAB-completion now no longer works for these commands and you'll have to type them out in their entirety.


Friday, February 20, 2009

What's next with acts_as_state_machine?

I've been using acts_as_state_machine and there seems to be at least two convenience methods missing that I find quite useful:
  1. next_state - what state will we be transitioning to
  2. next_states - list of states that we still need to transition to
Drop the following in a suitable place (your model using acts_as_state_machine will do) and abuse it to your hearts content:
# Monkey patch acts_as_state_machine to allow us to see what the next states are
# for the current state
module ScottBarron
module Acts
module StateMachine
module InstanceMethods
def next_state(state=nil)
state ||= current_state()
self.class.read_inheritable_attribute(:transition_table).each_value do |event|
event.each do |transition|
return transition.to if transition.from == state
end
end
nil
end

def next_states(state=nil)
state ||= current_state()
states = []
while state = next_state(state)
states << state
end
states
end
end
end
end
end
Thanks to Lourens Naude's' post on acts_as_state_machine for providing me with some ideas and to Scott Barron for the great acts_as_state_machine plugin.


Wednesday, November 26, 2008

Massaging Rails Models (with a happy ending)

How do you alter data in a model so that the data which is stored to and gathered from the database is first filtered/transformed?

Two ways come to mind:

  • Insert the required behavior into your model's before_save, before_create and after_initialize callbacks.
  • Manually modify your attribute accessors for the attributes in question to do the magic for you.

We'll use the following contrived Model as our example:

class Gogga < ActiveRecord::Base
end

CREATE TABLE  `foo`.`goggas` (
`id` int(11) NOT NULL auto_increment,
`secret` varchar(255) default NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=latin1

A Gogga has an id and a secret field. Let's pretend we need to keep Gogga.secret encrypted (using the super secret ROT13 algorithm) in our db and we would like the fact that it is encrypted to be transparent to our rails app. We need to therefore handle en/decryption of the secret data transparently from the rest of the app within the model.

Callbacks
The before_save, before_create and after_initialize callbacks are well documented in the Callbacks API documentation.

The strategy behind using the callbacks is to simply insert the behavior we want at the relevant stage of the object's life cycle. Here's one way to accomplish this using the mentioned callbacks:

class Gogga < ActiveRecord::Base
def before_save
self.secret = rot13(self.secret)
end

def before_create
self.secret = rot13(self.secret)
end

def after_initialize
self.secret = rot13(self.secret)
end

protected
def rot13(corpus)
return corpus.tr!("A-Za-z", "N-ZA-Mn-za-m")
end
end

If everything works as advertised our secret attribute should now be encoded when you call create or save on its model and decoded when you call a new on its model. The major drawback of this strategy is that if you are manipulating a large list of Goggas you will be post/pre-processing each of those instances.

The Lazy Way
An alternative would be to override the default behavior of the model to auto-generate attribute accessors via method_missing in the mystical black guts of ActiveRecord::Base. There's some info on this in the API docs as well in the Overwriting default accessors section.

This would look something like this:

class Gogga < ActiveRecord::Base
def secret
return rot13(read_attribute("secret"))
end

def secret=(value)
write_attribute("secret", rot13(value))
end

protected
def rot13(corpus)
return corpus.tr!("A-Za-z", "N-ZA-Mn-za-m")
end
end

This technique has the advantage that you never really mess with the internals of the model (as the view you have of it from outside is tinted by the accessor transformations) and of course work only gets done when you need to read/write the specific attribute.

Now, when you have one or two attributes that need to be protected writing out two accessors for each is not the end of the world. However, when you have several things become messy, tedious and downright boring.

Maybe we can look at some meta-magic to DRY things up a bit in another article.


Sunday, August 12, 2007

Prototype, IE and Edge Rails Failures

I recently wrote a little conceptual file upload application with scaffold_resource that used the iframe remoting pattern with some baked-in AJAX goodness to minimise the amount of data returned from the server as well as making the UI a lot more snappy.

Everything went well and tested a-OK in Firefox. Unfortunately my default development platform does not support IE so I only tested the app in IE a little later. To my horror IE rendered the page differently as well as spewing the following JS error:

Line: 1629
Char: 9
Error: Invalid target element for this operation.
Code: 0
URL: <REMOVED>

Prototype goodness wherefor art thou?
A few concise questions to the Oracle of Google and I found a thread started by Rob Sanheim which detailed the same problem I was seeing.

A fix, that worked for many of the thread readers, was proposed by Andy (12 December 2005 @ 1pm) in the same thread to deal with the way in which prototype inserts content in a tbody or tr tag.

So, off I went and upgraded my app to the latest Edge Rails using:

rake rails:freeze:edge

Edge Rails will you be the end of me?

Starting my app up I got a similar nasty to this:

/Applications/Locomotive2/Bundles/standardRailsFeb2007.locobundle/i386/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require': no such file to load -- active_resource (MissingSourceFile) from /Applications/Locomotive2/Bundles/standardRailsFeb2007.locobundle/i386/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `require' from /Users/joelmeyer/src/FotoDir/vendor/rails/activerecord/lib/../../activesupport/lib/active_support/dependencies.rb:495:in `require' from /Users/joelmeyer/src/FotoDir/vendor/rails/activerecord/lib/../../activesupport/lib/active_support/dependencies.rb:342:in `new_constants_in' from /Users/joelmeyer/src/FotoDir/vendor/rails/activerecord/lib/../../activesupport/lib/active_support/dependencies.rb:495:in `require' from ./config/../vendor/rails/railties/lib/initializer.rb:160:in `require_frameworks' from ./config/../vendor/rails/railties/lib/initializer.rb:160:in `each' from ./config/../vendor/rails/railties/lib/initializer.rb:160:in `require_frameworks' from ./config/../vendor/rails/railties/lib/initializer.rb:88:in `process' ... 8 levels... from /Applications/Locomotive2/Bundles/standardRailsFeb2007.locobundle/i386/lib/ruby/gems/1.8/gems/capistrano-1.4.1/lib/capistrano/cli.rb:12:in `execute!' from /Applications/Locomotive2/Bundles/standardRailsFeb2007.locobundle/i386/lib/ruby/gems/1.8/gems/capistrano-1.4.1/bin/cap:11 from /Applications/Locomotive2/Bundles/standardRailsFeb2007.locobundle/i386/bin/cap:16:in `load' from /Applications/Locomotive2/Bundles/standardRailsFeb2007.locobundle/i386/bin/cap:16

Seems that when you're using Edge Rails you need to use the active resource gem in tandem or you world will turn pear-shaped. Installing the gem, with dependancies did the trick:

gem install -y activeresource --source http://gems.rubyonrails.org

IE: Dark side of the moon
So, I have now upgraded to the latest prototype lib (via the Edge Rails upgrade) and got Edge Rails to run with the active_scaffold and active resources I was using in my app.

Was everything fine then and did I get to live a more fulfilling existence? NAY!

Back to the entry by Rob Sanheim and I noticed that Chris Nolan reported that the prototype lib had been fixed but that he was still experiencing the same issue.

There is a boat and I feel that Chris and I are both in it together, brothers in misery.

While debugging the issue with trusty old JS alert() he found that he was trying to insert content at an id that belonged to a table tag which was not supported. The tags supported for this were of course tbody and/or tr.

Even though the tbody tag seems to be optional according to the W3C IE still does not allow you to insert the content based on a table id.

The simple addition of the required tbody tags cured all.

Tuesday, July 17, 2007

Links

Sequel
Sequel is a light-weight ORM that fills in the gaps where using ActiveRecord without rails doesn't fit your needs.

Gregory Brown has a nice little expose on it as the winner of the June 2007 Ruby Project Spotlight.

Introduction to .NET 3.0 for Architects
Keep your friends close and competition^H^H^H^H^H^H^H^H^H^H alternative platforms closer. InfoQ has a great 50000 foot look at .NET 3.0.

Profiling your ails app with ruby-prof
Charlie Savage wrote a great article on doing some profiling on your rails app using ruby-prof. He discusses using both flat and graph (with associated call tree information) profiles to nuke performance hogs.

ruby-prof is a fast code profiler for Ruby. Its features include:
  • Speed - it is a C extension and therefore many times faster than the standard Ruby profiler.
  • Flat Profiles - similar to the reports generated by the standard Ruby profiler
  • Graph profiles - similar to GProf, these show how long a method runs, which methods call it and which methods it calls
  • Threads - supports profiling multiple threads simultaneously
  • Recursive calls - supports profiling recursive method calls
  • Reports - can generate both text and cross-referenced html reports
  • Output - can output to standard out or to a file

AR-Delegation
This plugin extends ActiveRecord::Base to add useful delegation features. For example: has_columns :from => :source, :only => ["title", "name"] has_column "title", :from => :source, :as => "source_title".

It really improves the conciseness of your code but in so doing hides your implementation adding a layer of indirection that may make your code a little more difficult to understand if the person reading your code does not know that AR-Delegation was used.

Exception Notifier
I use this with most of my projects that reside on remote customer networks where the only way for the application to give me a heads up is if it sends me an email with an attached problem report.

Friday, March 30, 2007

(Almost) Overriding helpers in rails

In a project I am currently working on I use the select_datetime() helper to provide me with year + month + day + hour + minute drop down boxes. After using the UI for a while the customer mentioned that they do not require the full minute listing (from 00 to 59 minutes) for the minute drop down box and would prefer to only have 15 minute increments to shorten the drop down and waste less scrolling time when choosing the minutes.

Umm ... OK!

This type of functionality can be provided in a few ways (that I
know of ):
  • Wrapper: Create your own helper wrapper
  • Replace: Replace the original helper method in the module where it comes from
  • RTFM: Read the docs first and then code
select_datetime() is actually a composition of two other helpers:
select_time() is another composition of:
Because we're simply interested in changing the behaviour of the minutes we will only be changing select_minute().

Wrapper
This approach is pretty straight forward. All you need to do is add your own helper as a wrapper for the helper you wish to masquerade. Unfortunately we cannot use this technique here because the helper we're using (select_datetime()) cannot provide us with the relevant functionality we want by simply massaging the helper's input or output.

Bummer.

Replace
Your intuition may be itching at this point and that annoying inner voice you never ignore could be saying:

'Why don't you just simply override the select_datetime() in application_helper.rb or ActionView::Base class, lilly-livered limp-wristed sissy!'.

In this case your inner voice needs to be muzzled. Helpers are included as mixins. This means the methods in the mixed in module become available to the classes using them as if they were part of the actual class.

The PickAxe book says it succinctly:

'In fact, mixed-in modules effectively behave as superclasses.'

Unfortunately this also means that if you change the methods in a module they will change for all classes that use the mixed-in module. If you want to change the helper behaviour you need to change the behaviour in the module that is providing the helper as a mixin, in this case ActionView::Helpers::DateHelper.

Off we go and open ActionView::Helpers::DateHelper to start mucking with select_minute() so that we can support periods as the customer requested. Here is the relevant code:
# File vendor/rails/actionpack/lib/action_view/helpers/date_helper.rb, line 190

190: def select_minute(datetime, options = {})
191: val = datetime ? (datetime.kind_of?(Fixnum) ? datetime : datetime.min) : ''
192: if options[:use_hidden]
193: hidden_html(options[:field_name] || 'minute', val, options)
194: else
195: minute_options = []
196: 0.step(59, options[:minute_step] || 1) do |minute|
197: minute_options << ((val == minute) ? 198: %(#{leading_zero_on_single_digits(minute)}\n) : 199: %(#{leading_zero_on_single_digits(minute)}\n) 200: ) 201: end 202: select_html(options[:field_name] || 'minute', minute_options, options) 203: end 204: end

Alrighty! Let's add that stepping code to provide the periodic minutes ...

RTFM
Damn! Had I spent a little time drilling down through the documentation I would have noticed that the select_minute() helper already provides the required functionality via its :minute_step option.

Here is what the documentation says about this helper:
select_minute(datetime, options = {})

Returns a select tag with options for each of the minutes 0 through 59 with the
current minute selected. Also can return a select tag with options by minute_step
from 0 through 59 with the 00 minute selected The minute can also be substituted
for a minute number. Override the field name using the :field_name option, 'minute'
by default.

So, all of this may seem a little pointless but the lesson here is check the docs before you take-off at light speed. I look forward to writing an actual article on overriding helpers in rails soon.

About Me

My photo
I love solving real-world problems with code and systems (web apps, distributed systems and all the bits and pieces in-between).