Broken memcached gem/libmemchached port on OS X

Update (2009/04/22):
This post was originally only targeted at OS X but I have since noticed Ubuntu also having this same issue. Jump to my additional instructions for Ubuntu.

Recipe for disaster:

1. Use MacPorts for your package management
2. Use the ruby memcached gem
3. Use the libmemcached port as a dependency on the gem above
4. Upgrade your ports

Synopses
I upgraded all my ports on my MBP recently and one of the libs that was upgraded in the process was libmemcached (upgraded from v0.25 to v0.26). Unfortunately this has broken my access to memcached on my system (the script dies when trying to connect).

Not knowing what the problem may be I also upgraded my gem to v0.14 which then could not build. A quick search for the issue found this entry on the memcached discussion forum.

How now brown cow?
Evan Weaver, the maintainer of the memcached project, mentioned that libmemcached v0.26 lacks some critical patch and suggested people may have to wait for v0.27 to rectify things.

For those who cannot wait and have inconsistent, broken systems, Evan goes on to mention that you should currently use the libmemcached lib and memcached gem pair from here.

Here he also provides a convenient compatibility matrix so you can see how to align the different versions of libmemcached and the memcached gem.

Fix
Unfortunately this messes with things because libmemcached now needs to be installed as a source package in a system managed with MacPorts.

Yuck!

Unfortunately the choices are limited as the versions of libmemcached that ships as a port are either v0.26 (the latest, but broken for our purposes) or v0.25 (which aligns with the older v0.13 of the memcached gem).

First off remove any libmemcached ports you may have installed:

$ sudo port uninstall libmemcached

Download the libmemcached source archive, extract, build and install it:

ibm-99tvvxc:msp charl$ cd /var/tmp/
$ wget http://blog.evanweaver.com/files/libmemcached-0.25.14.tar.gz
--2009-02-24 10:50:40--  http://blog.evanweaver.com/files/libmemcached-0.25.14.tar.gz
Resolving blog.evanweaver.com... 208.78.102.192
Connecting to blog.evanweaver.com|208.78.102.192|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 428047 (418K) [application/x-gzip]
Saving to: `libmemcached-0.25.14.tar.gz'

100%[==========================================================>] 428,047      135K/s   in 3.1s 

2009-02-24 10:50:49 (135 KB/s) - `libmemcached-0.25.14.tar.gz' saved [428047/428047]

$ tar zxvf libmemcached-0.25.14.tar.gz
libmemcached-0.25.14/
[...]
$ cd libmemcached-0.25.14
$ ./configure
checking build system type... i386-apple-darwin9.6.0
checking host system type... i386-apple-darwin9.6.0
checking target system type... i386-apple-darwin9.6.0
checking for a BSD-compatible install... /usr/bin/install -c
[...]
$ make
Making all in docs
/usr/bin/pod2man -c "libmemcached" -r "" -s 3 libmemcached.pod > libmemcached.3
/usr/bin/pod2man -c "libmemcached" -r "" -s 3 libmemcached_examples.pod > libmemcached_examples.3
[...]
$ sudo make install
Password:
Making install in docs
make[2]: Nothing to be done for `install-exec-am'.
test -z "/usr/local/share/man/man1" || ../config/install-sh -c -d "/usr/local/share/man/man1"
/usr/bin/install -c -m 644 './memcat.1' '/usr/local/share/man/man1/memcat.1'
[...]

You may want to keep the source directory around so that you can easily uninstall (run sudo make uninstall in the source directory) the source installation later when the relevant aligned packaged versions are available for installation.

The final step is to install the gem:

$ sudo env ARCHFLAGS="-arch i386" gem install -r -V memcached --no-rdoc --no-ri
GET 200 OK: http://gems.rubyforge.org/latest_specs.4.8.gz
GET 200 OK: http://gems.github.com/latest_specs.4.8.gz
connection reset after 2 requests, retrying
GET 200 OK: http://gems.rubyforge.org/quick/Marshal.4.8/memcached-0.14.gemspec.rz
Installing gem memcached-0.14
/opt/local/lib/ruby/gems/1.8/gems/memcached-0.14/BENCHMARKS
/opt/local/lib/ruby/gems/1.8/gems/memcached-0.14/CHANGELOG
[...]
Successfully installed memcached-0.14
1 gem installed

Success!

We now have libmemcached v0.25.14 and the memcached v0.14 gem installed and we can go on with our lives.

Silly Rails Environment Optimisation

Do you find supplying the ./script base directory for any of the commands you want to run in it annoying?

Why not practice some Convention over Configuration and assume these commands will always be run from within a valid rails project directory.

Based on this assumption you can drop the following into your .bash_profile (OS X) or some system-wide profile to facilitate some laziness:

# Rails aliases
alias about="./script/about"
alias console="./script/console"
alias dbconsole="./script/dbconsole"
alias destroy="./script/destroy"
alias generate="./script/generate"
alias performance="./script/performance"
alias plugin="./script/plugin"
alias process="./script/process"
alias runner="./script/runner"
alias server="./script/server"

Don't forget to load it into you current shell with:

$ . ~/.bash_profile

Now you can simply run 'console', 'dbconsole', 'server', etc. when you're in a valid rails project directory without having to append that annoying base directory.

Caveat: If similar commands exist on your system or are introduced by a non-rails package a conflict will arise and the alias version of the command will win. Also, TAB-completion now no longer works for these commands and you'll have to type them out in their entirety.

What's next with acts_as_state_machine?

I've been using acts_as_state_machine and there seems to be at least two convenience methods missing that I find quite useful:

1. next_state - what state will we be transitioning to
2. next_states - list of states that we still need to transition to

Drop the following in a suitable place (your model using acts_as_state_machine will do) and abuse it to your hearts content:

# Monkey patch acts_as_state_machine to allow us to see what the next states are
# for the current state
module ScottBarron
 module Acts
   module StateMachine
     module InstanceMethods
         def next_state(state=nil)
           state ||= current_state()
           self.class.read_inheritable_attribute(:transition_table).each_value do |event|
             event.each do |transition|
               return transition.to if transition.from == state 
             end
           end
           nil
         end
        
         def next_states(state=nil)
           state ||= current_state()
           states = [] 
           while state = next_state(state)
             states << state            
           end            
           states          
        end       
      end    
    end  
  end
end 

Thanks to Lourens Naude's' post on acts_as_state_machine for providing me with some ideas and to Scott Barron for the great acts_as_state_machine plugin.

Signing your own gems

Chapter 9 (Signing Your Gems) of the RubyGems User Guide contains everything you need to know to do this.

Here's a quick overview of the steps required.

Build a certificate and private key pair

$ cd /to/where/you/wish/to/keep/your/cert_and_key
$ gem cert --build gemmaster@example.com

Two files are generated from this: gem-private_key.pem & gem-public_cert.pem.

Ensure you keep gem-private_key.pem somewhere safe that only you have access to.

Modify your build specification
Add the following spec to your existing gem's gemspec or to the relevant Rakefile ("s" is your Gem::Specification instance):

$ s.signing_key = 'gem-private_key.pem'
$ s.cert_chain  = ['gem-public_cert.pem']

Rebuild your package and you're done!

Your own gem repository

Setting up your own repository is quite simple. All you need is the following:

• Web server
• Builder gem
• A repository
• Client setup

Web server
I used nginx but you should be able to tailor this to your needs without too much effort.

Create a directory to contain your local gem repository:

$ sudo mkdir -p /var/www/gems-repo/gems

Note: to get everything working correctly you need to ensure you have that /gems subdirectory off your base directory configured in nginx. You can find more info on this at the RubyGems FAQ.

Add a virtual host config in /etc/nginx/sites-available for your gems repository (I'm going to assume gems.example.com):

$ cat /etc/nginx/sites-available/gems.example.com
server {
 listen   80;
 server_name  gems.example.com;
 access_log  /var/log/nginx/gems.example.com.access.log;
 root /var/www/gems-repo;
}

Ensure you drop a symlink in /etc/nginx/sites-enabled/ for your virtual host and restart nginx.

Install the builder gem

$ sudo gem install builder

Build your repository
Jump into the base directory defined above (/var/www/gems-repo). Copy all your own gems into /var/www/gems-repo/gems/. Then generate the relevant resource files for rubygems:

$ sudo gem generate_index -d /var/www/gems-repo
Loading 1 gems from /var/www/gems-repo
.
Loaded all gems
Generating quick index gemspecs for 1 gems
.
Complete
Generating specs index
Generating latest specs index
Generating quick index
Generating latest index
Generating Marshal master index
Generating YAML master index for 1 gems (this may take a while)
.
Complete
Compressing indicies

You would re-run this every time you add to or make changes to the repository.

Client setup
Finally run the following on all local clients that need access to the repository:

$ sudo gem sources -a http://gems.example.com/

You can ensure the extra source was added by running 'gem sources'. If you now do a search for you gem that you added above you should see it listed as a target.