Sunday, July 29, 2007

Tracking fast-paced packages on debian based systems (aka debian-volatile project)

debian-volatile
If you run some ISP services (your own mail server with virus and/or spam scanning tools) you will have run into the age old problem that the scanning tools in the stable distribution do not evolve as fast as they should to keep up with their fast-paced projects.

Even continual updates of the software in your distribution are not enough to stay up to date as the release cycle of the stable distribution is out of sync with the speed at which things change in the wild.

According to the debian-volatile project page:

The main goal of volatile is allowing system administrators to update their systems
in a nice, consistent way, without getting the drawbacks of using unstable, even
without getting the drawbacks for the selected packages. So debian-volatile will
only contain changes to stable programs that are necessary to keep them functional.

volatile-sloppy
Great effort goes into ensuring that no functional changes are made to packages in debian-volatile (so that configuration file changes, etc. are not required) for painless upgrades. Unfortunately painful upgrades are not always avoidable so a volatile-sloppy section was created to contain packages that are fast-paced but also require some functional change to how it runs, is installed or configured.

Security
You should note that the debian-volatile project is not supported by the _official_ security team. This responsibility falls to the debian-volatile team who currently has at least one member that is shared with the official debian testing security team.

How do I use it?
Add the relevant repository (volatile and/or volatile-sloppy) to your /etc/apt/sources.list file:

Sarge
deb http://volatile.debian.org/debian-volatile sarge/volatile main contrib non-free
deb http://volatile.debian.org/debian-volatile sarge/volatile-sloppy main contrib non-free

Etch
deb http://volatile.debian.org/debian-volatile etch/volatile main contrib non-free
deb http://volatile.debian.org/debian-volatile etch/volatile-sloppy main contrib non-free


Save sources.list and run _atp-get update_ which should generate something like this (your listing will vary depending on the repositories you have listed in your sources file):

# apt-get update
Get:1 http://archive.ubuntu.com dapper Release.gpg [189B]
Get:2 http://us.archive.ubuntu.com dapper Release.gpg [189B]
Get:3 http://us.archive.ubuntu.com dapper-backports Release.gpg [191B]
Get:4 http://archive.ubuntu.com dapper-updates Release.gpg [191B]
Get:5 http://volatile.debian.org etch/volatile Release.gpg [189B]
Hit http://us.archive.ubuntu.com dapper Release
Hit http://archive.ubuntu.com dapper Release
Get:6 http://volatile.debian.org etch/volatile Release [40.7kB]
Hit http://archive.ubuntu.com dapper-updates Release
Hit http://us.archive.ubuntu.com dapper-backports Release
Hit http://archive.ubuntu.com dapper/main Packages
Hit http://archive.ubuntu.com dapper/restricted Packages
Hit http://us.archive.ubuntu.com dapper/universe Packages
Hit http://us.archive.ubuntu.com dapper/universe Sources
Hit http://archive.ubuntu.com dapper/main Sources
Hit http://archive.ubuntu.com dapper/restricted Sources
Hit http://archive.ubuntu.com dapper-updates/main Packages
Hit http://archive.ubuntu.com dapper-updates/restricted Packages
Hit http://us.archive.ubuntu.com dapper-backports/main Packages
Hit http://us.archive.ubuntu.com dapper-backports/restricted Packages
Hit http://us.archive.ubuntu.com dapper-backports/universe Packages
Hit http://archive.ubuntu.com dapper-updates/main Sources
Hit http://archive.ubuntu.com dapper-updates/restricted Sources
Hit http://us.archive.ubuntu.com dapper-backports/multiverse Packages
Hit http://us.archive.ubuntu.com dapper-backports/main Sources
Hit http://us.archive.ubuntu.com dapper-backports/restricted Sources
Hit http://us.archive.ubuntu.com dapper-backports/universe Sources
Hit http://us.archive.ubuntu.com dapper-backports/multiverse Sources
Ign http://volatile.debian.org etch/volatile Release
Get:7 http://volatile.debian.org etch/volatile/main Packages [3953B]
Hit http://volatile.debian.org etch/volatile/contrib Packages
Hit http://volatile.debian.org etch/volatile/non-free Packages
Get:8 http://security.ubuntu.com dapper-security Release.gpg [191B]
Hit http://security.ubuntu.com dapper-security Release
Hit http://security.ubuntu.com dapper-security/main Packages
Hit http://security.ubuntu.com dapper-security/restricted Packages
Hit http://security.ubuntu.com dapper-security/main Sources
Hit http://security.ubuntu.com dapper-security/restricted Sources
Hit http://security.ubuntu.com dapper-security/universe Packages
Hit http://security.ubuntu.com dapper-security/universe Sources
Fetched 44.8kB in 5s (7554B/s)
Reading package lists... Done
W: GPG error: http://volatile.debian.org etch/volatile Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY EC61E0B0BBE55AB3
W: You may want to run apt-get update to correct these problems
#

The inclusion of the debian-volatile release fails because we do not have a key to authenticate the repository. Adding the following will import their key (mentioned as EC61E0B0BBE55AB3 above) into your key ring:

# gpg --keyserver subkeys.pgp.net --recv-keys EC61E0B0BBE55AB3
gpg: directory `/root/.gnupg' created
gpg: new configuration file `/root/.gnupg/gpg.conf' created
gpg: WARNING: options in `/root/.gnupg/gpg.conf' are not yet active during this run
gpg: keyring `/root/.gnupg/secring.gpg' created
gpg: keyring `/root/.gnupg/pubring.gpg' created
gpg: requesting key BBE55AB3 from hkp server subkeys.pgp.net
gpg: /root/.gnupg/trustdb.gpg: trustdb created
gpg: key BBE55AB3: public key "Debian-Volatile Archive Automatic Signing Key (4.0/etch)" imported
gpg: no ultimately trusted keys found
gpg: Total number processed: 1
gpg: imported: 1
# gpg --armor --export EC61E0B0BBE55AB3 | apt-key add -
gpg: no ultimately trusted keys found
OK
#

Another spin of _apt-get update_ (and possibly _apt-get upgrade_ if their are any outdated packages) should then do the trick.

Tuesday, July 24, 2007

Links

IRB
Dr Nic has a great little article describing his favourite additions to his .irbrc. He covers some common productivity wins such as:
  • TABed auto-completion
  • Map by method which allows you to get rid of constructs like articles.columns.map {|p| p.name} or articles.columns.map &:name and simply replace it with a plural: articles.columns.names or articles.columns.name
  • MethodFinder/Object.what?
  • pp
  • Auto-tabbing
GuessMethod
This little gem is the best bad idea ever! Gone are those frustrating typos that waste extra cycles finding and fixing them.

Wirble
Continuing on the irb enhancements theme do yourself a favour and have a look at Wirble. It offers you tab-completion, history, and a built-in ri command as well as colorised results and a couple other goodies.

Tuesday, July 17, 2007

Links

Sequel
Sequel is a light-weight ORM that fills in the gaps where using ActiveRecord without rails doesn't fit your needs.

Gregory Brown has a nice little expose on it as the winner of the June 2007 Ruby Project Spotlight.

Introduction to .NET 3.0 for Architects
Keep your friends close and competition^H^H^H^H^H^H^H^H^H^H alternative platforms closer. InfoQ has a great 50000 foot look at .NET 3.0.

Profiling your ails app with ruby-prof
Charlie Savage wrote a great article on doing some profiling on your rails app using ruby-prof. He discusses using both flat and graph (with associated call tree information) profiles to nuke performance hogs.

ruby-prof is a fast code profiler for Ruby. Its features include:
  • Speed - it is a C extension and therefore many times faster than the standard Ruby profiler.
  • Flat Profiles - similar to the reports generated by the standard Ruby profiler
  • Graph profiles - similar to GProf, these show how long a method runs, which methods call it and which methods it calls
  • Threads - supports profiling multiple threads simultaneously
  • Recursive calls - supports profiling recursive method calls
  • Reports - can generate both text and cross-referenced html reports
  • Output - can output to standard out or to a file

AR-Delegation
This plugin extends ActiveRecord::Base to add useful delegation features. For example: has_columns :from => :source, :only => ["title", "name"] has_column "title", :from => :source, :as => "source_title".

It really improves the conciseness of your code but in so doing hides your implementation adding a layer of indirection that may make your code a little more difficult to understand if the person reading your code does not know that AR-Delegation was used.

Exception Notifier
I use this with most of my projects that reside on remote customer networks where the only way for the application to give me a heads up is if it sends me an email with an attached problem report.

About Me

My photo
I love solving real-world problems with code and systems (web apps, distributed systems and all the bits and pieces in-between).