This is done using ssh-agent (and ssh-add) which will be on your Debian or Ubuntu system if you have the openssh-client dep installed. For other flavours of Linux, OS X or UNIX please refer to your package management documentation (or install from source) to see how you can install the required software.
On an Ubuntu system ssh-agent is started for you by default. Please refer to your system documentation for ssh-agent to find the correct way to run it on your system.
The following approach should work for situations where the client (the computer with the private key) is either a server (access is generally restricted to it via remote shell) or a desktop (includes laptops) with a graphical terminal program.
Add the following to the end of your ~/.bashrc (or other suitable shell setup configuration file):
Save the addition to your .bashrc (or suitable alternative) and log out and back in.
# Run ssh-add if it has not been run already.
if ssh-add -l | grep -q 'The agent has no identities.'
You will be presented with a request for your pass phrase you chose when creating your public/private keys. Enter it and sigh with relief as your default key(s) are cached in memory.
When you now try and log into the remote system again there will be no passwords or pass phrases required for this session.