At my previous position as a Systems Administrator, I got to experience firsthand the convenience of using SSH keys. My personal SSH key was encrypted and password protected, of course. This allowed for quick and easy authentication to systems as my user account. As long as you kept your SSH daemon up to date this was actually reasonably secure. This greatly reduces the amount of passwords you have to remember in a Unix or Linux environment which is not utilizing any kind of directory services.
The second and even more useful aspect of SSH keys is from an automatic administration standpoint. For example, I once had to devise a method for devices that were at customer's sites behind various firewalls, proxies, etc. to "phone home" with the minimal amount of configuration on the client's end. After considering various ideas I came up with a solution that wasn't elegant but got the job done effectively. I had the remote devices automatically connect to the central SSH enabled server via an SSH key and open a remote port forwarding connection using a randomly assigned port on the central SSH server. The remote device wrote a line in a log file indicating it's machine name, the IP it had connected from, and its currently used port. This allowed me to use a simple script to connect to the machine by host name. This allowed for various automated remote administration techniques to be utilized. As the automated connection back to the other remote device was not using the root user, we found this technique to be an acceptable risk.