I use removable SATA hard drives on virtually all my new Windows Server installs. If a drive bay in a server is available, then I'd use SATA drives in removable trays.
http://granitedigital.com trays and receivers have worked well for us at numerous clients. You'll likely have to add a hot-swap-capable SATA controller card, since most built-in SATA controllers don't hot-swap properly.
If needed, you can even add a separate XP box with a removable SATA drive receptacle.
I avoid using pre-built external drives if I can. They tend to have a high failure rate because most don't have fans and the drives get very hot during many-hours-long backups.
No matter how small the office, I recommend a MINIMUM of three hard drives to start. One is left in the Server (a week at a time is typical). The other two are kept offsite and one is brought in to swap. That ensures that at least one backup drive is offsite at all times. An additional drive is useful for making periodic ARCHIVAL backups, which can be handy if old data is accidentally deleted and isn't included on your current set of backup drives.
I ALWAYS test the integrity of the backups by running the built-in verify function during the backup, as well as doing a test restore of a file on a monthly basis.