The only reason I can think there might be a performance difference is with regards to cached writes. In theory at least if you know the hard drive is going to be there then you can cache and combine writes in a way that isn't possible if the hard drive is hot swappable and might just get pulled. But in practice I don't think there is much in the way of a genuine difference, they could implement one but see how hot swap works in Windows it has time to go about preparing the drive.
I always disable it for drives installed in the machine, because if I am pulling out that drive I am guaranteed to have the machine off, but I know of no known reduction in performance for having it on.
Write-caching is set by the OS, and Windows doesn't seem to care if the drive is marked as hot-swappable.
The setting exists because the BIOS doesn't know how your drives are set up.
For example, I have a mini-ITX board with 6 SATA ports. Two of those ports connect to internal screwed-in drives. Four of those ports connect to hot-swap bays. So in the BIOS, I disable hot-swapping for the internal drives and enable them for the bays. Then, in Windows, the safe-removal dialog shows removal options for the 4 bays that I've marked, but not for the 2 screwed-in drives.
Anyway, controlling what shows up in the Windows hardware safe-removal dialog* is the reason I bother with those settings.
* Assuming that your storage controller driver isn't brain-dead. iaStor, for example, is brain-dead and does not properly report to the OS on whether a drive is hot-swappable. Which makes hot-swapping a royal pain since I need to go through other means to make sure that everything is properly flushed (since caching is turned on--i.e., these drives are not optimized for quick removal). The latest driver (which also removed support for the 6-series chipset) apparently finally fixed this problem (gee, and it only took how many years?). Intel's broken handling of hotswap is the reason I use msahci on my storage server and my desktop (which also has a hot-swap bay as well as an eSATA port on the same controller) instead of iastor. (Intel has never had a great track record when it comes to driver quality.)