The only reason I can think there might be a performance difference is with regards to cached writes. In theory at least if you know the hard drive is going to be there then you can cache and combine writes in a way that isn't possible if the hard drive is hot swappable and might just get pulled. But in practice I don't think there is much in the way of a genuine difference, they could implement one but see how hot swap works in Windows it has time to go about preparing the drive.
I always disable it for drives installed in the machine, because if I am pulling out that drive I am guaranteed to have the machine off, but I know of no known reduction in performance for having it on.