Maybe it's a ram issue.
If your not using a program it still takes up ram space, however the operating system needs to keep itself efficient. So if it isn't used for a long time then the OS will clear it's space out of main memory and make room for the apps that you are using. Then when you open the program back up then it's going to have to read all that cached information back off of the harddrive which is slow. Stuff like IE isn't affected much like that in windows, since the shared programming libraries in IE are used by the rest of the OS. So most of what is used in IE is already loaded and running, because it's nessicary to use windows explorer (the file manager and windows from openning my computer). However Firefox isn't a native windows app and has it's own unique set of libraries.
I don't know if this would cause a huge delay... Certainly less, or in worst case equal to the amount of time it takes to get it running in the first place from a newly rebooted machine. Time is very subjective in computers, something that takes 2 seconds may seem like forever if your used to everything comming up in a split second.
Plus it's only going ot be a issue if your running with little RAM. 256megs or less for windows XP may cause this situation.
Does your harddrive activity pick up as it's maximizing?
If it's REALY slow or hangs up for more then a few seconds, then it's probably a software bug in Firefox.
edit:
oops! thats definately NOT caused by what I described!