I'm having the same problem on Vista 64 (and I also have 8 Gyg). Task Manager only shows explorer.exe doing something. It loads the CPU to 4% on a E8500 dually clocked at 3.7. When that entry goes back to 0, the menu finally pops up. I could have sworn the change happened right after a recent Windows update (maybe a week ago). The only reason it caught my eye was because Vista was updating something involving nVidia, which I've never seen happen before. I was curious and launched my nVidia panel by another route and, sure enough, it was slower now too. The Windows update history only showed security stuff, so no idea what it loaded. I've since removed the nVidia panel from the desktop context list, but both still load slowly (about 6 seconds). Everything else pops up quickly. I was accessing the nVidia panel ofter from context before because so many apps just can't handle SLI.
Anyway, I don't want to co-opt the thread. Just thought my situation might shed some light.