I don't think there is such a thing as "too much RAM".
To see if you "need" the RAM though, with Photoshop open check your scratch size. If you don't know where it is (no insult intended, I am always surprised how many PS users don't know of this function) with a file open in the lower left corner next to the zoom level is the little triangle flyout. From the flyout choose the option "Scratch Sizes". The number on the left is how much RAM you are currently using, the number on the right is the amount of RAM Photoshop has available. You'll note that the scratch size is completely unrelated to the filesize in explorer. It's keeping not only the image data (of all images open), but your history, cache, etc. And if you have a many-layered image it can pile up REAL quick.
As long as the number on the left is smaller than the number on the right, you are good to go. But as soon as that left number gets bigger than the right, that means PS is writing to the hard drive. And THAT is super-duper-sloooooow.