A many-core CPU is only good at doing many unrelated things at once. You could end up with situations where 31 of your CPU cores are waiting for that last core to finish and deliver the results, before they can continue working (extreme example, but you get the idea). You can "optimize" for multiple cores/threads by trying to split up a big algorithm into smaller, independent operations, or find some other task for the idle cores to occupy themselves with, but there's only so much you can do. The ultimate limit is probably the humans that write the code.
A multi-core GPU traditionally also did many unrelated things at once - it determined the color of each pixel, and there are a lot of pixels. However, the same thing is now happening in graphics. Each pixel isn't an isolated instance - advanced effects may force the GPU to take into account the colors of other pixels in a scene to determine the final color.
So both will eventually hit a brick wall, IMO. However it will take *much* longer before this happens with GPUs, because there are so many pixels to work with.