It's a layered question. Couple of things going on:
Chimps and Gorillas are stacked because of their muscle composition. They sacrifice fine motor control for ass kicking ability. Incredible if you have to beat the shit out of another 800 lb Gorilla, terrible for interpretive dance.
Dense, powerful muscles require a lot of energy. Gorillas and Chimps need a ton of calories, which often means a lot more time spent eating. If you're spending 16 hours a day foraging for food, how the hell are you going to develop society and culture and all those nice things.
High energy requirements also make things tough during lean times. It's postulated that part of the reason Humans out completed (and out fucked I guess) Neanderthals despite relatively similar intelligence levels is Humans needed less food to survive and were able to ride out the last Ice Age while Neanderthals starved (and got literally fucked into submission).
I'm sure there is other stuff, but that's what leaps to mind at the moment.
Not mutually exclusive (Neandetals were turbo built humanlike intelligence hominids) but it makes long term survivability and adaptability much tougher.