Which capcitors are you interested in?
The mains reservoir capcitors (the biggest ones) - these are high-voltage and under relatively low stress, ripple here isn't a desperate problem, their most important function is to keep the power stable in case of a transient loss of power. Unless your mains power is really awful, I probably would leave these, as these are expensive and replacing them is likely to give only little benefit.
The output filter capacitors - these are under high stress, with high ripple current. Some of these may be inside the regulation feedback loop of the PSU, so some care is required in selection of replacements in case they destabilise the feedback loop. If you improve ripple current rating and ESR as well as increasing capacitance, you shouldn't run into problems. Simply swapping in higher voltage rated capacitors will usually improve ESR at the same time (at the expense of a larger physical size). Whatever you do, don't replace the capacitors with ones with higher ESR - this will degrade regulation quality and phase margin in the feedback loop.
There are potential benefits for upgrading these capacitors - increasing capacitance can decrease ripple voltage and improve transient response. Decreasing ESR can improve voltage regulation.
[Edit - snip safety advice of dubious relevance to the present task]
Another potential gain can come from replacing the diodes in the PSU - again, I don't mean the high voltage ones. I mean the low-voltage rectifiers after the transformer. These are often packaged as a pair of diodes in a 3 pin package looking exactly like a transistor, and are attached to a sizeable heatsink. Replacing these with high-grade rectifiers (e.g. dual 50A SMPS rectifier Schottky diodes) can reduce heat and potentially improve regulation.