Reaction time/distance is directly impacted by speed. I'm travelling at 50, you're travelling at 80 and an object enters the road. Simple physics tells you that the object travelling faster will be further down the road (closer to the object) by the time they can react and will need significantly more room to stop. Alternatively, if your intent is to avoid the object the handling of your vehicle will be diminished by the speed you are travelling. I can slow down to a speed that allows me to safely swerve while maintaining control before you can. Either you can't swerve soon enough or you swerve and lose control of your vehicle.
Knowing the condition of your vehicle and your own driving skill is key to knowing what speed is safe in any given area. A 1995 Chevy Cavalier with poor brakes and worn tires and shocks/struts is not going to be able to react fast enough to sudden obstacles at almost any speed. A car designed to go fast that's in good condition will have no problem stopping/changing direction in time to avoid the obstacle at 80mph.
Drivers of cars that aren't well-maintained should not go 80mph. Drivers of cars that are... and that are designed to go fast... are, all things being equal, perfectly safe to go 80mph.
A driver that's not paying attention will hit anything he/she encounters whether they're going 5, 50, or 80mph.
In the end, speed is far from the most relevant or significant factor.
Speed of travel is the only thing impacting that. Unless you're psychic you can not anticipate every item that may cross your path. Travelling at the posted speed limit for the stretch of road you are on should limit the impact as it takes into consideration many aspects you state.
The posted speed limit is arbitrary, and often set in such a way as to produce money. It will suddenly go from 55mph outside of a small town to 35mph or 25mph as a way to trap motorists. If it was truly about safety it would go from 55mph to 45mph and then to 35 or 25.
There's nothing necessarily less safe about going 65mph in a zone that's normally 55mph, either.
Posted speed limits do not adequately account for the variety of vehicles (both in type and maintenance history) and drivers. Some drivers and some vehicles aren't safe at 25mph, much less 55. If limits are necessary, variable limits make more sense than fixed.
Law enforcement should be making up for that variety by judging each situation as it occurs, but far too often their analysis is weighted in favor of generating revenue and totally distinct from any true safety concern.