If you had not been so close to him or rushed to pass him, would the situation had occurred? No.
People will drive for miles upon miles with their turn signal on and then fail to use it when they do something as simple as changing lanes on a light to no traffic interstate. NEVER trust a turn signal, or lack theroff. Until you see a person come to a complete halt, don't assume anything. That's the lesson you gotta learn from this, but be lucky you didn't learn it the hard way.
I know some people are probably thinking I'm nuts for being overly cautious, but waiting a whole 5 to 10 seconds more to be certain of what someone else is doing is much better than risking an accident. If you being at your destination is so important you can't spare 5 or 10 seconds, perhaps you should request a police escort, call an ambulance, or learn to leave earlier.