Years ago I was always buying the latest and greatest, new, but this is when high-end GPUs cost a $400 max.
For some time already, I am buying most h/w used, motherboards, CPUs, GPUs although my last "used" purchase was pretty much almost new price, an EVGA GTX 970. (In this specific case, the fact that EVGA warranty is transferable of course was great ..low and behold I HAD TO RMA this card a week after I got it used).
But I am making it a habit only to buy H/W from forums where people have a history and rep, at least for more expensive things. Stuff like HDs, memory etc. I also buy from classifieds forum.
If you buy second-hand, warranty CAN be a criterion, note that not all warranties are transferrable. For example, people often advertise things like corsair memory "still has 5 years warranty" etc...however Corsair warranty only applies to the initial buyer. (At least here in the EU!)
When you buy, make sure people have receipts etc. so you know they bought the stuff otherwise its difficult to trace back the history of h/w, how many people owned it already and what they did with it. There is a certain kind of people on h/w forums who seem to buy large amounts of stuff like GPUs, CPUs etc. all-the-time to cherry pick the good stuff for themselves and then re-sell the bad ones, bad overclockers etc...so always use some common sense when buying used.