Suggestion for testing, and IN THIS SEQUENCE:
(you will easily find each test available free on the web)
1st:
See if your motherboard brand has a web forum showing other peoples' troubleshooting and tweaking experiences. Study them~!!!
2nd:
Do a careful visual inspection of everything, every tiny component, every electrical connection, all wires around fans & heatsinks, etc.
3rd:
Run Memtest86+. Some motherboards include Memtest86+ already in the bios (DFI, perhaps other brands) If not, download it.
Some recommend that you set Memtest86+ to run ONLY tests 5 and then 8, for maybe 100 passes each. Then if that's error free, go ahead & run the entire gamut of tests for 100 or so passes.
If you get errors, then retry tests on just one stick of ram at a time.
When you see red errors you'll immediately think your RAM is defective, but in most cases the errors are really due to your ram timing and voltage settings in BIOS, to tweak later, a fine art.
Tweaking bios settings can take a lot of time, brute force trial & error figures into it a lot, with plenty of restarts, adjustments, restart again.
You can probably tweak some of your BIOS settings WHILE MEMTEST86+ is running.
If you do, then look to see if each tweak results in any RAM bandwidth increase or decrease, which you can read in real time as a number directly in the Memtest86+ test display while it's running. Achieve any increases that you can.
4th:
Then run OCCT! 30 minute torture test from France. This is the best quick predictor of how you may or may not pass the Prime 95 test.
Forget about SuperPi that you'll hear about, it's basically worthless because you can pass SuperPi but fail OCCT!. The whole idea here is to get a reading QUICKLY on whether or not you can pass Prime95. If you fail OCCT! you will certaintly fail Prime95 test. If you fail, that means you need to change your bios settings, e.g.; perhaps loosen the RAM timings a bit, change voltages a bit, to achieve more stability.
5th:
The BIG TRUE RELIABLE STABILITY TEST is Prime95.
There's a nice GUI version of it called SP2004 that gives you a little bit better info, but if you plan to report your Prime95 test results to anyone else, such as on an overclocker web site, they may not accept the SP2004 version, which is the version that I much prefer, & recommend.
Your goal: To run Prime95 (or Prime 95 dressed up as SP2004) for 24 hours with no errors. SUccess means you can run your rig like crazy, stress it with major game play, etc. without any crashing or other problems. It means your computer is STABLE.
6th:
After that, you go into video card testing.
Lots of differing ideas float around about gpu testing.
A test result doesn't mean anything until you compare it to someone else's for a similar computer.
I usually rely on running 3DMARK 01, 03, 05 and 06 tests, then compare my resulting scores to scores for similar rigs as posted on some overclockers' web sites.
You'll see reference to Aquamark which is an old test that's been defunct for more than a year, and several other brands of video tests.
Again, opinions on what constitutes a good video card test are all over the map.
I've had outstanding overclocking success with the above testing procedure.
EDIT: I just want to add: Monitor your temperatures of CPU, chipset, etc. during all testing!!!!!