So after day's of it working flawlessly, it just happened again in BF4.
After playing for about 30min this morning, I started to lag, I knew what was about to happen, everything basically goes 1fps, but doesn't crash or freeze completely.
Again, task manager say's 96% of my memory is in use. WTF is going on here???