Some audio chipsets have a test feature and/or in windows audio dialog somewhere to test each channel.
What I finally did, which seems to have worked is that I got the latest Realtek Audio drivers for my mobo. They were only for Windows 8.1, but I ran them in compatibility mode. Then, once inside the Realtek software, I was able to unplug and replug the headphone in the Realtek HD audio 2nd output mode I wanted it to be used in, then, switching to the audio mode for "Speakers", I replugged and unplugged each of the three analog pairs assigning them one by one to the right output.
I think when a Windows reinstall happened several months ago, I completely forgot to do anything other than letting the computer just take basic Windows audio drivers. And since I hadn't been actually using the 5.1 from the computer for months, I hadn't noticed. Nevertheless, this is far far too complicated and should be a simpler setup.
What I'm now wondering is...
My receiver has no HDMI inputs. So for 5.1 HD video on my computer (but mostly for games) I send HDMI video to my TV for video and these analog audio cables go to the receiver. But I also send my gaming consoles and raspberry pi (both with 5.1) to the TV via HDMI and then simply passthrough their 5.1 audio to the receiver via SPDIF out from the TV itself. That allows me to send 5.1 to the receiver via one main optical to deal with the lack of HDMI inputs on the receiver.
So, what I'm thinking is... What if I sent video AND audio from my GPU to the TV via HDMI, which is already hooked up and which I already do for video, then I could output the audio via optical through my tv like with the other devices. The only question is if there isn't some problem with windows sending 5.1 from my GPU (AMD R9 280). Does that sound like it would work?