My guess is that coaxial and digital output are unlikely to both be found on an entry level audio card.
You could either buy two separate sound cards (one with an optical output, the other with a coaxial output), or combine your mobo's digital output with that of your sound card of choice.
You would only be able to specify one device as the default device, so whatever htpc programs you are using hopefully have some sort of configuration that allows you to select the sound card output you want to use.