Info Handbrake now supports AMD VCE encoding.

Mar 11, 2004
Thought I'd give people a heads up if you use Handbrake for encodes. Yes, software (CPU) is still the best as far as quality (and if you need Filters like noise removal it uses the CPU still; actually apparently it only uses the encode feature so everything else is done on the CPU), but after doing some testing, especially doing h265 of Blu-ray source, the quality is good enough for me, and the speedup is much appreciated. Doing 3500 average bitrate I am getting h265 MKVs about 1/10th the size of the original. Some of that is it cutting off black bars, and a good chunk because I'm ditching most of the audio (downmixing highest quality main surround into high bitrate stereo AAC since I don't have a surround system; which saves several GB). Doing constant quality with 20-23 level, was getting file sizes about 1/3 of the original with those audio settings (and about 3x larger than 3500 avg bitrate), and while I'm sure there's quality loss, both of the encodes showed similar quality loss compared to the original such that the differences between them didn't bother me, so I'll go with the smaller file size (which is partly one reason I'm doing it, as I might be throwing videos on an iPad and so file size will be an issue).

I will say DVD is a different issue (I use the h264 for that, since from what I've gathered h265 won't produce much smaller file sizes; the VCE can do either) in that I haven't found what's the adequate level yet (if I do constant quality of 20, I get about 1/2 the file size of the original with decent - not sure I noticed much quality difference compared to the original; my older CPU encodes of DVDs got closer to 1/3-1/4 file size).

Also kinda bummed that it can't denoise as I have some movies that are very grainy (not a big fan, yes I know its sacrilege, I'm not going Predator plastic or anything), plus the grain prevents it from gaining (er well losing) as much in file size, but since that's still CPU it adds hours upon hours (using VCE I think h265 encodes of Blu-rays were like around an 1-1.5 hours, with h264 DVD encodes were like 20-30 minutes). I don't even wanna think about how long it'd take to do h265 encodes on this CPU (FX8320e).

Which, since there's apparently a bit of a format war going on there, not sure if people would want h265 stuff, which is why the time saving of using the VCE is fine with me. I have the discs so I can redo them when I have more storage and better CPU, and probably utilize newer format. Plus I might end up with 4K versions to boot.

From the little I gleaned in some of the rough comparisons, seems like its about equal to Nvidia's as far as time and quality (Handbrake has had support for it for some time, just the AMD VCE that was recently added), although that was with I believe Pascal and Nvidia claims Turing has better encoder, so worth a try for Nvidia users too.

Support is limited to RX 480 and newer (so Polaris and Vega) and Windows 10. Some extra options if you dabble in that:
Aug 25, 2001
Neat, thanks for the info, might have to pull Handbrake out again sometime to play with it.

Thunder 57

Senior member
Aug 19, 2007
So I gave it a spin, nothing in depth.

CPU: 2600X
GPU: RX 480


VCE was slower than my CPU in fast and very fast profiles at 1080p. Not sure there is much use for it here.


VCE was about 65% faster than CPU in fast and 25% faster in very fast.

This was more of a speed test than a quality test, but quality seemed good. Maybe I'll do a more in depth quality test next time I need to transcode something.
Mar 13, 2006
I thought this something new.

Turns out this has been in the nightly builds for nearly a year, and was released to the main channel in December.

Nice for AMD users to get support for their hardware the other companies have had for years.
Mar 11, 2004
Yeah, the H264 VCE seems to be just bad. Not sure if its Handbrake, AMD, or what the cause is, but its slower than CPU (my FX8320e even) and produces worse quality. I've played around with different settings and its just plain inferior to the H265 while being about the same speed when doing VCE. And file sizes of H265 tend to be better (supposedly, when doing software encoding on CPU, h264 on lower resolution stuff allegedly can be even a bit better than h265, with the latter coming into its own at higher resolutions). I might play around with it more and see if there's anything I can do (but probably will be stuck waiting for either Handbrake or AMD to improve it).

My best settings seem to be 1500kbps average bitrate for DVD quality level stuff (dropping below that quality seems to suffer quickly, and constant quality either is worse quality or similar for 2-3x the size; I get about 1GB files from typical feature movie length videos) and 3500kbps on Blu-ray quality (1080p) stuff (I haven't seen if I could go lower still, I was happy with the quality and file size at that rate, end up with 2-4GB sizes; I might test some more just to see if I can get it even smaller but would probably require dropping to 720p - which if quality was about the same but was getting down to ~1GB sizes I'd probably be fine with it for now).

H265 is faster than I initially thought too, seems its more like ~45 minutes for 1080p typical movie length stuff, and about half that for DVD, and then about 7-11min for ~22min typical TV episode length.

I have run into a serious issue though. I use Avidemux to trim unnecessary parts of videos, but it is just plain broken with h265 stuff for me. In edit mode it just shows constant black (and will output constant black video if you manage to edit anything and then finalize it, which I can't even do that now after I managed to on one video; I updated the most recent version and rolled back to earlier ones just to check and was happening with all of them). I can play and it'll play the video fine but the moment I stop (which is the only thing the program will then let me do), it goes back to the same. And if I use different methods to seek (like jump to specific time) it pegs the GPU at 100% for like over a minute and then show a still (where I can start playing from that spot, but hit stop and it goes back to doing the same as before). I thought that maybe it was because Handbrake was defaulting to 6.2 for the Encoder Level, but going with other levels I'm getting the same result. I should probably drop it down some just for likely better compatibility (I didn't notice quality difference with my content at lower Encoder Levels on H265, it'd probably matter when doing 4K HDR stuff though).

Having some other hassles with audio (I prefer Avidemux's audio stuff it uses Audacity's DRC processor, and then has an automatic gain setting where I can set it to -5.00 which gives a good volume plus headroom, and lets me kinda normalize volume across the videos versus trying to guess at what gain to use in Handbrake, and it also has a higher quality AAC encoder, oh and Handbrake can only do DRC when converting from AC3 to like AAC or FLAC; but Avidemux can't do 24bit or lossless so I can't convert from those). I would think about doing the audio separately but if I were to do any trimming I'd probably end up with sync issues when muxing it back together (which MKVToolNix seems to be decent and quick about doing that, but I don't seem to see any trimming ability).

Something interesting is I've actually been taking some older h264 rips, trimming them in AviDemux (since it actually lets me), then encoding them from h264 to h265 in Handbrake, and getting, well I can't really tell a difference in quality, but I'm getting smaller file sizes (even though most of my gains there is coming from ditching multiple larger audio tracks, I'm still gaining like 500MB+ in file size just from the transcode, at least on Blu-ray quality videos don't think I tried it yet with DVDs they're alreayd ~1GB in size and would probably be better quality as most of those I did with some better settings in Handbrake). I know its not optimal for quality but its not exhibiting any particularly bad outcomes yet and its saving me time re-ripping those movies and then the issue with trimming them in AviDemux. Plus I will re-rip these later or posisbly get 4K versions (or Blu-ray of the ones that I had DVD) of many of these when I have a proper computer to do it. Also want to see what happens with formats since it seems like there's some eagerness to move on already (my guess is AV1 wins out).

I'd love to see people's results with Nvidia's encoder too. Seems like most people just dismissed it for being worse than software, but for me, H265 VCE is good enough that I'll go with it (but I also have a particularly woeful CPU at the moment). Nvidia's seems to be as good (and might be better even, and possibly especially when doing h264).
Mar 11, 2004
I sorta figured out my Avidemux issue. Apparently long MKVs is causing it problems (at least for me). If I make it MP4 (or short, like only a few minute long, MKVs) it doesn't have issue. ShanaEncoder seems to be good for changing between the two (without needing to re-encode), so that helps.

Still playing with settings. I played around with dropping resolution, but was surprised how often I couldn't tell a difference between ~720p and 1080p at same bitrate. Although I did finally start to notice some areas it made a significant difference. I'd dropped avg bitrate to 2500 and there was a scene that looked bad in 1080p, lots of blocking and loss of detail - when other scenes remained sharp (so it wasn't that the whole encode was bad quality like I'd been seeing before. Staying at that bitrate but dropping resolution it held more quality during that scene, while when fullscreened on the same display (a 1920x1200 24" monitor at close sitting) I couldn't tell much difference between the two in the stable scenes either.

Also so far I haven't gotten into doing any of the really grainy stuff, so kinda need to see how that behaves. Might need to play with some of the settings that should help with that. Some I'm just curious how much time it'd take for that (as it might be something that would be worthwhile in general even in less grainy stuff, if it doesn't take too long; would be curious if it might help the quality in the scenes I'd noticed being worse quality as well).

Now I wanna see a detailed comparison between hardware encoders, AMD, Nvidia, Intel, and even mobile SoCs. But I think the software situation makes a real direct comparison too difficult to discern much. Hopefully we'll see VCE support added to other programs though. I notice Nvidia's encoder being supported in quite a few programs. I'm tempted to see if I can nab one of the Quadro cards (I have a use for their "Grids" option that is locked to the Quadro line) and to play with their encoder.
Last edited:

Similar threads