The thing is, too, with downstream - if your pipe is stuffed, you need QoS to mitigate that issue, or a bigger pipe, but it has to be done on the ISP end. If your pipe isn't stuffed full of data, then you hardly need the overhead of the QoS engine, which usually (Ubicom or Qualcomm or whomever's StreamEngine hardware notwithstanding) runs on the router's CPU, and while it may offer more "regular" latency, it also can limit your maximum bandwidth through the router, and is generally thus pointless.
IOW, if you have gigabit FIOS (Gig up/down), there's really no need for QoS to "shape" your traffic. If you enable it, on most routers, other than possibly the most recent crop of quad-core AX routers, you'll limit your bandwidth to nearly 300Mbit/sec doing packet-processing in software on the router's (dual-core, in the case of my AC68U-class router) CPU.
About the only way that I would actually "recommend" running QoS on your router, is if: 1) You have DSL, or certain cable plans, and limited (congested) upstream bandwidth, and that can interfere with getting the full download bandwidth, if certain packets like ACKs aren't prioritized in the up-stream, or 2) you run services or clients for something like SIP / VoIP, that need a bounded latency, so as not to negatively affect service, and don't need maximum bandwidth from your connection.
Edit: I should mention, that bandwidth of certain TCP streams can be somewhat "controlled" incoming, depending on how the TCP/IP stacks are written on both ends, and "playing with" the ACK rate. But this is not guaranteed to work in all cases. (Edit: And can increase latency.) Also, UDP cannot be easily rate-controlled at all. (Edit: Without upstream router help.) Most gaming, AFAIK, uses UDP.