The quick answer is yes, they do affect each other. As you are downloading, ACK packets are still being sent to the computer you are downloading from. If you saturate your uplink as well, those packets won't be able to get to their destination as quickly, and you'll see a slower download.
I'm sure there are those out there (in here?) that have figured out how much of the upstream is used given a download speed...