*Maybe some mild spoilers*
Meaning, I often have quests added to my list just by talking to an NPC, without having a "no I don't want to do this" option. An example that I mentioned earlier was finding an unusual jewel, following the quest markers to "show it to an appraiser" and getting the "Join the Thieves' Guild" quest line added to your journal. I could have opted not to show it to an appraiser, leaving that quest on the list forever, or I could follow through, and then maybe later decide not to go ahead with the Thieves' Guild line, leaving both the quest and the item active forever. I currently have one of these items in my inventory, and I guess it will be there for the rest of the game.
Same thing happens if you follow up on the initial "Visit the College of Winterhold" or "Investigate the Bard's Guild" quests. As soon as you go to those areas and visit the NPC you get assigned further quests. You can tab out in the middle of the conversation, but that leaves the previous quest incomplete. Or that Aventino kid: it's not like he asks if you want to kill his nemesis: the job gets assigned as soon as you talk to him past a certain point, and then you have to decide whether to do it or leave it on the list. If you do it to clear it from your list, lo and behold you're now engaged in this whole Dark Brotherhood thing.
Last example: the House of Horrors quest. As far as I can tell once I spoke with the guy on the street and offered to help him I really had no choice but to follow through and do what Molag Bal wanted me to, or leave the quest unfinished. I never saw a "No Molag Bal, I won't do your dirty work!" choice.
Maybe it's me, but I'm not picking up early that these things are going to lead somewhere I don't want to take the character, and I'm not getting presented choices to back out, which is aggravated by the fact that I'm stupidly anal about clearing my quest list.