A designer sounds cool in theory, but its actually a lot of hard work. You have to keep tabs on what everyone's doing, how they are doing it, and if what person A is doing will fit into what person B is building. Sitting in a chair and poping out idea's is not what a designer does. Sure it helps, but the fact of the matter is EVERYONE has ideas for the hot new game, be it the programmer, an artist, or even the composer. Companies don't generally hire people to sit around and think out ideas and have him order everyone around. Not to mention, companies want some kind of experience, which means you'd have to have worked on a game before, or done something *really* outstanding and popular (the people who made counterstrike come to mind). And lastly, you need to have really really good connections to get into this business.
How do I know so much about this stuff? Cause this is what i want to do (or maybe, wanted to do?).
Actually I started off JUST like you. I thought programming would be my thing. I went to a technical college, and it turned out that i SUCKED at programming. I just couldn't understand any of it. So I had to leave the school cause they didn't really offer much else except computer science and engineering.
I still want to make games, but i'm not an artist, nor a musician. Only other 2 openings are level designer, or game designer. I've tried level design and while I can do it, I can't exactly make anything nice looking. So now i'm looking towards the game design option, but its turning out to be more of a 'dream' than anything else. Sure, i'll still work towards that goal and try to get into the field, but i'll have to do something else in the meantime to earn a living. I can't live with my parents until 35 until i can land a design job.
There is a lot of info about this stuff on the net if you look around.
A good article to read would be
THIS ONE which lists the events and hardships of making a game and getting it out the door. There are a lot of other sources too, one of the better ones being
http://www.gamasutra.com/
Good luck!