If you've never been on a bike before (it doesn't count if you rode a friends), you should take the MSF course.
http://www.msf-usa.org/
In some states, if you pass, you're excepted for the road portion of the test. In Ohio, I took the temp test, got my permit, then took the state sponsored MSF course for$25. I passed and went to the BMV and got my license without owning a bike. Well, that was the plan anyway, I actually bought a bike before the MSF course, but I didn't ride it until I had the course.
You also might get an insurance discount (Nationwide will give you a discount and is who I ride with).
Ideal first bikes include the Ninja 250/500 or old UJM standards under 500cc's. Try to avoid anything with an "R" in the name (ie: CBR600RR, GSXR, etc). The exception to this is the Ninja 250, which I think they're calling the 250R now. Don't buy new unless you're loaded. If you drop a bike, chances are you'll do some damage and the more plastic it has, the more expensive it will be to fix.
Budget AT LEAST $500 for gear. $1000 would be better. You need pants (NOT jeans), gloves, jacket, helmet, and boots. You need boots with full ankle support and ideally something with covered laces. Exposed laces are something you can get caught in your bike and if you can't put your foot down at a stoplight, you're going to fall over on the bike. I don't even think $500 is enough for gear, to date I've spent $100 for my jacket, $75 for my helmet, $50 on gloves, and I still need to buy boots ($120) and pants ($175). Add in "extras" like a tailbag and a tinted visor and I'm pushing $650 for my gear. I bought my jacket and helmet on closeout so I saved some money there.
I haven't ridden much at all, maybe 20 miles tops. The weather got bad shortly after I bought my bike so I put it in storage and it's been there since.
Keep in mind, the used bike market is about to go from a "buyers" market to a "seller's" market because it's getting warmer and a lot of guys are buying. Towards the fall, some guys just want to get rid of the bike so they don't have to store it so you can get better deals.