Specs:
Neon: 2588 lbs, 25MPG/36MPG (C/H), 132HP, 130ftlbs $13385us
Cavalier: 2617lbs, 25/33 (C/H), 140HP, 150ftlbs $14595us
They are close enough, that I'd go with whatever one had the better options in that pricerange. I haven't driven either of the new version of these vehicles so that would really be a huge factor.
The NEON has a slightly better crash rating FWIW. I have a 97 Neon and have had a few minor problems and the headgasket did need to be replaced under warranty, but, they used the new MLS(?) one that's supposed to last a lot longer. The new gens have that standard.
Keep in mind that a coupe may look sportier, but, the sedan is much more convenient for lugging friends (and eventually kids) around. That may be a pro or a con depending on your age group and how many of your friends/coworkers have cars.
At that price range you could go with a entry pickemup truck. Ford Ranger, S10, etc. Just to throw a curve.