I am having the same thoughts..
On a full charge, listening to music for about 30 minutes, moderate web browsing (maybe 5 pages max), and fairly heavy texting resulted in the battery diminishing to the 10-15% region in just 5 hours.
This is fairly disappointing. I love WP7, but given my lack of choices at TELUS (and the fact that I have two weeks to try this phone before I am stuck with it) I might be switching to an iPhone 4.
I have also noticed that after fully charging the phone and taking it off the charger, so little as a single text and flipping through the menus a few times will reduce the battery life from full on the meter.
Notes: I have WiFi and Bluetooth off, and I have screen brightness set at medium. Push email for one account, updating every 15 minutes for 3 accounts.