For very minimal use (exactly like what you mentioned, a couple of times use per week), and if you are in a good AT&T native coverage area and already have either an AT&T locked GSM phone or any unlocked GSM phone, check out h2owirelessnow.com
They are AT&T MVNO and their network coverage is the same with AT&T GoPhone (basically native AT&T towers, no roaming).
I'm talking about their 'Minute' plan. It's true pay to go with the highest rate of 14c/min and 5c/txt (10c the first text of the day). Their $10 and $20 refill expires in 90 days, which is pretty much what makes this a great deal.
Their 'starter pack' of sim card costs $10 + $5 shipping from their website, but I found some reseller on eBay that's selling it for less than $6 shipped. The kicker is that when I activated the sim card, it gave me a bonus of $7 in that account, which practically means that they gave me a buck to open a line + sim card

(not sure if this is how they all are though, just my experience).
Added: if you don't already have a phone you of course can buy it from them from as low as $15 or get it from somewhere else.