I was there recently. OP, as long as you stay in the French Quarter (where you'll likely be touring since you won't have a car), it's relatively safe. My own rule of thumb (I'd been there once before) is not walk/visit north of Rampart Street, go beyond Canal Street to the west or go east of Elysian Fields Avenue.
If you're going to do your sightseeing during the day AND be relatively sober, you'll be just fine. It may get dicey at night (after 9pm as someone else mentioned).
My suggestions are:
For Beignets, go to New Orleans Famous Beignets And Coffee. Cafe du Monde is overrated. The 1st time I tried Cafe du Monde (2014), I wasn't impressed. I tried it again recently.... and yep, suckage confirmed. Cafe du Monde is open 24/7. New Orleans Famous Beignets & Coffee is open 7am-2pm daily.
You can eat the beignets on a bench overlooking the Mississippi River (a few minutes walk from either Cafe du Monde or New Orleans Famous Beignets).
Definitely go to Central Grocery for a Muffuletta sandwich, which they sell in whole or half-order. Unless you want leftovers, stick with the half-order size.
Jackson Square has plenty of vendors selling their gorgeous artwork and crafts. Back in 2014, there were street performers across the street (so in between Cafe du Monde and New Orleans Famous Beignets); I didn't see any this time. (Let me know if you wanna see the video clip I took of the street performance).
St. Louis Cathedral (a stone's throw from Jackson Square) is pretty cool to see inside. Hubby and I also visited another church (a 15 minute walk from Jackson Square), the Immaculate Conception Church, which IMHO is waaaay cooler than St Louis Cathedral. Both churches have mass services, so you'll want to avoid going during those times.
If you're interested in hearing live Jazz music, there are two places I highly recommend: 1) New Orleans Musical Legends Park, which is on Bourbon Street. There's also a Beignets shop there, but I didn't try it. The second place is the southern part of Frenchmen Street. You'll hear/see jazz musicians playing outside the restaurants/bars. Walking to/from French Quarter to Frenchmen Street is safe, even during the early evening hours.