Nothing wrong with using "that said", unless it's used wrong, as you did ;^)
That said, exactly.
Maybe if you used it correctly, it wouldn't seem as annoying.
That being said, my most recent bowel movement weighed 2.2 lbs.
discuss.
2.2 lbs is a massive bowel movement, that being said, everyone knows you are full of shit.
'nuff said.