I think the problem highlighted by Dave's attempt at (probably) drunken trolling is this tension between the idea of judging historical figures in the context of their times versus judging them by present day standards. Yet it isn't really an either/or. We can judge them in context, to a point. We can look at all the bad and good things they did and make a judgment as to whether they are worth honoring.
As was typical of wealthy white men of that age, Thomas Jefferson owned a small number of slaves. That is incontrovertibly a terrible thing. But he was a vocal opponent of the institution of slavery. He called slavery a "moral depravity" and a "hideous blot" in spite of owning slaves himself. He probably thought the fact he was gentle with his slaves excused him owning them. He was wrong.
Yet he also drafted the DoI, and gave us "all men are created equal," something which, I'm sure, all of us would rather have than not. And while he was in Paris when they drafted the US Constitution, it was largely based on the Virginia Constitution which he drafted. In my opinion, his accomplishments outweigh his personal failings of owning slaves during that era.
Compare and contrast with Robert E. Lee. Not only a slaveowner. A vocal proponent of slavery. Was a traitor to his country. Fought a war which cost over 600,000 American lives to protect the institution of slavery. His accomplishments? He was evidently a good tactician. Would have been better had he been inept, frankly, as that would have meant an earlier end to the war.
These historical figures need to be judged as worthy or not of honoring on a case by case basis. I don't think we can strictly apply present day moral standards to their behavior, but those standards can't be entirely ignored either.
Don't fall into the conservative trap of trying to hold us to our strictest modern day standards as expressed often times by our most extreme elements. Saying if we tear down the statues of Lee we have to tear down the statues of Jefferson is very parallel to saying we have to believe Tara Reid because some idiot said we have to believe all women. These reductio ad absurdum arguments are a tactic used to divide the left into circular firing squads. Don't fall for it.