IMO, negative feedback for a buyer means almost nothing. You can still bid on stuff. Most of the time, if it is just a few (-)s than the seller won't cancel your bid. With BuyItNow, you cannot stop someone with negatives from purchasing so even if you lots of negatives, you can still BIN. So, if you are mostly a buyer, it doesn't matter. It's just a psychological issue with receiving a negative feedback. As often if you bid late enough, the seller has no chance/choice to reject your bid.
As a seller, negatives are a hinderance. Your feedback is there for buyers to evaluate. Many negatives will lead to some buyers clicking the back button. I think that there are some people that won't care about the hundreds of good feedback. They won't even read those. They might fixate on the few/one negative feedback and question you on that.
So a long answer to your post, if you are mostly a buyer, don't sweat it. Just calmly respond to the negative and move on. IF you are a seller mostly, do the same and yeah try to get the neg. removed.