Obviously depends on the workplace culture. But always keep in mind that if someone decides something is offensive or sexually harassing, the damage is done even if nobody else seems to care. If push comes to shove, you'll hear, "I never spoke up before because I felt intimidated and feared for my job." They'll get other co-workers to agree that such things had been said many times which makes it sound even worse.
I once worked at a place where the managers would talk like they were at a strip club. If a woman was around they would tone it down slightly but still it was way over the line for a workplace.
Then they hired a new executive secretary for the owner (who maybe wasn't the worst offender, but definitely in the top 3 and a real neanderthal). Nothing happened for a couple of months, then she told the owner she felt sexually harassed and offended by the language used in her presence. The owner invited her to quit. She said no, they had to stop talking like that and the owner again told her to quit if she didn't like it. She said if they didn't stop, she would get a lawyer and sue for a hostile workplace, she had already documented over a hundred examples of the things they had said to her and in her presence (she took minutes of the executive staff meetings) and also documented her previous complaint where he told her to quit. The icing on the cake was she told him that he ought to talk to his own lawyer before giving HER any more career advice.
A couple weeks went by and most of us noticed a distinct change in the language - not exactly PG-rated but hardly XXX any more. A little while after that, she left. I figured she couldn't take it and quit, but found out from a female co-worker later that she got many thousands to agree to quit without taking any legal action, and told a few other female employees who her lawyer was and what to do in case they went back to their old ways.
From some loose-lipped managers I learned that the owner was furious that a woman took his money and could force them to clean up their language but his lawyer told him to expect more of the same if they didn't straighten up, and that it wasn't the 1950's any more.