That's a difficult question for sure.
I would definitely say you need to look at it on a case-by-case basis. Most of the time we're talking about sexual harassment. I think most people would agree that is not acceptable, and you should definitely look into it.