Not really there are rude people all over the place. The vagrants were most likely homeless people. Here is a very brief explanation. Up until the 1980's insane people & troubled people (the kind that are not capable a caring for themselves but are capable of walking) were kept in homes or hospitals for mental patient's. Turns out many of these homes were not taking good care of them and some were abusive. Society decided that this was not what we wanted and closed many of them. The idea was family care was better. We also made it harder to lock someone away for treatment against their will. The problem is many homeless people have distanced or don't trust their relatives, since they are ill its impossible to hold a job or pay rent. Yes the homeless can be rude, its more about their health than something you did.
OP where are you from?