The crowd hates those people who want to be individuals, who want their own way, who want their own style of life. The common people wants you to be just one of them. Your separation reminds them of their position, and it is not surprising that the common people has always been against the individual
