Why are so many gay men against twinks?

My whole life people have automatically attached me to the label of "twink", but instead of embracing it they say as something negative that needs to be looked down upon. I don't understand why I'm socially ranked below another gay man just because I'm thin and act more on the effeminate side. When are people going to finally drop the labels and just accept people based on their character instead of the label they are put into?