Why is it that many heterosexual people can't understand (though accept) how being attracted to the same sex works, but vice versa...

Firstly, I am writing this with the presupposition of countries where homosexuality is not universally condemned / discriminated etc.It's a brief discussion I've had with one of my best friends. Just saying, I do not blame anyone for not understanding, how could I? Now, I want to emphasise the discrimination between "understand that it can biologically be the case" and "put oneself in a gay's shoes". And it's about the latter.I think it is merely odd that, while many straights can't seem to get the hang of it, I, for one, can perfectly understand how someone can feel attracted to a woman (though I myself do not). I think I am just missing something essential here. Is it caused merely by the fact that society is, for the most part, heterosexual and therefore people who are not affected don't ever have to think about it growing up? And that many homosexuals only at some point in there youth (for example) become aware of their sexuality, and having only been confronted with heterosexuality have developed a deceptive sense of understanding?And don't get me started on the fact that people can't differentiate between aesthetic beauty and sexual appeal. I find the fact despicable that it is so direly stigmatised, as a heterosexual, to find a man attractive (and state it). I don't see where this direct association with "feeling sexually attracted to someone" comes from, but it sures has made a felt 80% of male heterosexual population completely incapable of openly saying whether a man is good-looking. From my experience, anyway.I hope I do not sound all too naive - just stuff to philosophise about, really, but I do wonder.