My Straight Best Friend Doesn't See The Problem with the word 'Fag'

Let me make this post more clear in context. My best friend and I have been close for years. He is one of the most open-minded people I know. This situation took me aback, and I would like your opinions for clarity and guidance.I was out a couple weeks ago with some friends at a bar. Drinks were being had, so, by later that evening, the group was feeling a little inebriated. My friend's boyfriend, Saul, looked at me and asked me why I had to be a fag. I looked up and asked him not to call me that. He responded that liking men is a faggoty thing to do...I left the bar to make sure I wouldn't get in an altercation or say something I didn't mean or would regret.On the way home, I called my best friend. Him and I talked for a while about the situation. He could tell I was upset. His response was to not be upset and let it go. I, being inconsolable, went off on him about not understanding what it was like to be called that and what the words means to me and my community. He responded that he was bullied growing up because people thought he was gay, so he was called those names and felt it on good authority to know what I was going through. He then proceeded to tell me that I take these situations out of high emotion and shouldn't look too deep into them.I ended the conversation there. The other day, he brought it up again. He told me I was too sensitive and that the word "faggot" is just a word. Even if there is hate intended by it, it's me who associates it and acts on it. That I should just be the bigger person and let it go...So...I ask...am I being overly sensitive? I won't pretend like I haven't used the word amongst my friends as a joke in context. But I've never approved of the word being used as a derogatory term for hate. Is my friend right? Am I wrong? I would love everyone's input.