Wish there was another word besides homophobia

The generally accepted definition for homophobia (off the top of my head) is someone who's bigoted against gay people in some way and genuinely has a problem with them. However in my life I have male friends who are totally disinterested in physical contact between males, even if both are straight. I'm talking anything more than a handshake: hugs, pats on backs, holding hands. I often find myself calling these people homophobes without really thinking about it. I thought about it and what I really mean is that they in some way have a phobia of male-to-male or "homoerotic" (loosely) behavior. You could claim this is because they really are bigoted but for many of my straight friends I doubt it. So I wish we had a word or could make up a word that described this kind of behavior so I had something else to say that wouldn't offend straights.LOL rant over