Do male friends "fear" you going near them?

Evening,My friends know I'm gay, I've never been very out there nor have I been the person to be rough in any way yet whenever I am to sit next to them or accidentally touch them they seem to recoil like they've just been burnt. Only seems to be male friends too.I'm just being paranoid aren't I?