Help!!

I'm a straight guy and I have a high pitched voice, everyone assumes I'm gay because of my voice. I often encounter gay guys that think I'm gay. Is there something I can say when when I reject a guy without looking like an asshole.