Feeling backed into a corner here... looking for advice.

I'm gay. I finally admitted it to myself and accepted it last year, my freshman year in college. I don't have a problem with it and the few friends who know don't have a problem with it either. I've only told a handful of people including my two roommates.My parents are pro-gay rights and I doubt they would take issue. Not quite sure about my brother -- he's anti-anti-gay but I still think it might be weird for him to find out.My boss is gay and married. I think he assumes I'm straight and encourages me to have a lot of children because "You're smart and we need those genes spread around more." I haven't figured out how to tell him that I'm not exactly interested in women.I had three girlfriends in high school. In the first two relationships I honestly liked the girl and the relationships ultimately ended, the first one so she could hook up with my best friend at the time and the second because our relationship wasn't really going anywhere. The third relationship was with a girl I had been friends with for a long time but hadn't necessarily wanted to date; I went with her to prom as friends and (I don't remember this well) apparently we started dating around that time. We broke up when I started going to college because of the time commitment on both our parts.This is where I start to feel backed into a corner. Until the last year or two I hadn't even admitted to myself that I was gay. I'd always had suspicions I might be bisexual, but never gay. If I come out publicly, I don't know how I'm going to explain my past. I come from a small town where everyone really does know everybody else's business and enough are anti-gay to generalize, even if it's unfair. As soon as one or two people know the whole town will know. How do I explain the girlfriends? What do I say if someone confronts me about it? I realize that I'm probably blowing this all out of proportion, but I feel woefully unprepared to come out.TL;DR: I'm gay, only admitted it to myself last year. Had girlfriends, how do I explain that when I come out?