Going to a sex party with a few acquaintances, two of whom I do not want to be sexual with. Say "no" to them or skip the party altogether?

My BF of nearly two years and I both were very slutty before we met. We both had a circle of friends who we fucked around with and we both were frequently asked to sex parties and orgies. After two years of monogamy, we are about to return to the playful scene. We live in a large, progressive city, but our gay community is small enough that any well-traveled gay knows at least half the other gay men in this town. We lead some significant gay fundraising activities and volunteer for prominent gay causes.A close mutual friend of ours is having a birthday party, and it's been made clear this will end with an orgy. It's probably going to be less than eight people, all of whom travel in the same social circles. I have a dislike for a couple of the guys coming. I'm not drama-prone or anything, it's just one has been so pushy about getting in my pants over the years it's gone from flattering to disrespectful and another is such a twit I prefer to keep him at arm's length.Given all this... if I am fooling around (say giving or receiving BJs with a couple guys) and one of these guys I dislike tries to join me, is it uncouth to politely indicate I am not interested in sexual contact with him? Usually at orgies with strangers, I have done this very directly by first giving a gentle touch and shaking my head, and if the guy persists telling him "thank you but I am not interested." How do I do this here where we all travel in the same social group? Or given that it's a small group do I need to skip this party if I am not open to the free-for all?Incidentally, my BF is a "radical-love" type, and is appalled that I would not just enjoy the sex regardless of who is giving it at this sort of thing. I wonder if I have to adopt that attitude ("Look, it's a sex party, who cares which dick delivers?") OR just not attend. I know my BF will be embarrassed if I turn down people at the party - again, small group of people who frequent the same circles. However, I think that is acceptable and understood (I'm never bothered if someone turns down sex with me).