Am I being too controlling?

So my boyfriend invited me to a small family get together in a few weeks. I originally told him I was interested but I might have to work. He took that as a no and invited some of his friends. I then found out that I could get the time off work, so i told him that I can actually go! He seemed excited at first and then started to act upset about it. We started talking and he first posed it as being uncomfortable with me meeting his parents for the first time while having friends there and everything else going on. I understand that, but then he told me I could still go if I wanted. We talked some more and he changed his stance from being uncomfortable around his family, to wanting a vacation with just his friends. He was saying that he doesn't get enough friend time and that I'm no longer invited on the trip. I don't mind if he wants to spend time with friends, I mind that he invited me and when I finally gave him a solid answer, a day later, he already uninvited me. It makes me feel unwanted and unimportant. Am I being too controlling in the situation or am I right to be upset?