No sex?

I have been dating this guy for about 6 months. We go to school in different states so we see each other for a few dates every few months and talk very regularly otherwise. This is my first relationship and it's been good, but rough sometimes. Most recently, we started fooling around, hands down the pants whatnot and it went well for me, but he didn't get hard and wanted to stop saying he was nervous. We talked a few days later and he’s realized that he just isn’t attracted to men and wants our relationship to continue as a purely romantic one. I won’t be seeing him for awhile now and we haven’t really had sex yet anyways so I said yes, but I’m disappointed. I know I want sex, how can we work this out?