Question for gay males: is it shallow if I'm turned off by my boyfriend's belly?

TL;DR My boyfriend's belly makes me sad. When we're having sex I avoid it because it's such a turn off to me, which is depressing me and distances us.I've been in a committed relationship for over a year with a guy I fantasized about before we ever met, so it's literally a dream come true to be with him. But there's one major problem. As selfish and shallow (he called me these when I brought this up to him) it may seem, it's true that I get saddened over his gut. His abdominal region is lackluster, not firm, no muscle, it's rolly and ... gross.We have an open-ish relationship, meaning: we have threesomes with other guys but two-somes with others is a no-go. When these threesomes happen I love the experience, but it really bothers me that oftentimes I'm more attracted to the bodies of the other guy we're with. They usually have abs and chest muscles. My boyfriend has a flabby gut and man-boobs. It's just depressing.We're going to the gym together tomorrow and we are planning on doing this regularly - for the health of both of us. This problem has evolved to a point that depression just hits me that I'm with a guy I'm unattracted to in this aspect. I didn't live in turmoil dating a female for years, pretending to be straight, having sex when it didn't turn me on, then drastically ending it - just to date a male where the same thing happens (other than pretending to be straight).He and I have discussed this several times over the past year and every time it becomes an argument in which I am selfish, shallow, "only care about aesthetic", etc. No, I just wanna feel some tight chest & abs, dude.What do you think, guys? Am I shallow?