I just need somewhere to vent

So me and my ex both have strong feelings towards eachother and love eachother. We broke up partly because of the distance and mostly because he is still in love with his ex. We've always just figured that when he is over his ex we can be back together and that's what we both really want.But earlier today I realised like I'm not sure if I could be in a happy relationship with him again. If he never loved his ex I would definitely be happy with him but I'm not sure id be happy knowing in the past he left me for someone else. Like I think if his ex had said he loved him and wanted him back when he left me then they would be together and he wouldn't really care too much for me, he's only gonna come crawling back to me because what he hoped with his ex didn't work out.It's frustrating because I want to be with him more than anything, but I know I can't.