Am I crazy?

So, I had a boyfriend that always wanted to be with his ex. Worked with him, had a band with him. We moved together, after nearly 3 months living together we finally had a weekend together. First thing he told me when we got to bed was: " I want to be with my ex more often". This and much more stories made me hate the guy. Eventually we broke up cause of it.We broke up and on the same day he started talking with some guy. Traded some bulge pics, some dirty sentences etc etc. ( Oh and went out 3 times in one week) Never actually happened anything between them. Now he wants me back, I told him to chose between me and that guy, that if he wants me, that guy has to leave his life forever. He thinks I am mad and that I am some jealous monster. Am I that wrong? Should I just let him go?