Is he right? Or am I being manipulated?

Ok. So, this is sort of a continuation of a previous post I made. It can be found here: http://ift.tt/2akHrlb, I ended up going through with breaking it off maybe a few months after I made the post. I was sort of "sad" at first, but surprisingly got mostly over it fairly quickly. I was surrounding myself with family and friends again, which I rarely was able to do. My ex, (Tom, forgot if I had specifed in previous post) obviously took it VERY hard, and usually messaged me on Facebook fairly often trying to change my mind. I never gave in. We went at least a little over a month without seeing each other. I had needed to retrieve something of mine I forgot and we discussed having him come bring it to me. We talked for maybe an hour and that was that. He texted me when I was at work maybe a few weeks after that and said he joined a rehab program, and just wanted to tell me that. I said I was very proud, etc. Fuck up #1 was agreeing maybe if he got clean and kept with it I could consider being with him again. Fast forward a few weeks, he finishes his program, we spend those weeks regularly seeing each other. I enjoyed it for a little, but ended up feeling like I did when we were previously together pretty quickly. Now, we werent back together this time, we were just friends. (more like fwb, possibly fuck up #2) I hang out with this group of friend's, who all live together in the same house. There's a guy there who I've developed a rather large crush on, and slightly flirt with. I don't plan on going any further, he's engaged, and straight (but sometimes he flirts back slightly) and it just reminded me how much I enjoy finally being single and out of that relationship, not having to worry about disappointing, pleasing, Worrying, or being constantly anxious anymore about anyone. Well, I didn't plan on breaking this news to Tom so soon, but it ended up coming out during a slight Argument, and he basically told me I should "feel bad" for "breaking his heart" and that he "put so much effort into our relationship, giving so much compassion, love, and passion as much as he fucking could" I told him that I guess it's more just me than it is him at this point, and he says "yeah I guess it is". I said i love him and always will, because that's very true, I'll always love him, and he said he didn't believe that anymore and just said "I believe you used to" which really kinda stung. Basically I'm asking is, should I actually be feeling bad? Am I really being unreasonable here? Him making me feel bad is kinda working.. :/TL;DR : Told ex I didn't want to pursue anything further anymore, lost his shit and said I should pretty much feel bad for feeling the way I feel. Which he has on other occasions.