Did I cross a line here?

I've been seeing a guy I think is pretty special. We've been on 5-6 dates and when we hang out he usually ends up crashing at my place for the night. Last Sunday we hung out and we spent 24 hours together. We were texting the next day and he mentioned that he was Going to the gym and to possibly hang out with a guy that he knew wanted to sleep with him but said he wasn't interested in sleeping with.We aren't a couple or anything so I know it's not really my business but jealousy got the best of me and I asked him if he slept with the guy. He said he didn't and he didn't seem mad that I asked but I just feel like since then the dynamic of our relationship changed. We would text a lot back and forth but Its slowed down the last day or so. And I noticed his dating profile is reactivated and he was online today.We've had such a good time together and share so many interests that it seems crazy that I could have ruined things between us just by asking that question but just typing this out is making me panic a bit. Ive never been in this situation with a guy I actually cared about before and I don't know what I should do to try to salvage things. What should I do?