I'm not sure what to make of a friendship with a guy I met on Match

About three years ago, I met a guy on Match, who was super nice, but not someone I'd want to be romantic with. We stayed in touch and built a fantastic friendship. One night, he got drunk and suggested we watch a movie (he never watches movies - doesn't even own a tv), and laid his head in my lap the whole time. I was a little weirded out, but chalked it up to him being drunk. But then he tells me the next day that that night was an apex for him.Now, he is inviting me to vacation with him alone. He's also setting up dinners with couples he is friends with so they can meet me. He admits to talking about me to his other close friends and family he is out to.Up to know, I've assumed he knew where we stood (as being only friends) because I was in a 1 year relationship with another guy during our friendship. But I can't help but wonder if he is trying to turn our friendship into a relationship without the labels. He is a super imposing guy (one reason I don't want to be with him), and it's definitely something I believe he would do. I feel like I should have a conversation with him to lay everything out on the table. But, I worry that if I'm wrong, it could come across as me being hurtful and rejecting his friendship.Any advice on the best way to approach this?