need some advice

I've been liking this guy who I'm fairly certain has interest in me too. we have known each other for awhile and I recently moved far away but we make attempts to meet up somewhat often. When we do meet up, things are very awkward. I have insanely low self esteem and regardless of how much I like this guy any kind of physical contact makes me really uncomfortable. I really do care for this guy and I want to be more than friends with him but my complete lack of confidence ruins this. Not sure what to do. If we did start something, it would be primarily long distance and I don't know how that would work out either.