In love with a twice my age

Im 24 and I've been seeing this man for 3 years now, just random hookups every now and then, but I fell badly in love with him. I love everybody moment with him, there's literally nobody that makes me feel as him, but he always said that he being 20 year older is too much and it's not stable, he thinks when he gets 60 year old I will ditch him and he cant take this risk; basically he told me that once he finds a bf we won't be seeing each other anymore. He wouldn't want me to always go back to him cause he doesn't want to hurt me getting more attached to him, but I cant help it I want to spend as much time as possible with him.Honestly I feel devasted and I just wanna cry because he's truly the perfect man and one not seeing him anymore breaks me. The fact that makes me so angry is that I would be perfect for him too if it wasn't for my age.