Closeted guys

I am an openly gay male, I have been for the last few years, and I’m happy with that. Im not going to delve into my personal life too much but i dont do relationships, never have done, i prefer one time things, and as a result of that lifestyle I’ve done stuff with a few people. But the one thing that always saddens me is that every guy I’ve been with always tells me not to tell anyone that they are bisexual/gay. They are scared of being bullied, seen differently, or they feel that they have been unfaithful to their religion (christianity and Islam). On the religion perspective i can understand why they dont want their sexuality to be open, but its the bullying and being seen differently thing that really upsets me. Does it really matter what other people think of you? I see these people be so down and unhappy about themselves and their lives because they are closeted, and it really is upsetting to watch. when i talk to these people about their unhappiness due to them being closeted, they blame me for ‘making them do what they did’ when consent of both people is the most important thing to me. They generally start getting physical aggressive towards me and other people, they stir up a big scene, and yet they still feel that they cant tell anyone whats wrong. Like i had one guy tell me at prom that he is bisexual, but then afterwards, when i ask him, he denies it and says he will beat me up if i tell anyone. this is just so upsetting for me to watch, has anyone else seen this happen to them?