Was called Shallow for Wanting To Be Friends and Not Date...

I'm 19 and started talking to a guy in my Biology class. Hes a really nice guy and super fun to hangout with but he is very overweight. While hanging out the other night he puts his hand on top of mine and puts his head on my shoulder. Not wanting to lead him on I get up and go to the bathroom, when I come back I sit with a gab in between us. He then says "I really like hanging out with you, would you want to be my boyfriend?" I said "Your sweet for asking but we should just be friends." He gets mad and asks why, say we would be better as friends. He just kept asking why. I finally tell him in the nicest way possible that I'm not physically attracted to him. He asks why again, I say I'm attracted to guys who look more like me(I'm Muscular and pretty toned.) He says "I cant believe how shallow you are!." I called him a dick and left. My question is am I shallow for wanting to be attracted to the person I date? I know looks are not everything but some attraction has to be there right?TLDR; Was called shallow becasue I didn't want to date someone I was not physically attracted to. Am I being shallow?