Should I be mad at my bf?

My bf and I have been together for 3 years. About 6 months ago, I got injured during a triathlon, and have been unable to exercise. I usually initiated going with my bf. Since, we stopped going to gym, we've both gained about 20 pounds. I got back into working out about a month ago, and can't convince him to go. I said to myself, I'm not going to force him because everyone has to do things at their own time. Yesterday, he told me a coworker said another guy in the office said "he used to be sexy and he's let himself go, what happened." My bf was crying when he told me this story and said he's coming back to the gym this week. I got so upset, because I said I try to be encouraging but never made him feel bad, now some guy in the off thinks he's not hot anymore and he'll do something for him. We got in a huge fight over it, and didn't speak last night. I don't want it to ruin our valentines plans, but im still pissed. Do I have a right to be?