There was a time in my life (when I was between 17-20 or so) when lots of gay guys assumed I was gay. I'm not sure why that was; probably because I was a homophobic teenager who was fascinated by gay guys and stared at them.
I would say the more likely reasons were:
1. You were thin.
2. You were neat.
3. Don't sell yourself short - you were a good looking guy.
Was ever the past tense more cruelly used?