What does it mean when a stranger stares at you for more than the average time of a random stare?

I find it hard to stare back, but I can see them still staring from the side of my sight