I used to be nervous, too..but then I realized this: some people are curious, some people are rude - people are going to stare. Sometimes I stare right back at them, then smile. Somestimes I just ignore them.
It took me a while to get to that point, though...but when I was younger I didn't do much. I take public transit and I'm working on getting my driver's liscense.