They are wrong. As long as you have normal periods, your weight shouldn't have anything to do with it. I am 5'2" and normally 95 lbs (about 10 lbs underweight according to the "charts", but like you, I eat healthy for the most part and have a high metabolism). I got pregnant with my daughter when I was 21 the first time I had sex without a condom - no problem there...LOL! I am pregnant with my 2nd child (I am now 31 and married to someone else) and got pregnant the 2nd cycle after stopping the pill. I have a regular cycle and my weight had no bearing on my ability to get pregnant either time.
I did have difficulty gaining weight durig pregnancy with my first, especially in the beginning. I ended up gaining 25 lbs by the end (the minimum they wanted me to gain) and was back to my normal weight within just a few months without much effort. My baby was full-term and healthy at 6 lbs 7 oz. With this one, I had gained 1.5 lbs by my 9 week appointment, which is amazing for me. You'll be fine. Don't listen to everyone else. Why cause yourself additional stress and force your body to unnaturally gain weight?