Maybe you can try a little more protein, less carbs in the morning/afternoon. I know sometimes that is difficult since your cravings change. Try maybe yogurt or a smoothie as a snack instead.
You tend to gain more the last 4-6 weeks of pregnancy because the baby is growing at a faster rate. Your doctor may be concerned that if you are gaining 6 pounds a month early on that you could be putting on too much weight by the end of your pregnancy. I know everyone is different, but for example when I was carrying my son I actually lost about 8 pounds in the first 3 months. I only gained 4 pounds between month 3-7. In the last 8 weeks I put on 15 pounds.
I believe you were underweight to begin with? If you gained a pound a week that would be 40 pounds which would be normal if you started underweight. If you gained at the rate you are now 1.5 pounds a week that would be 60 pounds by the time you deliver. Your weight gain right now is primarily increased blood supply, water and fat. The baby's weight is hardly noticable!
I would take a list of what you are eating, maybe journal...to your doctor so he/she can see that you are eating healthy foods. And definately ask next time what he/she is expecting.
In the end it is your body and you have to carry it around. I'm surprised that a doctor would comment about weight, usually they don't notice too much. To ease your mind just ask specifically what the concern is.