I wonder the same thing! I just wrote a post about weight gain in the first trimester, because although I REALLY try not to (especially in pregnancy), I tend to obsess about my weight. I am normally a healthy 115lbs and I'm only 5'2, so when I put on any weight, you can really tell. At 12 weeks, I've already gained 7lbs, and I think most of it is in the belly (this is my third pregnancy--2nd one ended at 7 weeks, so I think I "popped" early), although I can tell that I'm beginning to gain some in my face and arms as well, which I HATE. I think it just has to do a lot with your body type, and where you tend to gain weight easily, pregnant or not. I totally know how you feel seeing those thin ladies walking around, looking so nice in their bathing suits or summer clothes, and I feel like I'm just so gross and flabby looking (particularily because I'm still very bloated, and my tummy isn't really hard yet). I'm also so jealous when I see those pregnant women who ONLY seem to get a belly, and still look thin everywhere else!
With my ds, I also gained weight in the face and limbs, by the end of my pregnancy I looked quite a bit heavier all around, however I only gained the recommended 35lbs.
So, I have no answers for you (hopefully somebody will know how we can keep this weight off the rest of our bodies!), but I wanted to let you know you're not alone!!