I am somewhere between 6'2'' and 6'3'', and I wiegh about 170 lbs. I have decent strength in my legs, but my arms and chest are kind of pathetic. I'm just a little bulgy in the belly region, and I suspect I'm starting to develope a flabby face.

What I want to do is trade fat mass for muscle mass. I don't want to be a giant, but toned. I want to be able to move fast and dexterous without effort and without losing much energy.

So, I'm thinking that swimming would help the dexterity part a LOT, also jogging, jump rope, and maybe hacky sack too. But what could I do for muscle mass? I think a lot of easy exersises like push-ups, sit-ups, and light wieghts help.

Am I on the right track?

You are on the right track, but don't be affraid to hit the gym. I'm about your height, medium build. At one point I had 4% body fat, a great physique and still had not lost much in the way of speed and agility. I weighed about 185 pounds. Later in life, I kept going to the gym, put on a bunch of muscle mass and ended up weighing 205. I still looked pretty good, though, a little like a meat-head....but I was definitely slower. Until a few weeks ago, I was heavier than that, and all of my muscle had turned to fat, but that's a different problem altogether...one that I'm taking care of.

At that height and 170lbs, espcially if you've got a little flab, you need to do some weight lifting. Lower reps, higher weight. But, in my experience, if you just put on bulk without doing some cardio, you won't lose much fat, you'll just get bigger. So, I would recommend solid 60-90 minute (3 days a weeks), low rep, high weight work outs followed by swimming or running or cycling or some other cardio vascular exercise. If you are wanting to bulk up your upper body, then I would recommend sticking with the swimming.

Also, resist the tempation to significantly increase your calorie intake. When you start lifting weights, your appetite will increase, because your body is trying to build muscle. If you feed it everything that it wants, you'll put on the muscle, but you won't decrease the fat so much. If you eat just a little less than you want, your body will start breaking down the fat, so that it has the energy to meet the new demand you are placing on it. You may not build mass as fast, but you'll lose fat faster.