Your breasts are never empty. You are always making milk, so there is no need to wait for them "refill", just nurse your baby when she's hungry. If you just pumped it will be fine, you should still get a let down.
Just a little FYI - there is technically only one kind of milk (even though we refer to breastmilk as having hind milk and fore milk). All milk has fat in it, but it's when that fat sticks to the ducts and separates out of the foremilk that it looks more watery, or sometimes even blue-ish.
When you nurse long enough to let your baby get the "hind milk" you are essentially emptying the breasts of the watered down hind milk, and as your body makes more milk it sort of "flushes" out the fat that is stuck, and voila - hindmilk! Plus, the new milk has not had a chance to separate to allow the fat to stick to the ducts.
Hind milk is REALLY REALLY REALLY good for baby, so do not be afraid to nurse right after pumping. Hind milk will fill baby up faster and sustain them for longer, and it's great for weight gain. This is why it's better to feed on one breast at a time, because it can take a whole feeding to get to that hind milk. In fact, in women with over supply, it takes several feedings to get to the hind milk, so they'll feed on one breast for several feedings before switching so that baby gets the hind milk. It's called "block feeding".
But I digress.. just go ahead and let baby nurse when ever she's hungry, even if you just pumped.