As good as exercise is for you, it's also a good idea to take a rest day each week. Depending on what kind of exercise you're doing, you may also want to take a week or two off every few months to recover.
I've always been told that if you are strength training you need to give the muscles a day of rest between workouts so they can heal. So if you're working upper body today, give it a rest tomorrow and do lower body instead.
Not sure if you need to rest if doing cardio. I'd think it depends on the intensity.