I have to disagree with your doctor. If you have been taking even a low dose of valium for several years and then stop taking it suddenly, what you are feeling is more than likely withdrawl. It's just not the agony style withdrawl that comes with stopping large amounts of meds.
I think that because you were on a low dose, the withdrawls just aren't as pronounced and terrible as what others on higher doses would go through.
I was on low doses of valium before and stopped cold turkey and went through withdrawl that was not horrid. I felt a bit jittery and foggy brained and had some mild restless legs at night, but it faded fairly quickly.
Any time your body is occustomed to having something on a regular basis for a long time and that substance is stopped, the body lets you know it is missing whatever it has been used to getting.
It should subside fairly quickly for you. If not, then please do speak with your doctor.