No med should be stopped without your doctors okay, but at that dose, I can't imagine you would have more than a few days feeling not quite up to par, I doubt if you would have any serious withdrawal issues.
Still, Best to check with your doctor first, some people are more comfortable if they taper down to a single dose before stopping.
HTH, Fabby