I quit the drug after reading these horror stories you mention (I do think that it's safe to quit the drug at any time though but you should always consult your doctor and gradually decrease your dose). I hadn't taken it for long enough for it to have any effects after I quit. I think you have to have taken it for more than a year to see that sort of reaction but don't quote me on that.
My bigger brother quit it by gradually taking less, sometimes he just forgot to take it one day and then maybe for two days. He didn't have any side-effects because of that which I am aware of.
The drug also works very slowly, you might not notice the effects but you could ask those around you whether they notice a difference in your mood/attitude e.g. It's definitely worth it if it works