Are you flaccid or erect while doing this? Seems flaccid would be easier. Sounds like you're pulling it out then twisting it..
At any rate I know the penis is very flexible flaccid and unless you are extreme with it you probably wouldn't hurt it. Only you know how extreme you are in what you're doing.
In all seriousness do you think you're damaging your penis? Have you had any complications or abnormalities from doing such? How long has this been going on? Is it just a habit or a form of masturbation? Do you derive pleasure from it?
I don't get pleasure from it. My brother has also noticed I have a habit of it. I have probably been doing it for a few years now. I ask because I think I might have some nerve damage. But if it isnt painful when I do it, it's either a loss of sensitivity or I'm just not causing any harm. Thanks.