You're not going crazy! Your worries are reasonable, but I don't think you have anything to worry about. Does that make sense? Your doctor could be correct about the walls of your bladder being damaged (s/he was probably referring to the waxy coating that protects the cells underneath--it's what normally protects the bladder from irritation from the salts and other compounds in urine), and maybe that has something to do with your subsequent UTIs. It's hard to say what's a "normal" frequency for needing to go pee, but it doesn't sound like you have frequency problems--just that you don't want to get caught in a situation where you have to go pee but can't get to a bathroom.
Please go easy on the cranberry pills, though. Everything in moderation! Cranberry has been shown to help prevent UTIs--likely because it stops bacteria from binding to the walls of the bladder--but it's also a bladder irritant. I remember reading a study, possibly from Sweden, where the researchers concluded that drinking cranberry juice daily for a week (or maybe a month) reduced the subjects' likelihood of getting a UTI for the rest of the year. The best thing you can do for your bladder is to drink plenty of water and avoid excess intake of irritants (cranberry, alcohol, coffee, citrus, etc.).
Hope this helps!