There's no need to panic.

Most people have oral herpes, usually picked up in childhood from kissing relatives and friends. Most people know that cold sores are contagious but often don't realise it's a strain of herpes. Because cold sores are usually transmitted in non-sexual situations, they don't have the stigma that genital herpes does, even though the two viruses are very similar.
It's *possible* that what you have on your genitals could be oral herpes transmitted down there by yourself, but all things considered I think it's more likely that it's something else. It's possible you could have transferred it there if you touched your mouth near the time when you were having an outbreak (OB) and then immediately touched your genitals, so that your hand carried the virus from one location to the other. However, the virus dies very quickly once the air hits it, so that would be most likely if it was combined with some saliva, since it thrives in warm, wet environments.
Sometimes people will use saliva as a lubricant during masturbation, and with the abrasion that can happen from the friction (giving the virus an entry point into that location), it could happen that way. Although not unheard of, it's not really common either.
Basically, you should wash your hands before touching yourself or anyone else, after you touch your mouth when you've got a cold sore or even just have that itching or tingling there.
I still think you should get your doctor to take a look at it ASAP before the symptoms heal too much for him/her to see it clearly. Your doc might look at it and go "Oh, that's just such-and-such" and put your mind at ease. For example, ingrown hairs look similar to herpes and can happen if you shave around there, so that's one possibility.
Good luck! Let us know how you make out.
TheOneInFour