I have read that in rare cases, a pregnant woman may transmit the infection to her baby through the placenta if she gets herpes for the first time in her first trimester. I want to say you should be ok since you had gotten pregnant after you had your "initial" outbreak. Most of the risk that I have read about really is more in the third trimester if a woman gets her first outbreak.
I wouldn't worry much over it, just get your ultrasounds to monitor babies growth
I think you should be fine though , but this is definitely your OB should be able to answer.