I think that problem is probably more common than is talked about. I agree with him though-
certainly don't beat yourself up about it. I would let him know, especially since it seems as though he's not going to scoot you off to another doctor. He sounds like a good doctor, who does have your best interest at hand (due to the fact you said you told him before, and he was working with you on it).
I'm only 18, but I would say take the risk and trust yourself, and that your doctor would handle it in the correct manner- but he does sound totally understanding about it.