Well I'd ask again if you can go with him. But he may not feel comfortable talking about his feelings (for you as well as other feelings) to someone else in front of you, so you may just have to hope he does OK on his own.
I'm sorry you broke up

. I hope it works out in the long run. It is great that you are not giving up on him, but make sure you're taking care of yourself well enough that you don't let your worry for him run you down. Good luck!
[edit] Oh had one more thought, if he doesn't want you to go with him, you could offer to write a note to the doctor explaining your concerns, that way he can let the doctor know your input without having to have you present.