You can't go through your whole life basing your actions on what somebody else might think of you. It sounds like you have your priorities straight, and I really congratulate you on that! I wish I had been as smart as you when I was in high school!
And as for him thinking that you have found somebody else, your future activities should set him straight. If he sees you NOT hanging around with someone else immediately after you break up, which I understand to be your plan, won't he figure out that you didn't dump him for someone else? And won't he feel foolish for accusing you of that, if that's what he ends up doing?
Breaking up is never easy. If it were me, I would just tell him the truth (that you feel that God is calling you to concentrate on your education at this point in your life) as gently, kindly and quickly as I could. If he gets mad, tell him that you are sorry that he feels that way, but that you have to do what you think is right. That's all you have to say, and try not to drag your part of the conversation out by telling him how sorry you are for hurting him, and that he's given up so much for you, etc., etc. Just say your piece and let him respond. Just my 2 cents...