Should i just tell him i am done and be done with it?
YES! (I actually shouted "Yes!" at my computer screen).
I am sorry, but this guy is P-S-Y-C-H-O. He wants to see you cry and beg and break down when he tells you he isn't sure he wants to be with you anymore? Hon, this guy is trying to control you.
Look very hard, you have to be able to see this!! Please, please, please do not get involved with him again, ever. Hiya is right, it is all about HIM. Whatever the reason, immaturity or sociopathic-ness, he is not able to be in a mutually-loving relationship because he is unable to truly care about another person. He wants a girlfriend who will live in fear of him and exist just to make sure he is happy. You do not want this.
Nothing is going to change.