I know all to well about jumping to the worse possible conclusion, to then find out I was wrong. The problem really lyes with feeling insecure within the relationship you have.
It is interesting that he has now contacted you, the fact that he wants to talk to you is also quite interesting. Ofcourse it could go either way.
I would defenitely try to make some time for him to have the chat that he is asking for, take him up on that sooner rather than later. Otherwise your in limbo for longer than you really want.
I wish you the best Tyger