First off, don't ever let someone make you feel bad about yourself. You are who you are

I'm so sorry to hear this has caused you to such stress and sadness. That shouldn't be the case.
If he isn't willing to tell you, or discuss the problem, then that should also tell you something. What you described seems strange. How long after you met, did the sexual aspect hit such a low? If it was after quite some time, maybe he's jsut tired of it all the time. Sex is usually at a high peak in the beginning etc. It starts to die down as time goes on(in some cases), or just not as frequent etc.
It's really hard to suggest something, when you mentioned that an arguement pursues after the issue is adressed or brought up. That right there could be a warning sign of something. Maybe he's insecure, maybe he's developed some problems with sexual function, maybe he's just lost interest, maybe he's becoming more distant for a variety of reasons. I can't say, nor jump to conclusions, being that I don't know you guys or your situation.