Some men have sensitivity issues when they wear condoms. Sometimes, it can feel nice in the beginning but, as you say, the feeling goes away after a short while. If you can't get enough stimulation, then you will lose your erection.
If she is loose or overly lubricated, you will not feel as much. Also, if you masturbate, you can lose sensitivity if you use too much hand pressure. The female vagaina cannot clamp down nearly as hard as your hand, so you may need to basically retrain yourself to be able to remain aroused with a lighter touch. Refraining from masturbation for a few weeks, might do it, or making sure you use a very light touch.
Thinner condoms, or rubbing some lubricant on the head of your penis before putting on the condom might help, too.