No it is not wrong to feel that way - I too have had similar happen with my ex. I would take something to help me sleep, and he KNEW I was doing that, and I would wake up with him being 'half way through' - and my pain in my body would wake me up (He also knew that sex caused me huge amounts of pain in various areas of my body).
It is NEVER ok for someone to do that to you and the 'I thought you were awake' - good grief, if I could count how many times I have heard that and had a dollar for every time, I would be a wealthy woman.
Can you talk to him and tell him that you feel violated? And perhaps if he is genuinely 'shocked' by this, (which I doubt by the sound of it, but give him the benefit of the doubt perhaps THIS time), tell him that if you feel like having intimate relations with him, you will do so BEFORE you take your medication, or in the morning, and if he gets the 'urge' through the night when you have taken medication to make you sleep - perhaps he can fix himself up.. after all, he has a hand... yes thats a bit blunt, but its true. If you were not in the room or away - how would he deal with his 'urge'? He would have to do the job himself.
HUGS and hope you can get this sorted out soon. Perhaps get some information on sexual assault, and leave it somewhere for him to read... ?