I too know where your coming from.
NO, not at all do I think that was a crazy request...
Yes, raising a child alone and then trying to have a relationship is not easy, but as we all know the Child does come frist, relationship comes second.
I had an issue years ago where the guy I was dating brought up all this stuff that he really loved me, liked my son (who was two at the time and this guy was awesome w/my son) but mentioned how we couldn't just get up and go away for a weekend, do spare of the moment things, blah, blah, blah, I too told him I understood..mean while I was feeling like crap under my skin for him saying this, yeah, I know it was true...call me young & naive but I just didn't see my SON as a problem to him or myself or our relationship...
I found the problem to me was HE never ASKED me to do anything so How would he KNOW if I couldn't just take off for a weekend? That is what bothered me the most...My ex-husband was very involved and still is with his son so if I had asked my ex to watch our son...He would have said YES..
I never gave the guy another chance...My Pride was Hurt so I dumped him.
I hated that he used my child for a reason to think his love for me needed space...
You are alot smarter then me and mature enough to know what the reality of life is being a mom and raising a child.

I hope it works out for you in the end and the guys sees how speical you are.