How weird: I just happened to glance through an article yesterday about parenting, divorce, and the effect it has on children
. It said what you might expect: that unhappy parents tend to raise unhappy children, and an unhappy marriage is a breeding ground for all sorts of unhappinesses.
Actually, I am the product of divorced parents, and I feel that I turned out pretty well. My parents divorced when I was about six years old. They were constantly arguing and mad at eachother about everything, so I think that my mom did what was best for my brother and I when she filed for a divorce. I still see and talk with my dad a lot, so I don't think that I was cheated out of a relationship with him much at all by this. In fact, I don't think that I would have been nearly as happy as I am today if I had continued to witness what I did as a very young child.
It basically sounds like you're unhappy with the relationship and worried about his daughter. If you really don't like this situation, I think that you should consider leaving him (at least until you feel better about what is going on, if ever you do). Don't torture yourself about it. Be honest with him, let him know that it makes you uncomfortable to be around him, and see what he has to say about it. I don't even know how you're putting up with it, to be quite honest. I would be incredibly weirded out by the whole thing...