Then I don't think you should give up at all!!! After all, didn't you both say til death do us part, in sickness and health. HE'S SICK
He's gotta get some help. I'd be asking him if he remembered the law; and that she's way too young. Her daddy might even be willing to tell him that but it might not be as easily as if you tell your husband. Anyways, I'd ask him why he's disrespecting you, your parents , yours and his child, the girl etc.etc.etc. Wonder how he'd like it if the child found out one day when old enough to understand whats going on that Dear Ole Daddy tried to put the move on all the young girls and other women while married to mommy.
That really isn't something everyone must know. But he needs to stop the childish behavior and start thinking with his brain. Maybe you can sit and talk with him since he's not abusive to you and no chance of it happening. Of course, we don't fight fire with fire but wonder how he'd like it if he found you attracted to a man younger than him or better yet an older more mature 30 y/o
Just lay it in the line with him. Get him some indepth Christian counceling and for yourself as well.
Let us hear from you very soon as to what you do. Best Wishes
Be Good To Yourself & Your Baby