I don't think you'll look jealous by bringing it up to him, and if you are jealous so what? You have the right to any emotion you please
I think it's reasonable that two people who dated for 4 years would remain close friends..I mean they probably knew everything about each other, right? It's probably easy for the two of them to communicate and have fun with each other. Everyone needs a good friend they can depend on; your boyfriend is one of the lucky ones who found a good friend in his ex, someone he was already close to. Be happy for him!
I can kind of relate to this though. My boyfriend was best friends with a girl before he met me, and they liked each other (he even asked her out once.) Anyways, they stayed friends even when him and I were together. It bothered me, mostly because he never talked to me about her. EVER. I don't even remember how I found out they were best friends, he certainly didn't tell me. Well, I never said anything to him because they aren't friends anymore and when they were our relationship was only like 2 months old so I wouldn't have felt comfortable enough to say anything..I sure as heck would now though.
You should be able to talk to him. Tell him you are glad that he has such a good friend, but you need him to reassure you that she's only that; a friend.
Take care!