You have concerns.
First think and get clear to yourself what your concerns are and what your limits are for this relationship.
If you can not accept any oral or anal sex of any kind outside of your relationship make this clear.
Do not make him promise this but say something like “it would make me feel better if you could acknowledge that you understand how I feel and will respect this limit that I have for our relationship.”
You seem very open and communicative with this person and that is good.
But I think you need to talk a little more and maybe get a better understanding of where his thoughts are so that you can no and not worry so much about it.
What exactly do you mean by switching teams? What does the term mean to him?
I ask this question because you use the term gay and the term transgender.
I think the switching teams means something different when applied to each term.
And though they overlap somewhat they are for the most part very different ways of thinking and being.
If he is “into the moment” during sex and aroused by you and being with you (not thinking about other people or looking at porn) then he is attracted to women.
But even that attraction can result from two very different mindsets.
1 One mindset is the love of women… The biological attraction to the form and shape of women.
2 The other mindset that some men have is they have to be “real men” and “real men” do women. For these men they seem to be more in love with being a man then in love with women.
Now they same two mindsets can be applied to the fantasy.
3 Is the fantasy that of an attraction for the form and shape of a man a hard wired biological arousal when seeing a healthy attractive man.
4 Or is the fantasy more the fantasy of being feminine and being attractive and desirable in a feminine way. So that he is more fantasizing about being submissive or feminine then about being “with” a man.
It sounds like 1 applies most and then 4 to your situation. Try and first get and then use this information to strengthen your relationship. There can be many biological and developmental reasons for any of the 4 mindsets.
As far as role playing or using toys. It has to be something you are both comfortable with
If he enjoyed the fantasy with you that is all he would be enjoying … a fantasy with you.
It is VERY different then the REAL thing. Usually better.
Which is what 6foot2-190 was saying.
Most people are smart and can realize that fantasy is good and very different then reality, which can be very bad.
But there are a few people that play violent video games and then go out and shoot people
But most people do not.
So there is a small risk I guess. But if you know that he is logical and not impulsive or an addictive personality you should be fine.
[/QUOTE]
Transgender fantasies...should I be concerned? 
