I am sorry that you're isn't understanding of your lifestyle. I understand that can be tough, but maybe that's part of why you enjoy it. Perhaps you take pleasure in the fact that it's so taboo and forbidden.
I am trying to imagine my husband in women's clothing and it's really hard. It's just so not his personality and it's not what I am into sexually. I remember an old boyfriend of mine hide his penis between his legs as a joke and I told him that he came very close to never getting laid again.
I believe people have a hard time with the imagery. It probably causes their brain to short circuit and they can't process the information. I also believe that people make a lot of assumptions. When you confessed to your friends about being a crossdresser, their minds raced with other possibile secrets you may have. They might wonder why you're telling them this, they might even worry that you're coming on to them.
You have them thinking about you. You have them thinking about sex. You have them thinking about you and sex and a hailstorm of other things and it's just too much for the to deal with.
Have you sat down with your wife and tried discussing it? Did you tell her about crossdressing or did you just surpise her one day by showing up in high heels and a pretty, pleated skirt? Maybe if you slowly work up to it, allow her to get used to the idea, she'll be more open to it.
Unfortunately, it is more likely she is questioning your masculinity and sexuality and these things really need to be addressed.