I'm not familiar with the science of why and how smoking lowers androgens, but it does. It might make you more mellow, but avoiding it more often should help with your problem and help you stay focused during sex.
There are a lot of oily plants that have estrogenic effects that you probably want to avoid: soy, flax, canola, etc. Hemp might just be like other oily plants in having an estrogenic effect. You should probably learn where BPA is to avoid it too. It only takes the tiniest amounts to disturb hormones.
I've heard that tribulus is often worthless depending on the source. I'd only get something with (real) good reviews. What have you tried for bodybuilding? I've heard some whey protein has estrogenic effects. I've heard good things about taurine for androgens. (Taurine deficiency might be pretty common?)
Another thing that would help keep your focus during sex is increasing penile sensitivity. Though I could probably stand to keep that spiel to myself unless you actually wondered.
Wow, I just read some men saying tribulus caused gynecomastia in combination with other typical bodybuilding supplements, so that might be good reason to avoid it.