It's great to hear you're starting to get your life back...taking baby steps is the best way to go. You are on a very small dosage of clonazepam, and 50mg of zoloft is certainly enough to give you positive results (even though your doctor may want to increase your dosage soon or eventually). Now may be the right time to start weaning off the clonazepam- it will be a little scary at first, but if you continue to take it for much longer, you will most likely develop a strong emotional addiction to it, and your tolerance will go up, causing you to need more and more.
Your doctor is most likely trying to avoid this addiction/tolerance, so tapering off now when you're on such a low dosage is your best bet. This is just my opinion, however you can consider this sound advice- right now I am seriously addicted to benzodiazapines (I was on xanax for a few months, then was switched to clonazepam). I am taking up to 5mgs of clonazepam every day, and cannot function without it. I am seeing a naturopathic doctor this thursday because I plan on tapering off both the zoloft and clonazepam. Believe me, you want to avoid benzo (clonazepam) addiction at all costs. If you are extremely anxious about going off clonazepam, ask your doctor for a small "emergency" prescription, and promise yourself that you'll only take it when you absolutely need it.
Please don't be embarassed about asking your doctor personal questions such as your lack of sex drive. She is there to listen and help, and I guarantee she has heard MUCH more embarassing things from other patients. Sexual side effects are extremely common with SSRI's! Oh by the way, I'm taking 75mgs of zoloft, but slowly tapering. Good luck with your appointment, and let me know how it goes!