Yes, anxiety can most definitely affect your stomach and bowels. You do not necessarily need to have pain though, but can just anxious and have diarrhea. This is what happens.
Perhaps you have some lactose intolerance too, if you said that you could once drink milk without problems, but are now getting diarrhea from this, then perhaps you should try cutting dairy out of your diet and adding a fiber supplement such as Metamucil and seeing this helps. It could help to bulk up your stools a little for you. Take it twice daily, but remember to drink plenty of water with it.
Also, as you seem to have had diarrhea a lot, it could well be worth you taking some Acidophilis capsules to restore your gut flora which is cleared out by diarrhea. Keep the capsules in the fridge and take them twice daily on an empty stomach and this should help restore the balance and could help stop the diarrhea somewhat in conjunction with the Metamucil.