I had that before or something very similar under only one arm like you. The itching drove me crazy, so I went to the doctor. He said it was folliculitis, which happens when the hair follicles get inflamed or infected, usually from bacteria on your razor (but sometimes from other things, too.)
He told me not to shave for several days, to use small amounts of Neosporin on the delicate (but not irritated) skin around the rash, prescribed a hydrocortisone cream to put on the rash when it itched, and prescribed a round of mild antibiotics. He also said that disposable razors should really be thrown away after each use to prevent any bacterial contamination. I followed his advice, and several days later, the rash was gone.
If you are on good terms with your doctor and see him/her regularly, you might be able to call the doctor's nurse, describe the problem, and the doctor may be able to call in a prescription for you. Otherwise, you may need a quick visit to the doctor or a "first-care" type clinic. Good luck!!
P.S. You may not have what I did; I don't know. I just thought I'd share!