I think 4-5 times a day and soft is a little much, perhaps he has a mild lactose intolerance. cut down on dairy and foods with lactose for a feww days and see if it makes a difference, by the way there's even lactose in french fries!
About the training, I am also training my almost 3 year old son. He started about 2-3 weeks ago and is doing great so far, he figured out both the pooping and peeing but his accidents are with the peeing, the pooping part is excellent.
I have also been told (numerously) that the peeing is supposed to come first and that the pooping takes a little while longer but all 5 of my kids learned pooping first http://www.healthboards.com/ubb/confused.gif
let us know if taking away lactose for few days helps. You can buy lactaid pills that you put in milk and leave it cure for 24 hours, cheaper than buying the carton of milk - I did this with my eldest when she was between 1-3 years old. She outgrew the lactose intolerance around 5-6 years old and she is now only slightly affected when her system is down (a cold, lack of sleep,etc) she's soon to be 12.
Also I'm curious what kind of formula your son had as a baby? or did you breastfeed? Because my eldest was extremely colicky and had very loose stools as an infant and baby she was put on soy based formula (but it makes them gassy - they just toot away!). Since she was lactose intolerant all my other babies were put on either a soy based formula or the one with cows milk but with the lactose removed.