Unfortunately, it is a vicious cycle. You might have a flair up, then get worried that you will have one, then you do because you worried - it just sometimes seems to never end.
I suffer from these panic attacks when I know that I have something to do or somewhere to be, especially in the mornings. So what I do is take a relaxing shower or bath before I go to bed. I do some deep breathing exercises and listen to some soothing music. A warm shower first thing in the morning also helps relax my tension.
My mom also suffers from IBS and has a much better handle on it. Her attitude is, "as long as there is a bathroom somewhere, then go and have fun, or do what you need to do." I sometimes have trouble following that advice, but I find that once I have a "good" day, and can do even just to the grocery store without being paranoid, that it gives me some confidence and reassurance that it will be an okay day.
Hope some of this helps. Just know that you're not alone!