Hi there, if it's an allergy reaction, there would be immunological symptoms like hives, swelling, itch, difficulty breathing. Well, I'm not an expert here... but I can share my experience... I can't take coffee. You mentioned caffeine so I assume you drink lots of tea, cola, coffee?
Anyway, whenever I take coffee regardless with or without food, my stomach starts to feel yucky and I will throw up. I get dizzy, breathless and generally sick! Although I like the aroma of the coffee, I know I have to stay away from it.
Since caffine is making you sick, should you stay away from it then? Hope you find an answer to your problems. Take care!