We have told family that DS is intolerant. If they ask, we tell them how it bothers him - sore stomach, gas, night wakings, everything. I always try to look at it as them trying to learn, because they have never had to deal with that sort of thing. So I try to explain to the best of my ability what it is like for him. For situations where it won't be a factor very much, or you may never see the person again (like SIL's wedding a few weeks ago), we just say he is allergic. Really we shouldn't use that word, but it is just easier and less stressful.
For situations like birthday parties, or groups where someone else will be providing the food , I always bring my own. If it is a birthday party, I call the parent ahead of time, explain that DS can't have X, so is it okay if I bring the food for him, so they don't have to put the extra effort in. So far we have only ever had one child ask why he was having different food, and we just explained that what he was eating would hurt DSs tummy. That was enough for him. For playgroups, I bring my own food, plus extra in case any kids want to share. I don't bring it out until I see what was brought. If it is something that DS can't have, I pull the mom aside, explain, and ask if it is okay to offer the food I brought to DS.