An allergy is the response of the body's immune system to normally harmless substances, such as pollens, foods, house dust mite and cats. Whilst in most people these substances (allergens) pose no problem, in allergic individuals their immune system identifies them as a threat and produces an inappropriate response. It is this response that causes you the problem( e.g. itchy skin, swollen lips, difficulty breathing).
A food intolerance is difficulty digesting certain foods and having an unpleasant physical reaction to them such as bloating and diarrhoea. Intolerances are real but unfortunately there are relatively few tests to diagnose them. You should suspect them if your symptoms resolve when you remove the suspected food from your diet.