Tap water is safe only in the largest and most developed of African cities. If you know that you have a sensitive stomach it may still be advisable to seek out bottled water to avoid missing part of your holiday! If you are traveling to remote areas, plan on drinking only bottled water or water that has been boiled for at least one minute.
If you suspect that you might not have access to bottled water, bring iodine tablets with you to disinfect your water. Also, be mindful of the ice that is put in your drinks! Ice is almost always made with tap water and may foil your efforts to avoid an upset stomach, so kindly ask your water for your beverages with no ice.