Goats have beards while sheep do not, and sheep have a divided upper lip (philtrum).
Both male and female goats have horns, and hornless varieties are very rare, while hornlessness is more common among sheep.
A goat's tail will almost always be short and pointed upwards, unless they are scared or sick, and a sheep's tail will hang downwards.