The Overall Significance of Goats in the Bible

Goats in the Bible generally have a good significance. They were among the clean animals that could be eaten (see Leviticus 11; Deuteronomy 14:3-21), and they were commonly used in sacrifices (see, for example, Leviticus 1:1-2, 10). Goat's hair was spun to make cloth for use in the Tabernacle (see Exodus 35:4-29).

The negative significance of goats comes mostly from their being made into idols and worshiped as gods in pagan religions. This is reflected within the Bible itself in a prohibition against offering sacrifices to such idols. For example:

The priest shall dash the blood against the altar of the Lord at the entrance of the tent of meeting, and turn the fat into smoke as a pleasing odor to the Lord, so that they may no longer offer their sacrifices for goat-demons, to whom they prostitute themselves. This shall be a statute forever to them throughout their generations. (Leviticus 17:6-7, NRSV)

The Hebrew word here translated "goat-demon" is שָׂעִיר (sa`iyr), "he-goat, wood demon, satyr." Goats worshiped as idols in pagan religion is the most likely source of the Baphomet imagery linked in the question.

However, there are also three Bible stories within the Judaeo-Christian tradition itself that have been drawn on, rightly or wrongly, in lore connecting goats with evil and the Devil.

Leviticus 16: The Scapegoat

Leviticus 16 contains instructions for the rituals that the Israelites were to observe each year on the Day of Atonement. A key part of that observance was the ritual of Azazel, traditionally translated "scapegoat":

Aaron shall offer the bull as a sin offering for himself, and shall make atonement for himself and for his house. He shall take the two goats and set them before the Lord at the entrance of the tent of meeting; and Aaron shall cast lots on the two goats, one lot for the Lord and the other lot for Azazel. Aaron shall present the goat on which the lot fell for the Lord, and offer it as a sin offering; but the goat on which the lot fell for Azazel shall be presented alive before the Lord to make atonement over it, that it may be sent away into the wilderness to Azazel. (Leviticus 16:6-10)

In the Hebrew Bible, the word עֲזָאזֵל (`aza'zel) occurs only in these three verses plus verse 26 in the same chapter. As covered in the linked definition, its exact meaning and origin has been a matter of some dispute.

Azazel, or the scapegoat, had a positive function in Israelite worship, in that it symbolically carried away the sins of the people on the Day of Atonement. However, because of its association with sin, in ancient lore it was also sometimes associated with evil and the Devil. For example, in the book of Enoch in the Apocrypha, Azazel is the name of a fallen angel.

This function of the goat as "Azazel," or "the scapegoat" is one source in the Bible for the association of goats with evil.

Daniel 8: The Ram and the Goat

Daniel 8 contains a vision of two great beasts: a ram and a male goat. In the chapter itself, the ram, the goat, and their various horns as described in the vision are identified with the kings of various kingdoms. The male goat, in particular, is associated with Greece and its kings. The final horn that grew on the forehead of the male goat in the vision is described in this way:

Out of one of them came another horn, a little one, which grew exceedingly great toward the south, toward the east, and toward the beautiful land. It grew as high as the host of heaven. It threw down to the earth some of the host and some of the stars, and trampled on them. Even against the prince of the host it acted arrogantly; it took the regular burnt offering away from him and overthrew the place of his sanctuary. Because of wickedness, the host was given over to it together with the regular burnt offering; it cast truth to the ground, and kept prospering in what it did. Then I heard a holy one speaking, and another holy one said to the one that spoke, “For how long is this vision concerning the regular burnt offering, the transgression that makes desolate, and the giving over of the sanctuary and host to be trampled?” And he answered him, “For two thousand three hundred evenings and mornings; then the sanctuary shall be restored to its rightful state.” (Daniel 8:9-14)

This prophetic vision, and its interpretation later in the same chapter, is another source in the Bible for the association of goats with evil and the Devil.

Matthew 25:31-46: The Sheep and the Goats

Moving to the New Testament, in Matthew 25:31-46 Jesus speaks of the judgment of the nations using the imagery of sheep and goats:

When the Son of Man comes in his glory, and all the angels with him, he will sit on his glorious throne. All the nations will be gathered before him, and he will separate the people one from another as a shepherd separates the sheep from the goats. He will put the sheep on his right and the goats on his left. (Matthew 25:31-33)

The king then praises the sheep for doing good deeds for him by doing them for "the least of his brothers and sisters" (verse 40), and condemns the goats for failing to do good deeds for him by failing to do them for "the least of these" (verse 45). And he concludes:

Then they [the goats] will go away to eternal punishment, but the righteous [the sheep] to eternal life. (Matthew 25:46)

Because Jesus here uses goats as symbols of evil people who fail to do good deeds for God and the neighbor, in Christianity goats have also commonly been associated with evil and the Devil.

Summary

Although goats generally have a positive meaning in the Bible, there are a few stories in the Bible and in related extra-biblical literature that provide a basis for the common association of goats with evil and the Devil.