Why did the pre-Islamic Arabs identify themselves as descendents of Ishmael?

That is actually a false perception that All Arabs consider themselves to be descendants of Ishmael.

According to Arab traditions, Arabs categorized themselves into two main branches¹:

1. Adnanites عدنانى
2. Qahtanites - قَحْطَانِي

Among these two, only the Qahtanites, those who are believed to have originated in Southern tips of Arabian peninsula (Mainly Yemen nowadays), were considered to be "original" or "pure" Arabs.

Pictured below, a tree of Qahtanite Arabs and their tribes:

Note the tribal leaders, many of them are founders of important tribes such as Banu-Kalb, Banu Juhainah, Banu Aslam, Banu Tayy, Banu Aus, Banu Khazraj etc.

The Adnanites were called "Arab-e-Mustarba / عرب مستعربة" or "Arabized Arabs". They are the ones who are supposed to have been descended from Ishmael, son of Abraham through Ishmael's descendant Adnan and had adopted the Arab identity and culture. As Abraham was founder of Hebrews and Hagar was Egyptian, that would make the Adnanites to be Hebrew or Hebrew-Egyptian in actual descent as per the legend. But since they adopted Arab identity and culture, they became "Arabized Arabs". Pictured below, a tree of Prophet Muhammad's descent from Adnan:

Again, note the tribal leaders, many of them are founders of important tribes such as Banu-Ghatfan, Banu Sulaim, Banu Thaqif, Banu Assad, Banu Tamim, Banu Hashim, Banu Umayyah etc.

The Adnani and Qahtani rivalry may have caused this legend or the legend may have some truth after all (Assuming that Abraham and Ismael and Isaac existed in the first place of course), who can say now? There can be no definite proof for or against this until there is a proof for or against the very existence of these people in the first place.

As already mentioned, Arabs held that legend long before Islam so it wasn't Islam which created that legend. Not to mention, it wasn't just them who held that legend of Ishmaelite descent. I remember a letter from a Byzantine clergy man too who referred to Arabs as Ishmaelites. Sadly I can't find it right now but when I do, I will link it here.

Moreover, Yahweh is also known as Elohim in Hebrew bible. The Word Allah is the Arabized form of that word. So yes Arabs did believe in Yahweh. The deities they worshiped before Islam were considered to be daughters/sons of Allah, while the Abrahamic deity Allah remained the highest deity of even the polytheist Arabs. Even before Muhammad, as linked by Ted, Many Arab thinkers tried to preach monotheism, claiming it to be the original faith of Ishmael, one which was corrupted by polytheism. Although some Abrahamic traditions were preserved, to quote a few:

1. Belief that Kaaba was built by Abraham
2. Belief in Supreme deity Elohem, Yahweh, Allah, choose your word.
3. Circumcision in infants

Islam is considered by Muslims to be restoration of original faith of Abraham which according to them had been corrupted and changed by the Jews, Christians and Polytheist Arabs. That's precisely the reason why Muslims consider People like Adam, Abraham, Isaac, Moses, Jesus etc to be prophets of Islam as they see Islam as restoration of the original Faith, not a new religion in itself.

We do know however that:

1. Not all Arabs claim to be descended from Ishmael

2. The claim/legend pre-dates Islam by several centuries

3. The Adnanites Arabs who claim that are considered to be Arabized or culturally assimilated Arabs, not pure/real ones.

4. Adnanites and Qahtanites have been traditional rivals. Their in-fighting has been observed many times in Pre-Islamic Era², Umayyad Era and Andalusian Era.

5. Arabs and Jews are both Semitic people so some distant kinship can't be completely ruled-out.

6. Arabs did believe in Yahweh/Elohem/Allah even in their polytheistic days.

7. Arabs do not consider Judaism to be original faith of Abraham. They consider it to have been altered just like they believe Christianity is also deviated from the original Abrahamic faith. They see Islam as restoration of Abraham's original faith, not a new one.

_{1. The first documented instance of this division that I could find was in Umayyad Era which is post-Islamic. Ibn Kathir and Ibn Khaldun, noted Arab scholars have also noted this divison however they were both born in post-Umayyad era. Nevertheless Pre-Islamic poetry does contain claims of being descended from Ishmael as a matter of pride. I couldn't find any documented references of this division before Islam which may have been due to the fact that Arab literacy was very low and they were a people who mainly relied on oral tradition. Or it may have been that Umayyads created this division (Note, division, not the legend itself) for some unknown political motives (Divide and rule?), who can say now? It could also be political opponents of Umayyads who may have sowed this division as Umayyads relied completely on Arabs for their power while holding non-Arab subjects out of power, which lead to them flocking to banners of opponents of Umayyads. So it would be imperative for any rebel leader to sow dissent in supporters of the ruling Imperial dynasty.}

_{2. As per legends, not documented history.}