When was the name of Sinai first applied to the Sinai peninsular?

According to R. El. Pirke, Sinai received its' name "only after God had appeared to Moses in the bush."

Different Names.

—In Rabbinical Literature:

The Rabbis consider "Sinai" and "Horeb" to be two names of the same mountain, which had besides three other names: (1) "Har ha-Elohim" (= "the mountain of God"), the Israelites having received there the knowledge of the divinity of God; (2) "Har Bashan," the latter word being interpreted as though it were "beshen" (= "with the teeth"), that is to say, mankind through the virtue of this mountain obtains its sustenance; and (3) "Har Gabnunim" (= "a mountain pure as cheese"). The names "Horeb" and "Sinai" are interpreted to mean, respectively, "the mountain of the sword," because through this mountain the Sanhedrin acquired the right to sentence a man to capital punishment, and "hostility," inasmuch as the mountain was hostile to the heathen (Ex. R. ii. 6). Shab. 89a, b gives the following four additional names of Sinai: "Ẓin," "Ḳadesh," "Ḳedomot," and "Paran," but declares that its original name was "Horeb" (comp. Midr. Abkir, quoted in Yalḳ., Ex. 169); according to Pirḳe R. El. xli., it acquired the name "Sinai" only after God had appeared to Moses in the bush ("seneh"; comp. Sinai, Biblical Data).

What does the Bible say:

Mount Horeb.

—Biblical Data:

Mountain situated in the desert of Sinai, famous for its connection with the promulgation of the Law by God through Moses (Ex. xix. 1-xx. 18). The general opinion of modern scholars is that the name "Sinai" is derived from the name of the Babylonian moon-god Sin. Mount Sinai is often referred to as "the mountain" (that is, the mountain par excellence), "the mountain of Elohim" (Hebr.), and "the mountain of Yhwh" (Hebr.; Ex. iii. 1, iv. 27, xviii. 5, xix. 2, et passim; Num. x. 33), and in many other passages it is called "Horeb" (Ex. iii. 1; Deut. i. 2 et passim). The Biblical text, indeed, seems to indicate that this last was its proper name, while "Sinai" was applied to the desert. According to one theory, Sinai and Horeb are the names of two eminences belonging to the same range; if this be so the range became prominent in the history of the Hebrews some time before the promulgation of the Law. When Moses led the flocks of his father-in-law to the desert and came "to the mountain of God, even to Horeb," an angel appeared to him from a flaming bush, and then God Himself spoke to Moses, telling him that where he stood was holy ground, thus foreshadowing the great event that was to occur there. From that mountain God persuaded Moses to go to Pharaoh and deliver the Israelites from his yoke. After the Exodus, when the Israelites who had encamped at Rephidim were suffering with thirst, Moses, by command of God, smote water from a rock in Horeb (Ex. xvii. 6).

As for as the etymology is concerned here is what some scholars say:

According to biblical scholars, Sinai most likely derives from the name of Sin, the Semitic lunar deity.2[3] In the Hasidic tradition, the name Sinai derives from "sin-ah," (meaning hatred), in reference to the other nations hating the Jews out of jealousy, due to the Jews being the ones to receive the divine laws.[4]

Additionally, classical rabbinical literature also mentions the mountain having other names:

Har ha-Elohim, meaning the mountain of God or the mountain of the gods[3]

Har Bashan, meaning the mountain of Bashan; however, Bashan is interpreted in rabbinical literature as here being a corruption of beshen, meaning with the teeth, and argued to refer to the sustenance of mankind through the virtue of the mountain[3]

There is nothing written in stone on this subject!