I'm not sure if this is how it came about but it could have contributed to it. This was told to me by a friend so I don't have sources for it right now. I'll look for sources on this later.

In the early church days Christians were persecuted by both Jews and Romans. In order to keep their worship services safe and secure they needed to develop a signal to inform everyone that they were 'safe' and not there to harm everyone or be a spy. So they used the hand gesture of making the sign of the cross on their bodies, the same sign that many Christian faiths use today while praying.

If you were outside of the church and looked in, you couldn't see what was being done or what sign was made. However, if you were inside the church, you could watch people as they came in to determine if they were a Christian or not. Anyone coming in who did not know the sign would be noticed quite easily.

”The shape of the [two-beamed cross] had its origin in ancient Chaldea, and was used as the symbol of the god Tammuz (being in the shape of the mystic Tau, the initial of his name) in that country and in adjacent lands, including Egypt. By the middle of the 3rd cent. A.D. the churches had either departed from, or had travestied, certain doctrines of the Christian faith. In order to increase the prestige of the apostate ecclesiastical system pagans were received into the churches apart from regeneration by faith, and were permitted largely to retain their pagan signs and symbols. Hence the Tau or T, in its most frequent form, with the cross-piece lowered, was adopted to stand for the cross of Christ.”

-An Expository Dictionary of New Testament Words (London, 1962), W. E. Vine, p. 256.

The way it was explained to me once is that it was a literal interpretation of Christ's commission to take up the cross and follow him.

"Then said Jesus unto his disciples, If any man will come after me, let him deny himself, and take up his cross, and follow me." (Matt 16:24; see also Mark 8:34; Luke 9:23; Matt 10:21;)

The Encyclopedia Britannica attributes Emperor Constantine as the one that initiated the use of the cross as a symbol of Christianity:

[...] Before the time of the emperor Constantine in the 4th century, Christians were extremely reticent about portraying the cross because too open a display of it might expose them to ridicule or danger. After Constantine converted to Christianity, he abolished crucifixion as a death penalty and promoted, as symbols of the Christian faith, both the cross and the chi-rho monogram of the name of Christ. The symbols became immensely popular in Christian art and funerary monuments from c. 350.

Long before the cross was the staple of Christianity, Heraldic symbolism was used to help identify units on the battlefield. The two places in which the symbol of the cross featured most prominently were in divisions and in Ordinaries (the primary symbols on the emblem). From the Wikipedia article on Heraldry in relation to Ordinaries,:

In the early days of heraldry, very simple bold rectilinear shapes were painted on shields. These could be easily recognized at a long distance and could be easily remembered. They therefore served the main purpose of heraldry: identification.

In another source I read on the subject (but do not have as a reference), it pointed out that early ordinaries were created from simple euclidian shapes because they were easy to reproduce and easily distinguishable from one another. The Cross was used as such a shape for identification well before the conversion to Christianity primarily for this reason.

From the Wikipedia article related to the Cross as an Ordinary:

Pre-Christian

Due to the simplicity of the design (two intersecting lines), cross-shaped incisions make their appearance from deep prehistory; as petroglyphs in European cult caves, dating back to the beginning of the Upper Paleolithic, and throughout prehistory to the Iron Age.[citation needed] Also of prehistoric age are numerous variants of the simple cross mark, including the crux gammata with curving or angular lines, and the Egyptian crux ansata with a loop.

Speculation has associated the cross symbol - even in the prehistoric period - with astronomical or cosmological symbology involving "four elements" (Chevalier, 1997) or the cardinal points, or the unity of a vertical axis mundi or celestial pole with the horizontal world (Koch, 1955). Speculation of this kind became especially popular in the mid- to late-19th century in the context of comparative mythology seeking to tie Christian mythology to ancient cosmological myths. Influential works in this vein included G. de Mortillet (1866), L. Müller (1865), W. W. Blake (1888), Ansault (1891), etc.

In the European Bronze Age the cross symbol appeared to carry a religious meaning, perhaps as a symbol of consecration, especially pertaining to burial.

The cross sign occurs trivially in tally marks, and develops into a number symbol independently in the Roman numerals (X "ten"), the Chinese rod numerals (十 "ten") and the Brahmi numerals ("four", whence the numeral 4).

In the Phoenician alphabet and derived scripts, the cross symbol represented the phoneme /t/, i.e. the letter taw, which is the historical predecessor of Latin T. The letter name taw means "mark", presumably continuing the Egyptian hieroglyph "two crossed sticks" (Gardiner Z9).

According to W. E. Vine's Expository Dictionary of New Testament Words, worshippers of Tammuz in Chaldea and thereabouts used the cross as symbol of that god.

Christian cross

The shape of the cross (crux, stauros "stake, gibbet"), as represented by the letter T, came to be used as a "seal" or symbol of Early Christianity by the 2nd century. Clement of Alexandria in the early 3rd century calls it τὸ κυριακὸν σημεῖον ("the Lord's sign") he repeats the idea, current as early as the Epistle of Barnabas, that the number 318 (in Greek numerals, ΤΙΗ) in Genesis 14:14 was a foreshadowing (a "type") of the cross (the letter Tau) and of Jesus (the letters Iota Eta). Clement's contemporary Tertullian rejects the accusation that Christians are crucis religiosi (i.e. "adorers of the gibbet"), and returns the accusation by likening the worship of pagan idols to the worship of poles or stakes. In his book De Corona, written in 204, Tertullian tells how it was already a tradition for Christians to trace repeatedly on their foreheads the sign of the cross.

While early Christians used the T-shape to represent the cross in writing and gesture, the use of the Greek cross and Latin cross, i.e. crosses with intersecting beams, appears in Christian art towards the end of Late Antiquity. An early example of the cruciform halo, used to identify Christ in paintings, is found in the Miracles of the Loaves and Fishes mosaic of Sant'Apollinare Nuovo, Ravenna (6th century). The Patriarchal cross, a Latin cross with an additional horizontal bar, first appears in the 10th century. A wide variation of cross symbols is introduced for the purposes of heraldry beginning in the age of the Crusades.

The first chapter of John Stott's The Cross of Christ provides a history of the development of the use of the symbol of the cross in Christianity. I'll briefly summarize it here.

Pre-cross images

The earliest images used by Christians did not include the cross. Persecution required that they be circumspect, which meant that an image clearly associated with both Christ and execution could not be preferred. Thus, the earliest images included peac**ks, doves, and particularly fish. The greek word for fish, ichthys, served as an ancronym for Iesus Christos Theos Huios Soter ("Jesus Christ, Son of God, Saviour"), which drew less attention from the uninitiated.

Second-century images included biblical scenes like Noah's ark, Abraham killing the ram, and the raising of Lazarus. These had a closer relationship to the redemption of Christ, but were still not obvious enough for the public at large to understand the reference.

Choosing the cross

Christians had a variety of emblems to choose from, such as the manger where Jesus was born, or the boat from which he taught, or the stone that rolled away from his grave. But, says Stott, Christians wanted to:

commemorate as central to their understanding of Jesus neither his birth nor his youth, neither his teaching nor his service, neither his resurrection nor his reign, nor his gift of the Spirit, but his death, his crucifixion.

The cross image at this time was not a crucifix, that is, a cross with Jesus' body on it; such images apparently arrive in the sixth century. (for more on this, see When did the split in crufix/empty-cross symbolism occur?)

Early use of the cross

Stott argues that the cross was in use by the 2nd century. He interprets Tertullian's AD 200 writing, "we trace upon the forehead the sign," as being the sign of the cross (De Corona, chapter 3). He also presents the writing of Hippolytus (The Apostolic Tradition, AD 215):

He mentions that the sign of the cross was used by the bishop when anointing the candidate's forehead at Confirmation [...] [i]t is also, he adds, a protection against evil.

Hippolytus, says Stott, was a conservative who would not have been describing rites and customs unless they were "already long-established," perhaps a generation or more.

Additional evidence from the early church comes from Cyprian, in the middle of the third century, who also appears to make reference to the symbol of the cross in his work De Lapsis.

Constantine

Constantine famously propelled the use of the cross as a symbol when he reportedly saw a vision at the Battle of the Milvian Bridge. The vision showed a cross and the words, "Conquer by this sign." The subsequent victory inspired Constantine's conversion, and he began using the cross on the standards of his army. The cross thus became widely known across the Roman Empire as the symbol of Christianity.