Why do many old churches face east?

The main Christian dominations that have churches facing East are the Anglican Church, Roman Catholic and Orthodox. However this tradition was not totally universal in its' usage because of the fact that some churches were the result of pre-existing buildings being converted into churches, or because of some other local reason, such as the direction of streets and so on.

Five years before he became Benedict XVI, Joseph Ratzinger wrote that, notwithstanding various liturgical innovations, “one thing has remained clear for the whole of Christendom: praying towards the east is a tradition that goes back to the beginning.” As he wrote in The Spirit of the Liturgy:

The common turning toward the east was not a “celebration toward the wall;” it did not mean that the priest “had his back to the people.” . . . For just as the congregation in the synagogue looked together toward Jerusalem, so in the Christian liturgy the congregation looked together “toward the Lord.” . . . They did not close themselves into a circle; they did not gaze at one another; but as the pilgrim People of God they set off for the Oriens, for the Christ who comes to meet us. - Facing East

Let us not forget what the Gospels have to say about Christ's second coming:

For as the lightning cometh out of the east, and shineth even unto the west; so shall also the coming of the Son of man be. - Matthew 24: 27

Cathedral oriented to the East

The first Christians faced east when praying, for which various explanations have been offered. In a tradition well established by the time of Christ, Jews in the diaspora would pray facing Jerusalem, which in most of the Roman Empire would have been to the East. Another explanation is that Christ's Second Coming would be on the clouds coming from the East: "For as the lightning cometh out of the east, and shineth even unto the west; so shall also the coming of the Son of man be." (Matthew 24:27). Due to this eastward posture of prayer, Tertullian (c. 160 — c. 220) says that some non-Christians thought they worshipped the sun. Origen (c. 185 — 253) says: "The fact that [...] of all the quarters of the heavens, the east is the only direction we turn to when we pour out prayer, the reasons for this, I think, are not easily discovered by anyone." Later on, various Fathers of the Church advanced mystical reasons for the custom.

At first, the orientation of the building in which Christians met was unimportant, but after the legalization of the religion in the fourth century, customs developed in this regard. These differed in East and West.

The Apostolic Constitutions, a work of eastern Christianity written between 375 and 380 AD, gave it as a rule that churches should have the sanctuary (with apse and sacristies) at the east end, to enable Christians to pray eastward in church as in private or in small groups. In the middle of the sanctuary was the altar, behind which was the bishop's throne, flanked by the seats of the presbyters, while the laity were on the opposite side. However, even in the East there were churches (for example, in Tyre) that had the entrance at the east end and the sanctuary at the west end. During the readings all looked towards the readers, the bishop and presbyters looking westward, the people eastward. The Apostolic Constitutions, like the other documents that speak of the custom of praying towards the east, do not indicate on which side of the altar the bishop stood for "the sacrifice".

The importance attached to orientation of churches declined after the 15th century. In his instructions on the building and arrangement of churches, Charles Borromeo, archbishop of Milan from 1560 to 1584, expressed a preference for having the apse point exactly east, but accepted that, where that is impractical, a church could be built even on a north-south axis, preferably with the façade at the southern end. He stated that the altar can also be at the west end, where "in accordance with the rite of the Church it is customary for Mass to be celebrated at the main altar by a priest facing the people". - Orientation of churches

The Catholic Encyclopedia explains the Orientation of Churches as thus:

According to Tertullian (155-240) the Christians of his time were, by some who concerned themselves with their form of worship, believed to votaries of the sun. This supposition, he adds, doubtless arose from the Christian practice of turning to the east when praying (Apol., c. xvi). Speaking of churches the same writer tells us that the homes "of our dove", as he terms them, are always in "high and open places, facing the light" (Adv. Val., c. iii), and the Apostolic Constitutions (third to fifth century) prescribe that church edifices should be erected with their "heads" towards the East (Const. Apost., II, 7).

The practice of praying while turned towards the rising sun is older than Christianity, but the Christians in adopting it were influenced by reasons peculiar to themselves. The principal of these reasons, according to St. Gregory of Nyssa, was that the Orient contained man's original home, the earthly paradise. St. Thomas Aquinas, speaking for the Middle Ages, adds to this reason several others, as for example, that Our Lord lived His earthly life in the East, and that from the East He shall come to judge mankind (II-II, Q. lxxxiv, a. 3). Thus from the earliest period the custom of locating the apse and altar in the eastern extremity of the church was the rule. Yet the great Roman Basilicas of the Lateran, St. Peter's, St. Paul's (originally), St. Lorenzo's, as well as the Basilica of the Resurrection in Jerusalem and the basilicas of Tyre and Antioch, reversed this rule by placing the apse in the western extremity. The reasons for this mode of orientation can only be conjectured. Some writers explain it by the fact that in the fourth century the celebrant at Mass faced the people, and, therefore in a church with a western apse, looked towards the East when officiating at the altar. Others conjecture that the peculiar orientation of the basilicas mentioned, erected by Constantine the Great or under his influence, may have been a reminiscence of the former predilection of this emperor for sun-worship. In the Orient the eastern apse was the rule, and thence it made its way to the West through the reconstructed Basilica of St. Paul's, the Basilica of S. Pietro in Vincoli, and the celebrated basilica of Ravenna. From the eighth century the propriety of the eastern apse was universally admitted, though, of course strict adherence to this architectural canon, owing to the direction of city streets, was not always possible.

As a side note: This question reminds me of the hymn (Christmas carol), People look East by Eleanor Farjeon (1881-1965).