How did the monastic tradition evolve?

There are basically three forms of monastcism and they are still extant.

Anachoretism - the first form to emerge was living in complete solitude. Hermits however sometimes accepted disciples that would watch their life and then leave. There are still hermits living e.g. in rocky parts of Mt. Athos that don't see other monks for long periods of time.

Idiorythmia - when monasticism became popular, some areas fit for monastic life started attracting monks and monastic societes as e.g. Nitra emerged. Monks would meet at Liturgy and would counsel some things together, yet every brother would be econimically independent.

Cenobitism - as Bob already said - this form was instituted by St Pachomius. Monks live in a commune. They work together and their material needs are fullfilled by the community. This form is the safest and easiest, because monks can comfort each other and the strict obedience protects them from straying on their spiritual path. This is why it is most popular form today. But the two former are not extinct.

Orthodox monastery in New Mexico

I think the other answers addressed the question well. Kallistos Ware's section on "Saints, Monks, and Emperors" in The Orthodox Church fills in some additional details.

[Not that it is particularly relevant to the question, but I have been blessed to know a number of monastics and have stayed myself for a time at an Orthodox monastery in New Mexico. There is also a Russian Orthodox hermit and a women's convent close to where I live in Texas - I have visited both. A few years ago I was able to visit a monastery (Alexander Nevsky Lavra) and convent (St. John of Rila) in St. Petersburg, Russia. I also was educated at a Roman Catholic monastery/high school and an elementary school run by a Roman Catholic convent, so I have personally been exposed to both Eastern and Western monasticism, male and female.]

The monastic life first emerged as a definite institution in Egypt and Syria during the fourth century, and from there it spread rapidly across Christendom. It is no coincidence that monasticism should have developed immediately after Constantine's conversion, at the very time when the persecutions ceased and Christianity became fashionable. The monks with their austerities were martyrs in an age when martyrdom of blood no longer existed; they formed the counterbalance to an established Christendom. People in Byzantine society were in danger of forgetting that Byzantium was an image and symbol, not the reality; they ran the risk of identifying the kingdom of God with an earthly kingdom. The monks by their withdrawal from society into the desert fulfilled a prophetic and eschatological ministry in the life of the Church. They reminded Christians that the kingdom of God is not of this world.

Monasticism has taken three chief forms, all of which had appeared in Egypt by the year 350, and all of which are still to be found in the Orthodox Church today. There are first the hermits, ascetics leading the solitary life in huts or caves, and even in tombs, among the branches of trees, or on the tops of pillars. The great model of the eremitic life is the father of monasticism himself, St Antony of Egypt (251 – 356). Secondly there is the community life, where monks dwell together under a common rule and in a regularly constituted monastery. Here the great pioneer was St Pachomius of Egypt (286 – 346), author of a rule later used by St Benedict in the west. Basil the Great, whose ascetic writings have exercised a formative influence on eastern monasticism, was a strong advocate of the community life, although he was probably influenced more by Syria than by the Pachomian houses that he visited. Giving a social emphasis to monasticism, he urged that religious houses should care for the sick and poor, maintaining hospitals and orphanages, and working directly for the benefit of society at large. But in general eastern monasticism has been far less concerned than western with active work; in Orthodoxy a monk's primary task is the life of prayer, and it is through this that he serves others. It is not so much what a monk does that matters, as what he is. Finally there is a form of the monastic life intermediate between the first two, the semi-eremitic life, a ‘middle way’ where instead of a single highly organized community there is a loosely knit group of small settlements, each settlement containing perhaps between two and six members living together under the guidance of an elder. The great centres of the semi-eremitic life in Egypt were Nitria and Scetis, which by the end of the fourth century had produced many outstanding monks – Ammon the founder of Nitria, Macarius of Egypt and Macarius of Alexandria, Evagrius of Pontus, and Arsenius the Great. (This semi-eremitic system is found not only in the east but in the far west, in Celtic Christianity.) From its very beginnings the monastic life was seen, in both east and west, as a vocation for women as well as men, and throughout the Byzantine world there were numerous communities of nuns.

St. Katherine's monastery in the Sinai Desert

Because of its monasteries, fourth-century Egypt was regarded as a second Holy Land, and travellers to Jerusalem felt their pilgrimage to be incomplete unless it included the ascetic houses of the Nile. In the fifth and sixth centuries leadership in the monastic movement shifted to Palestine, with St Euthymius the Great (died 473) and his disciple St Sabas (died 532). The monastery founded by St Sabas in the Jordan valley can claim an unbroken history to the present day; it was to this community that John of Damascus belonged. Almost as old is another important house with an unbroken history to the present, the monastery of St Catherine at Mount Sinai, founded by the Emperor Justinian (reigned 527 – 65). With Palestine and Sinai in Arab hands, monastic pre-eminence in the Byzantine Empire passed in the ninth century to the monastery of Stoudios in Constantinople. St Theodore, who became Abbot here in 799, reactivated the community and revised its rule, attracting vast numbers of monks.

The Orthodox Church, pp. 36-37

What follows pertains principally to monasticism in the Orthodox Church:

Since the tenth century the chief centre of Orthodox monasticism has been Athos, a rocky peninsula in North Greece jutting out into the Aegean and culminating at its tip in a peak 6,670 feet high. Known as ‘the Holy Mountain’, Athos contains twenty ‘ruling' monasteries and a large number of smaller houses, as well as hermits' cells; the whole peninsula is given up entirely to monastic settlements, and in the days of its greatest expansion it is said to have contained nearly forty thousand monks. The Great Lavra, the oldest of the twenty ruling monasteries, has by itself produced 26 Patriarchs and more than 144 bishops: this gives some idea of the importance of Athos in Orthodox history.

There are no ‘Orders’ in Orthodox monasticism. In the west a monk belongs to the Carthusian, the Cistercian, or some other Order; in the east he is simply a member of the one great fellowship which includes all monks and nuns, although of course he is attached to a particular monastic house. Western writers sometimes refer to Orthodox monks as ‘Basilian monks’ or ‘monks of the Basilian Order’, but this is not correct. St Basil is an important figure in Orthodox monasticism, but he founded no Order, and although two of his works are known as the Longer Rules and the Shorter Rules, these are in no sense comparable to the Rule of St Benedict.

A characteristic figure in Orthodox monasticism is the ‘elder’ or ‘old man’ (Greek gerōn; Russian starets, plural startsy). The elder is a monk of spiritual discernment and wisdom, whom others – either monks or people in the world – adopt as their guide and spiritual director. He is sometimes a priest, but often a lay monk; he receives no special ordination or appointment to the work of eldership, but is guided to it by the direct inspiration of the Spirit. A woman as well as a man may be called to this ministry, for Orthodoxy has its ‘spiritual mothers' as well as its ‘spiritual fathers’. The elder sees in a concrete and practical way what the will of God is in relation to each person who comes to consult him: this is the elder's special gift or charisma. The earliest and most celebrated of the monastic startsy was St Antony himself. The first part of his life, from eighteen to fifty-five, he spent in withdrawal and solitude; then, though still living in the desert, he abandoned this life of strict enclosure, and began to receive visitors. A group of disciples gathered round him, and besides these disciples there was a far larger circle of people who came, often from a long distance, to ask his advice; so great was the stream of visitors that, as Antony's biographer Athanasius put it, he became a physician to all Egypt. Antony has had many successors, and in most of them the same outward pattern of events is found – a withdrawal in order to return. A monk must first withdraw, and in silence must learn the truth about himself and God. Then, after this long and rigorous preparation in solitude, having gained the gifts of discernment which are required of an elder, he can open the door of his cell and admit the world from which formerly he fled.

The Orthodox Church, pp. 39-40

A really excellent movie to watch to get a feel for Russian monasticism is The Island (Ostrov). It used to be available on Netflix, but I think today it is only available on DVD. As an aside, the actor who plays the main character - a monk - was previously a famous rock star in Russia. After he finished the movie, he took his family and went to live in seclusion in Siberia or somewhere remote.

Another good book involving the monastic life is Everyday Saints, written by a Russian monk. Before being translated into English, it was the number one bestselling book in Russia. Several million copies have been sold (It's really hard to put down).

Based on my personal experience and knowledge, I would say that Orthodox monasticism has not changed much from what it has been since the 17th century or so. One exception was the form it took in Russia after the Bolshevik Revolution. It is estimated that over 200,000 monks and nuns were executed or died in Gulags, so many monastics went underground or became "secret monks" or "secret nuns". There are true stories of such people in the book, Father Arseny: Priest, Prisoner, Spiritual Father.

See Pachomius the Great (Wikipedia)

St. Pachomius the Great is one of the figures instrumental in moving monastic life from the life of hermits (old as dust) and small communities.

As for cenobitism, the widows in the New Testament were parochial nuns. If you consider the function of old widows in almost every church, you will find they generally fulfill the duties that nuns do, though in a local setting. A large church would have a fair number of these celibate (since post-reproductive age), prayerful and godly women. If you consider how Paul treats 'the widows' it is precisely as an order of nuns would be treated by the overseeing abbess or bishop.

The mens' tradition seems to have grown out of the former mens' tradition of hermit life, such as what St. John the Baptist lived. It is with Antony (my own patron) and Pachomius that we see people gathering around a single elder or abbot and forming a community. It is with Pachomius that the communites became sizable; Scetis (Antony's) still exists, but it has never been large. Pachomius' communities did not themselves survive (as many of the pilgrim cities did not) but his tradition and practices which allowed this organization did.

Monasticism was unknown until the end of the third century. Paul expressed that he preferred celibacy, but there was no "command from the Lord" to remain unmarried. (Simon) Peter, according to Catholic tradition the first Pope, was himself married.

Luke 4:38 And he arose out of the synagogue, and entered into Simon's house. And Simon's wife's mother was taken with a great fever; and they besought him for her.

MATTHEW 8:14 And when Jesus was come into Peter's house, he saw his wife's mother laid, and sick of a fever.

Widows were treated with special respect, but encouraged to remarry and bear children if still within child bearing age. More emphasis was placed on missionary and charity work than meditation or spiritual development.

Over time, the custom developed of going into the desert (or otherwise being alone), when faced with an important life decision, a desire to commune with God, etc. These people were called hermits, a term which means "desert dwellers". The most famous of these early hermits was Anthony of Egypt (251-356). Athanasius' Life of St. Anthony began to make the hermit life popular, both because it portrayed it as a path to eternal life, and because Anthony was portrayed as an athlete for Christ, a heroic figure.

Now there were so many hermits that it was starting to get crowded. A hermit named Pachomius (290-346) instituted some rules; that the monks should live in isolated huts, produce their own food and clothing, and should never speak to one another. Because they were no longer dependent on public charity, their growth was unlimited, and monasteries and monks began to spread rapidly.