What does Augustine mean by the "hymns and psalms" which were introduced into the liturgy?

His spiritual life

Regarding his saying he was still "untouched by the warmth of your Spirit," Augustine's spiritual course before baptism was largely intellectual, involving a philosophical quest plus a search for relief from his inability to control his sexual appetites.

In Milan, Augustine met the Christian bishop, Ambrose, who impressed him with his intellect and answered his objections to the Bible. Augustine also learned about saints who had conquered sexual temptation by surrendering themselves to God. This was the right combination: a faith that would overcome his sexual temptations and let him be a thinker. (Augustine Converts to Christianity)

It seems that Augustine's personal relationship with God, after his baptism, grew gradually. Also, in Book 9, prior to the passage quoted in the OP, Augustine follows his account of his baptism by describing three deaths, including that of his son, Adeodatus, about two years after Augustine's baptism. He also discusses at length the illness and death of his mother, Monica, who was also his spiritual parent. His feelings about these deaths may have been a block to his experience of God's warmth, especially as he mentions the people's "depression and exhaustion," in the next sentence. He may include himself in this description of the church's people.

The Eastern hymns

The hymns in question are those composed by Augustine's mentor, Ambrose of Milan and probably also by Hilary of Poiters. The latter had spent years in Phrygia during a period of exile where he learned the music of the Eastern Church. Among the hymns heard by Augustine would might have been the Te Deum known later as “the Ambrosian Hymn," although whether it was composed by Ambrose himself is unknown. In any case, the musical tradition developed in Milan by Ambrose would eventually be known as the Ambrosian rite. It appears (see below) that the tradition in Milan prior to this time was dryer, less melodic, and not as emotionally moving.

The purpose of music in the Church

Augustine's feelings about music were mixed. On one hand in Book 9 he says that new hymns were introduced "to prevent the people from succumbing to depression and exhaustion," possibly implying that he was among these people. On the other hand, in Book 10 he changes his tune, so to speak, and cautions against music's seductive power:

I vacillate between dangerous pleasure and tried soundness; being inclined rather... to approve of the use of singing in the church, that so by the delights of the ear the weaker minds may be stimulated to a devotional frame. Yet when it happens to me to be more moved by the singing than by what is sung, I confess myself to have sinned criminally, and then I would rather not have heard the singing. (10:33.50)

Earlier in this Book 10, Augustine wonders if perhaps the tradition of St. Athanasius might be preferable, in which the reader is urged to recite the psalms with no inflection or melody. We can deduce that prior to Ambrose's introduction of the musical tradition learned from the East, the tradition in Milan was less melodic, and possibly not melodic at all.

Summary: the passage indicates Augustine's recollection of his post-baptismal spiritual challenges and the role that music played in his movement from a more intellectual faith to a more fully spiritual relationship with God. The Eastern tradition he speaks of refers to the hymns learned in the East by Hillary of Poitiers and the tradition introduced in Milan by St. Ambrose. Although psalms were surely recited previously in Milan, they were not as melodic and emotionally moving as they became in the Ambrosian tradition.

Here is a rendition of the Ambrosian Te Deum.