Why do the faithful have to say "And with your spirit " in the Catholic Mass?

Why do the faithful have to say “And with your spirit ” in the Catholic Mass in English?

First of all the “sign of peace” at Mass is optional, even on Sundays and holy days of obligation.

The sign of peace is optional. The General Instruction of the Roman Missal (154) states that the celebrant asks the congregation to exchange the sign of peace “when appropriate” (GIRM 154). When the GIRM describes a Mass with a deacon (181), it repeats this language by stating the deacon asks the congregation to exchange the sign of peace “if it is appropriate.”

It is therefore at the discretion of the celebrant whether it is appropriate at a particular Mass to exchange the sign of peace. - The Kiss of Peace Is Optional

A few years ago, the English version of the Mass had the wording for the congregation’s part as such: “And also with you.”

The New Translation has the words as such: ”And with your spirit.” This was put into motion because it more closely resembles the Latin Mass in the Ordinary Form of the Mass.

The changes (and their continuities with the older form of the Mass) are obvious from the first time the priest says, "The Lord be with you." In place of the familiar "And also with you," the congregation responds, "And with your spirit"—a literal translation of the Latin "Et cum spiritu tuo," found in both forms of the Mass. The Confiteor (the penitential rite), the Gloria ("Glory to God in the highest"), the Nicene Creed, and the dialogue between the priest and the congregation after the Agnus Dei ("The Lamb of God") and immediately before Communion all hark back to the older form of Mass—as well they should, because both forms of the Mass share the same Latin text for these parts.

Still, it would be a mistake to think that the new translation significantly alters the Novus Ordo. The changes put in place by Pope Paul VI in 1969 remain, as do all of the major differences between the Traditional Latin Mass and the Novus Ordo. All the new translation does is to tighten up some very loose translations of the Latin text, restore a certain dignity to the English text of the Mass, and reinstate a few lines at various points in the Mass that had simply been dropped in the earlier translation from Latin to English. - The New Mass: Continuities and Changes

More on this subject can be found in the following article(s):

Vatican announces desire for more restrained sign of peace at Mass

The rite is optional, the congregation reminded, and there certainly are times and places where it is not fitting.

The simple answer is that "thy spirit" is a Hebraism for "you" (to be specific in this case, "you," but more 'endearing,' 'elevated,' or 'prayerful;' cf. Lk. 1:46). St. Paul uses this blessing in 2 Timthy 4:22: "The Lord Jesus Christ be with thy spirit..."

This was almost certainly imported into, or was always part of, the Roman liturgy very early, and it has remained in it ever since.

Indeed, it appears that all the apostolic Liturgies involve this Hebraism, explainable only by the primitive apolostic foundation each of the Liturgies have.

For example, the Liturgy of St. John Chrysostom, the most popular Liturgy in the East, one hears the priest say, "Peace be with you all," and the deacon responding, "And with thy spirit." The Liturgy of St. Mark (Alexandrian Liturgy) also has this, as does St. James' (Antiochian) Liturgy.

As to why the laity (or the deacon on their behalf) are to respond to the blessing of the priest, the theological reason is that at Mass, there is to be kept a close unity of offerer and the others who offer by him, the Eucharist as well as the sacrifice of their praise and prayers.

For example, in many Liturgies we find this common prayer or ones very similar:

Liturgy of St. Thomas/Malankra/Indian Rite

Priest. Pray, brethren, that our sacrifice may be acceptable to God, the almighty Father.

Deacon. May the Lord accept the sacrifice at your hands for the praise and glory of his name for our good, and the good of his Church.

Roman Liturgy of the Extraordinary Form of the Mass

Priest. Pray, brethren, that my sacrifice and yours may be acceptable to God, the almighty Father.

Deacon. May the Lord accept the sacrifice at your hands for the praise and glory of his name, for our good and the good of all his holy Church.

The laity, via the Deacon (or servant), are to be always united and prayerfully attentive to the priest's intercessory prayer and offering of the Eucharist. That is why when blessed, they return the blessing, as it were: "The Lord be with you: and with thy spirit [also]!" eagerly awaiting what comes next. It involves the laity in the Sacrifice they are otherwise unable to offer themselves, via union with the priest who offers.