What makes a confession sacramental?

As far as I know, what makes a confession sacramental, so that the seal of the confessional applies to it, is that the penitent asks the priest for absolution. Even if the priest refuses absolution, he is still prohibited from revealing what he heard in that confession. Purple stoles, lines, and green and red lights are often involved, but I don't think the lack of any of them would make the confession not sacramental.

EDIT: A more authoritative answer, from Section 503 of "Moral Theology" by Fr. Heribert Jone (a book that was recommended to me by a traditional Catholic piest, and which I therefore consider reliable): "Sacramental confession consists in the telling of our sins to a duly authorized priest with the intention of being absolved by him."

Fr. Jone adds, in smaller print: "That confession also is sacramental which remains incomplete or which is sacrilegious. No sacramental confession is made when one knowingly tells one's sins to a priest who, he knows, has no jurisdiction. Likewise, if one confesses to a priest who has jurisdiction, but does so merely for mockery or to seek counsel or consolation."

Note that, in my original answer, "asks the priest for absolution" was apparently too restrictive; it suffices that the penitent has the intention of being absolved.

As Gregory says [Isidore, Etym. vi, ch. 19], "a sacrament consists in a solemn act, whereby something is so done that we understand it to signify the holiness which it confers." (source).

In order for a sacrament to be valid, all the following are necessary:

• proper matter (e.g., unleavened bread for the Eucharist, water for baptism, etc.)
• proper form (e.g., "This is my body/blood, etc." for the Eucharist, "I baptize thee in the name of the Father, Son, and Holy Ghost. Amen." for baptism, etc.)
• proper intention to do what the Church does in conferring the sacraments.

The proper matter for confession are "the acts of the penitent, the matter of which acts are the sins over which he grieves, which he confesses, and for which he satisfies." (source).

The proper form for confession is, at the bare minimum, "I absolve thee." (source).

The confession is automatically invalid if it lacks either or both of these.

You know you're in confession if:

• you started your conversation saying bless me father for I have sinned.
• the priest dons his purple stole
• you can't see the priest you're talking to
• you stood in line to talk to a priest
• you were given absolution for your sins
• you end your conversation by making an act of contrition.
• you don't go in to details, but merely count the number of times you screwed up.
• when you knelt down a light changed from green to red.

The most relaxed (and personally awful) confession I ever did was in the middle of a conversation. The priest merely whipped out his stole, told me that if I wanted more grace I should kneel and I told him all the awful things I was probably going to tell him anyway.

Let me know if you want a more canon law answer, rather than an anecdotal one. I don't think there's any mistaking when you're in confession, the most telling part is the absolution, but if you don't get absolution for whatever reason, I doubt you didn't know you went to confession.

The normal for of confession, according to the handbook of prayers (which is not a liturgical text, but a guide for the laity) is that confession begins with "bless me father... And it's been x number of days since my last confession" then you rattle off your calumnities and the priest gives you a little advice and asks if you're really sorry (you say yes, I don't want to know what happens if you say no, always say yes). Then the priest (acting in persona Christi) absolves you of your sins and you say an Act of Contrition (O my God I am heartily sorry, etc...) and the priest gives you some penance (usually a number of prayers to be prayed in earnest nowadays, although in the past, the punishement fit the crime) the most penance I've ever had was flowers and a date night. Then you leave feeling refreshed in body and soul. (Unless you did something dumb or self righteous in the confessional, then you feel worse than you can possibly imagine. But it's not so bad, you remind yourself, same absolution same succor of Heaven.

In the sacrament of penance the faithful who confess their sins to a legitimate minister, are sorry for them, and intend to reform themselves obtain from God through the absolution imparted by the same minister forgiveness for the sins they have committed after baptism and, at the same, time are reconciled with the Church which they have wounded by sinning.

(Code of Canon Law, canon 959)

Like all the sacraments, Penance is a liturgical action. The elements of the celebration are ordinarily these: a greeting and blessing from the priest, reading the word of God to illuminate the conscience [note: I don't ever remember this being a part of any of my confessions] and elicit contrition, and an exhortation to repentance; the confession, which acknowledges sins and makes them known to the priest; the imposition and acceptance of a penance; the priest’s absolution; a prayer of thanksgiving and praise and dismissal with the blessing of the priest.

(Catechism of the Catholic Church, paragraph 1480)

It is apparent from these two quotes that the first important aspect of confession (that is, of talking to a priest about one's sins) as Confession (that is, as the Sacrament of Confession) is the penitence of the penitent, the one who is admitting sin. Without sincere and deep sorrow for one's sins, and a desire to perform penance for them, an admission of sin is no more than an admission. The priest may or may not understand from such an admission (perhaps considering the individual's general demeanor as well) that one desires Confession; that's a pastoral judgment call. There's no specific "language of confessing a sin" that will of itself signal to the priest that Confession is being requested, short of a straightforward request like "Could you hear my Confession?" (Of course, the priest could initiate that on his part: "Would you like me to hear your Confession?")

Place is not an issue: for example

And the Pope himself:

Do note a few things, though:

• Although I just said "place is not an issue", you probably want to choose a place where you're not likely to be overheard.
• Priests, and canon law, prefer hearing confessions in a dedicated confessional if at all possible, as a reminder that the penitent is not "in this alone", but is part of the Church:

  The proper place to hear sacramental confessions is a church or oratory. ... Confessions are not to be heard outside a confessional without a just cause¹. (Canon 964; sections 1, 3)

Thus the quick answer is:

• There's no particular use of "confessory" language that in itself signals that a Confession is taking place, other than the blessing mentioned above in the Catechism.
• Thus, the individual and priest must specifically agree in so many words that a Confession is taking place; after which, the usual liturgical language of the Sacrament will be indicated.
• An unusual place does not in itself invalidate a confession; but the confessional is preferred.

¹ Just cause is a technical term meaning that the priest can see that justice would be violated if he did not take a given action. In this particular case, it means that the priest needs to see that the penitent urgently desires confession and cannot reasonably wait (perhaps they need to confess a mortal sin), and that a confessional is not reasonably available.

Cf. These YouTube videos:

• Sacrament Of Reconciliation: Explained
• How do I go to Confession?