At what point is a person considered to be married to someone else?

Since Protestants don't consider this book part of their Bible, this is from the Catholic perspective:

Tobias 7:14-17; 8:3-5 (as preserved in the Vulgate—the Bible of the Church for over 1500 years)

...I [the father] will give her to thee. And taking the right hand of his daughter, he gave it into the right hand of Tobias, saying: The God of Abraham, and the God of Isaac, and the God of Jacob be with you, and may he join you together, and fulfil his blessing in you. And taking paper they made a writing of the marriage. And afterwards they made merry, blessing God.

... Then Tobias exhorted the virgin, and said to her: Sara, arise, and let us pray to God to day, and to morrow, and the next day: because for these three nights we are joined to God: and when the third night is over, we will be in our own wedlock. 5 For we are the children of saints, and we must not be joined together like heathens that know not God.

1) Betrothal—"I [the father] will give her to thee".

2) Ask that God join the couple—"may he join you together, and fulfil his blessing in you" (the blessing being children—Be fruitful and multiply).

3) A legal record of the marriage—"taking paper they made a writing of the marriage".

4) A little 'wait' (for people who wish to be saintly) before coming together and to tend to their marriage to God foremostly—"for these three nights we are joined to God: and when the third night is over, we will be in our own wedlock".

So although this isn't a prescription (there is none in Scripture), it does describe the process.

The following is my understanding of a consistent Reformed understanding of Christian marriage, though it likely applies to others outside of the Reformed camp.

A marriage is a public, legally binding civil covenant recognized by God that binds together a male and female couple into a family that is consummated by sexual relations. It was created by God to be a model of his ultimate relationship between Christ and the Church (Ephesians 5).

This means that there will be some leeway in the precise marriage traditions and ceremony from culture to culture, though there are clearly biblical constraints on marriage and universal moral requirements of marriages that transcend culture. The Bible recognizes marriages between non-Christians as real marriages as a kind of common grace institution. It also recognizes that there are immoral marriages and non-ideal marriages that are possible.

This is why contemporary gay marriage debates should not be argued on the grounds that marriage is a religious institution. Though Christians might have a religious basis for scrupulously avoiding theft, the public, civil recognition of property ownership is not, therefore, a merely religious institution.

For a parallel, consider ordinary contracts and covenants in the Bible. We see all kinds of traditions at play for how people would ratify them: cutting animals (Genesis 15), exchanging sandals (Ruth 4), grabbing the thigh (Genesis 24:9), and so on. Yet they are all doing essentially the same thing but with different symbolism, occasions, and tradition.

Similarly, marriages can be culturally nuanced and yet bindingly so. In a culture where it was normal to deal with a woman's father and then immediately consummate your marriage without ceremony, this was a legitimate and proper way to begin a marriage (Isaac, Jacob, etc.) whereas it wouldn't be in a context where marriages are not ratified this way. For example, for Christian marriages in America, the marriage is traditionally considered "ratified" when the officiating minister declares the couple husband and wife after the exchange of vows. If this couple privately met a week earlier and rationalized having premarital sex on the grounds that they were getting married anyway and biblical couples were "married in the eyes of God" this way, they would be fornicating because in the "eyes of God" they should have officiated their marriage in the way that public covenants are normally ratified.

We could imagine a culture where business contracts were officiated by exchanging sandals. If you helped yourself to your side of the deal without following this sandal ritual this would be stealing in the eyes of God, not because sandals are necessary to prevent stealing but because contractual commitments matter to God.

This understanding should reconcile all of the Biblical examples of marriage while also addressing contemporary traditions. Marriage covenants needed to be "official," however the society generally officiates such matters.

For the issue @davidlol raised in a comment:

John 4 18 in which Jesus told the Samaritan lady at Jacob's well that she had had five husbands but the man she now had was not her husband may be relevant here, although I don't know how. What did Jesus mean that he wasn't her husband?

This is now easily answered. She was living with a man for whom she had not followed the appropriate culturally recognized method for officiating a marriage.