What is the biblical support for the original sin being an infinite sin?

The term infinite sin in this case is actually a misnomer. To be an infinite sin it would have to be a sin which could not be reversed.

The only sin which cannot be reversed is a sin which was committed and not repented of.

In the case of the original sin, the sin itself was repented of; however, the effects of that sin cannot be reversed.

Once man had the knowledge of good and evil, or if you will the knowledge of right and wrong, that knowledge would always be with man throughout his existence.

It is because the effects of that sin were irreversible that this sin began to be called infinite.

Jesus showed us on the cross that sin could be repented of up until the time of death.

Luke 23:43 KJV And Jesus said unto him, Verily I say unto thee, Today shalt thou be with me in paradise.

This question is about understanding humanity. This kind of infinity could be understood like this:

Adam's sin is the sin of the human nature. As a creation of God, the first ancestor of the human specie could be seen as a cumulation of many other infinite attributes which human nature has. He has speaking nature. All human descendants has speaking nature. Animal nature doesn't (at same level). He is able to learn and record his knowledge for future generations. His descendants the same. Animal nature doesn't has this at the same level. He is able to contemplate the nature and be amazed by the creation. Descendants same. He has consciousness. And so on.

When Adam sinned, he added a new attribute to the human nature. His sin wasn't only a simple act, it was a fully transformation with deep roots in his nature. One of the consequence was the losing the ability to communicate directly with God, he started to speak with himself (like Lucifer). And God decided to create Eve, to save his ability to communicate.

Holy Fathers of the church saw this transformation, by studying human nature thoroughly in the light of reveled wisdom. It is described largely in books of Philokalia.

When somebody is intrigued about the Christian teaching that we all sinned through Adam, in the same way should be intrigued by the fact that the good nature (virtues and abilities) also passes through generations until us, also the human knowledge passes and increases from one generation to the other. This is how human nature works. That's why we are so important in "God's eyes", as a specie comparing to animal nature. Holy Fathers described a human as "the big world", living in the universe which is "the small world".

There is a sense in which some sins can be measured and could be called finite: the size of stealing from a faceless corporation is the money you stole; the size of assault is the damage done - "an eye for an eye".

But sins are never done in isolation like that. All sins are expressions of our ultimate sin: our rebellion against God. Calling sin infinite means that we cannot measure the offensiveness, the grievousness, the injustice of our rebellion against our loving creator and rightful ruler.

I don't know if "infinite sin debt" has a more precise definition in some traditions, but as a concept, I'd say that the "infinite debt" alludes to the idea that the debt we owe because of sin is too large ever to be repaid.

Hebrews 10:4 (NASB)
For it is impossible for the blood of bulls and goats to take away sins.

The Hebrews writer says more about the idea in the entire chapter, but this is the thesis as far the question is concerned. Everyone has sinned (Ro 3:23). There is nothing we can do that will undo or make up for our sins (e.g. animal sacrifices, personal righteousness–Eph 2). In this sense, our sin (and the sin of Adam) could be thought of as infinite.

I don't think the saying you quoted is as much a precise statement of a specialized doctrine as much as it is a memorable, pithy statement in praise of Jesus.