What were the scriptural arguments used for modern western racism?

I'm no scholar, but based on my research of the Bible, the Epistle to Philemon has been used as a justification for slavery in the West.

Perhaps this is why he [Ones'imus] was parted from you for a while, that you might have him back for ever, no longer as a slave but more than a slave, as a beloved brother, especially to me but how much more to you, both in the flesh and in the Lord.
Philemon 15-16 (RSV-2CE)

Ones'imus was a slave to Philemon, and while Paul urged Philemon to accept Ones'imus as a beloved brother, Paul did not question the institution of slavery and acknowledged implicitly that Philemon had legal authority over Ones'imus.
See the Wikipedia link.

But the most explicit reference to racism in the Bible that I have heard is the mark of Cain. The mark of Cain has its origins in American Protestantism.

Then the Lord said to him, "Not so! If any one slays Cain, vengeance shall be taken on him sevenfold." And the Lord put a mark on Cain, lest any who came upon him should kill him.
Genesis 4:15 (RSV-2CE)

The mark of Cain was believed to be dark skin, and thus blacks were considered the descendants of Cain, and thus inferior.

One concept I heard someone mention, primarily in terms of interracial marriages, referred to the Old Testament commands to the nation of Israel after the exodus. They were told not to intermarry with the peoples whose lands they would pass through and/or conquer. A thorough study of the texts would indicate that it was not their race that mattered, but rather their beliefs, since the "stranger in the land" who chose to follow the God of Abraham, Isaac, and Jacob were welcomed (Ruth being one of the more famous examples, Rahab another). The idea that this was a specific command to a specific people at a specific time is lost on some, resulting in the commands being mutated and put into use even today.

Great question! Here are reasons starting from Southern Christian slave-owners to modern times.

First, the long-used King James bible never condemns "slavery". Rather, it provides guidelines for this "slavery" and encourages "slaves" to embrace their "bondage".

Let all who are under the yoke of slavery regard their masters as worthy of all honor, so that the name of God and the teaching may not be blasphemed. Those who have believing masters must not be disrespectful to them on the ground that they are members of the church; rather they must serve them all the more, since those who benefit by their service are believers and beloved. Teach and urge these duties. (1Tim. 6:1-5)

Slaves, obey your earthly masters with fear and trembling, in singleness of heart, as you obey Christ; not only while being watched, and in order to please them, but as slaves of Christ, doing the will of God from the heart. (Eph. 6:5-6)

Tell slaves to be submissive to their masters and to give satisfaction in every respect; they are not to talk back, not to pilfer, but to show complete and perfect fidelity, so that in everything they may be an ornament to the doctrine of God our Savior. (Titus 2:9-10)

Slaves, accept the authority of your masters with all deference, not only those who are kind and gentle but also those who are harsh. For it is a credit to you if, being aware of God, you endure pain while suffering unjustly. If you endure when you are beaten for doing wrong, what credit is that? But if you endure when you do right and suffer for it, you have God's approval. (1Pet. 2:18-29)

If you just read the surface level English, it seems as if God is okay with slavery. The issue become much trickier when you read the Bible in it's original language.

"In the ancient world, service and slavery were closely related, so much so that one can scarcely distinguish the one from the other. The original words used for “servants” and “service” carry a variety of meanings between which it is not always easy to determine what is meant (All the Trades and Occupations of the Bible 1969, p. 197).

Apologeticspress.org does a fine job of elaborating more on the subject:

Arndt and Gingrich documented that the Greek word doulos meant “slave,” but that it also was used “in a wider sense” to denote “any kind of dependence.” In 2 Corinthians 4:5, the apostles are called the douloi (plural of doulos) of the Christians. Christ took on the form of a doulos, as stated in Philippians 2:7. Paul designates himself as a doulos of Christ in Romans 1:1, Philippians 1:1, Galatians 1:10, and numerous other passages (1967, pp. 205-206). The term can describe a person who is obligated in some way, whether voluntarily or involuntarily, to another person. Due to this broad use, various translations have employed a wide range of words to render the meaning of doulos in English. Using Romans 1:1 as a case in point, the NKJV has “bondservant,” the New Living Translation has “slave,” the KJV and ASV have “servant,” and the Darby Bible has “bondman.”

The Hebrew word ebed is similar to the Greek doulos, in that it can be translated as “slave” or “servant.” In Exodus 4:10, Moses referred to himself as the “servant” (ebed) of God. Abraham called himself the ebed of the angels who came to visit him in Genesis 18:3. In Genesis 39:17-19, Potiphar’s wife described Joseph as the Hebrew ebed, and Genesis 24:2 talks about the eldest ebed in Abraham’s house, who “ruled over all he had.”

Because slavery was profitable, it makes sense that some Christians would be pre-disposed to slavery- but perhaps some did search the scriptures. The King James version never explicitly condemns slavery.

Some slave-owners said that the Bible supported slavery. Kenneth Stamp, in The Peculiar Institution writes:

"...when southern clergy became ardent defenders of slavery, the master class could look upon organized religion as an ally ...the gospel, instead of becoming a mean of creating trouble and strive, was really the best instrument to preserve peace and good conduct among the negroes."

Twisting the Bible has been very profitable to many different groups historically, and it was no different with slaveholders.

Slaves were taught the scriptures in a way so that they had a sufficient fear of Him and of course, knowledgeable about the "slave" verses. Thus, some of them saw their slavery as their cross they must bear in order to receive God's favor.

Additionally, there's the whole Cain/cursed descendent of Noah thing.

"Cursed be Canaan; lowest of slaves shall he be to his brothers" (Gen 9:25)

I had presumed that this was only a thing of history, but this article

says that the whole Cain thing is exactly what's happening in some churches still today. (I emailed the pastor about whether or not this article is true. I will update this article if I get a reply)

Additionally, it was said that different races should not mix. For a very long explanation of the arguments against interracial marriage, visit this site. Here is a small excerpt from the article:

Interracial marriage is discouraged throughout the Bible. In the book of Ezra, the Israelites repented of their abandonment of God. As part of this repentance they pledged to end their interracial marriages according to God's will:

“We have been unfaithful to our God by marrying foreign women from the peoples around us. But in spite of this, there is still hope for Israel. Now let us make a covenant before our God to send away all these women and their children, in accordance with the counsel of my lord and of those who fear the commands of our God. Let it be done according to the Law. Rise up; this matter is in your hands. We will support you, so take courage and do it.” (Ezra 10)

A similar experience is recorded in Nehemiah 13: “Moreover, in those days I saw men of Judah who had married women from Ashdod, Ammon and Moab. Half of their children spoke the language of Ashdod or the language of one of the other peoples, and did not know how to speak the language of Judah. I rebuked them and called curses down on them. I beat some of the men and pulled out their hair. I made them take an oath in God’s name and said: ‘You are not to give your daughters in marriage to their sons, nor are you to take their daughters in marriage for your sons or for yourselves.’” Other prohibitions against interracial marriage can be found in Exodus 34:12-16, Joshua 23:12, and Deuteronomy 7:3: “Do not intermarry with them. Do not give your daughters to their sons or take their daughters for your sons.”

Some have claimed the Bible's ban on interracial marriage was based on religion rather than ethnicity, and that the sole purpose of the prohibition was to keep the people pure of idolatrous influence.

The article is saying that God has sanctioned the pureness of races and therefore they should not mix with a white-supremist attitude.

This subject is further explained here:

The theology of Donny Reagan, pastor of Tennessee’s Happy Valley Church of Jesus Christ, should inspire Anderson to think again. “If corn was raised in a certain way, yellow corn, don’t mix it with white corn. If you do your mixing, you can’t bring yourself back again,” Reagan warned in a sermon recorded last year at his 600-member church in Johnson City, Tenn. Claiming that black athletic stars choose white wives in a willful attempt to make their offspring lighter, Reagan declared, “It’s another defiance of God’s law, it’s a worldly way.”

Summary:

1. The widely and long used KJV does not specifically condemn slavery and has passages that provide guidelines that were twisted to support slavery.

2. The long surviving Curse of Cain/Cursed descendent of Noah belief. (That blacks are cursed by God)

3. The belief that interracial marriage is bad. (with a flavor of white supremacy)