Economic status of whites in the US South at the height of slavery?

Most Whites during the Slavery era lived in abject poverty and worked hard just to barely feed their families.

Slavery only benefited the rich and the plantation owners.

This summary of the 1860 Census results shows that the percentage of slave-owning households in the South ranged from 12% in Maryland and 13% in Missouri to highs of just 46% in South Carolina and 49% in Mississippi.

Looking more closely at Mississippi, the most extreme example, we see that the ratio of total slave population to total slave-owning families is 436,631:30,943, or about 15:1. Also slightly more than half of all families (32,072 to 30,943) owned no slaves at all.

A summary of research by the Gilder Lehrman Institute of American History reveals additionally that, over the South as a whole:

which leaves 100 - (31 + 53) = 100 - 86 = 14% of slaves being owned by families with only 1 to 6 slaves.

So we can rebut Calhoun's claim categorically:

• Even in the state with the highest rate of slave ownership, more than half of all households owned no slaves.

• In the South as a whole, ~1/7 of all slaves were owned by families with a small number of slaves.

Tony Horwitz is paraphrasing the speech that Calhoun gave in the Senate in the debate on 6 February 1837 on the Abolition of Slavery.

Calhoun's speeches have also been collected and published, with this particular speech in Volume 2 (a slightly easier to read version of the text is also available on Wikisource).

What Calhoun actually said was:

"I hold then, that there never has yet existed a wealthy and civilized society in which one portion of the community did not, in point of fact, live on the labor of the other"

and

"There is and always has been in an advanced stage of wealth and civilization, a conflict between labor and capital. The condition of society in the South exempts us from the disorders and dangers resulting from this conflict; and which explains why it is that the political condition of the slaveholding States has been so much more stable and quiet than that of the North".

The rich whites (who provided the capital) were certainly freed from drudgery, but the poor whites of the South (who had nothing to offer but their labour) were clearly not. Even on the plantations, the slave overseers (who were white) had to work, and so were not able to just do what they wanted.

So, to answer your question, no, there weren't any "small communities in which slaves did all the unpleasant tasks and Whites were able to do what they wanted".

It is perhaps worth noting that Calhoun also said in that speech:

"I may say with truth, that in few countries so much is left to the share of the laborer, and so little exacted from him, or where there is more kind attention paid to him in sickness or infirmities of age."

Compare that claim with Calhoun's treatment of his own slaves.

One event mentioned in his correspondence (and noted on his Wikipedia) page illustrates the point.

In 1831, a house-slave named Aleck belonging to Calhoun ran away when threatened with a severe whipping. Calhoun wrote to his brother-in-law James Edward Calhoun (Letter to James Edward Calhoun, 27 August 1831, Correspondence of John C. Calhoun, p300), asking him to keep a lookout for Aleck. Calhoun wrote that if he was taken, he should be "severely whipped and sent home immediately".

Aleck was caught, and Calhound wrote the following to his captor:

"I am glad to hear that Alick has been apprehended and am much obliged to you for paying the expense of apprehending him . . . . He ran away for no other cause, but to avoid a correction for some misconduct, and as I am desirous to prevent a repetition, I wish you to have him lodged in Jail for one week, to be fed on bread and water and to employ some one for me to give him 30 lashes well laid on, at the end of the time. I hope you will pardon the trouble. I only give it, because I deem it necessary to our proper security to prevent the formation of the habit of running away, and I think it better to punish him before his return home than afterwards".

• Letter to Armistead Burt, 1 September 1831. Correspondence of John C. Calhoun, p301