Why didn't Roman emperors have harems?

One key lies in their treatment of illegitimacy, or bastardy. In Roman society, as is typical of the West in general, illegitimate children had no formal link to their fathers. This was true from the earliest times, and lasted well into the Imperial period before a softening of the laws occurred in the second and third centuries.¹

In the Roman setting, illegitimacy was chiefly a disability in inheritance law. It was a major problem in cases of intestacy, where the father either left no will or his will failed for technical reasons ... in Roman law illegitimate children had no legal relationship to their fathers.²

Conversely, in the Eastern societies (at least, the named examples with royal harems), all sons were acknowledged and given a place in the succesion line.

• Chinese inheritance laws gave precedence to the sons of the legally married wife. However, if the official wife had no issue, succession defaults to primogeniture for all other sons. While similar to the West in that "illegitimate" children or children by concubines were scorned, they were ultimately considered fully legitimate backup heirs.³
• Ottoman succession was initially a bloody competition open to all adult sons. Inevitably, the extensive fratricides threatened the lineage with extinction, causing a shift to agnatic seniority. Nonetheless, in a reflection of Turko-Mongol traditions (as was the case with the other great Islamic Empire of the Safavids), all sons were equally legitimate participants.⁴
• Mongolian customs assigned value to offspring based on the status of their mothers. Sons of the first, official, wife were thus entitled to a larger share of the inheritance, but all children were acknowledged. They all had a claim to their father's legacy, even if unequally.⁵

Let us disregard the role of sexual jealousy and focus on the practical impact of such succession laws. Since any of them could produce the future emperor, all sexual partners of the ruler must be guarded from other men. This is to ensure that they could only be impregnated by the ruler himself, so that the correct progeny continue to sit on the throne.

Indeed, this is why harems were commonly staffed by eunuchs⁶:

In many of these cultures [practising castration] eunuchs were employed as harem attendants or guards. This was done to keep the attendants or guards from sexually interacting with the women or getting the women pregnant.⁷

The lack of formal rights to succession in the Roman system precluded the need for highly secluded, guarded harems in the style of the Orient. However, this does not mean Roman emperors behaved any differently in their sexuality.⁸ The Roman Empire was unabashedly a slave society. Slaves, being properties, were essentially at the mercy of their masters' whims. As is typically the case, slaves were commonly sexually exploited by their owners from at least the late Republican period.

In [the first half of the second century B.C.] promiscuous, sometimes brutal, sex indulgences with slave women becomes noticeable in the Roman historical sources. In the following century this freedom of sex indulgence with female slaves become more marked and is openly discussed by Horace.⁹

Naturally enough, the Roman Emperors were no exceptions, though they may have been under more pressure to keep their activities hidden from the Roman public. Tiberius for instance was said to have bought numerous prostitutes to the relatively remote island of Capri. Similarly, the historian Publius (or Gaius) Cornelius Tacitus reported that Nero tried to hide his night life of sexual debauchery.

[Petronius] described fully the prince's shameful excesses, with the names of his male and female companions and their novelties in debauchery, and sent the account under seal to Nero ... When Nero was in doubt how the ingenious varieties of his nightly revels became notorious, Silia came into his mind, who, as a senator's wife, was a conspicuous person, and who had been his chosen associate in all his profligacy. [Annales. 16. 19-20]

The luridly detailed and particularly depraved tales of orgies, incestuous rapes and virgin fetishism involving emperors reported by Suetonius may or may not be accurate. Yet the mere fact that such rumours were preserved indicates that contemporary Romans did not find them so implausible.¹⁰

<sup>References:
1. Schäfer, Peter. The Talmud Yerushalmi and Graeco-Roman Culture. No. 93. Mohr Siebeck, 2002. "[I]n the 2nd century, the birth of illegitimate children could be registered in the official records previously reserved for legitimate children, and by the early 3rd century Romans interpreted an earlier law in a manner that set aside the rights of patrons to inherit form their freedwomen's estates, in favor of illegitimate as well as legitimate children."
2. Phang, Sara Elise. The Marriage of Roman Soldiers (13 BC-AD 235): Law and Family in the Imperial Army. Vol. 24. Brill, 2001.
3. Hinsch, Bret. Women in Early Imperial China. Rowman & Littlefield Publishers, 2010. "Chinese marriage was theoretically monogamous ... An emperor might enjoy hundreds of consorts, but he had only one true wife."
4. Faruqui, Munis D. The Princes of the Mughal Empire, 1504–1719. Cambridge University Press, 2012. ""Like the Ottoman Empire, [the Safavid dynasty's] early succession practices reflected Turco-Mongol-inspired ideas that vested imperial sovereignty in all male members of the royal clan or family."
5. Rossabi, Morris. "Khubilai Khan and the women in his family". From Yuan to Modern China and Mongolia: The Writings of Morris Rossabi. Brill, 2014. ""Yet even the lowest concubines could rest assured that her children stood to inherit property from their father. They were not treated as illegitimate children without any claims to their father's wealth. Though the first wife was accorded a higher status than the rest and her children inherited a larger share of their father's property, the children of the other wives and concubines received a portion fo the legacy."
6. Smith, Bonnie G. The Oxford Encyclopedia of Women in World History: 4 Volume Set. Oxford University Press, 2008. "Eunuchs were, and are, most familiar from their role of attending upon and guarding women ... the appeal of eunuchs was that they could not impregnate the women they were charged with guarding."
7. Giles, James. "Sex hormones and sexual desire." Journal for the Theory of Social Behaviour 38.1 (2008): 45-66.
8. Veyne, Paul. The Roman Empire. Harvard University Press, 1997. "Emperors, even when married, kept a harem of slave concubines in the palace, and Claudius was known to sleep with more than one at a time."
9. Westermann, William Linn. The Slave Systems of Greek and Roman Antiquity. Vol. 40. American Philosophical Society, 1955.
10. Icks, Martijn, and Eric Shiraev. Character Assassination Throughout the Ages. Palgrave Macmillan, 2014. "[T]he precondition for its publicity was the fact that it did not seem implausible to contemporaries: people apparently assumed that all kinds of things might have happened within the hidden confines of the imperial palace. Thus it did not seem at all absurd to Suetonius that Caligula had transformed his palace into a brothel and forced noble women and boys to prostitute themselves there."</sup>