Why were gravediggers, actors and former gladiators (but not slaves) banned from the Colosseum?

Slaves and prostitutes weren't of such a low status modern people like to assume. At least, not all of them. Also, some actors (like senators or even the 'bad' emperors, like Nero, and others fighting in the arena) could be famed and disregarded at the same time. Just like the virile sex symbols that were the gladiators.

By law, gladiators were not entitled to the full range of rights guaranteed to other Romans. They were considered infames, a category of shame that also included actors, prostitutes, pimps, and lanistae, all occupations that involved the submission of the body to the pleasure of others. These others, be they the audience, the lanista, the pimp, or the sexual client, controlled the body of the infamis; the absence of basic authority this entailed indicated to Romans that the infames were incapable of control, of the proper use of authority. Thus they were legally prohibited from a range of privileges that involved power. Infames were barred from running for office and from voting. The testimony of infames was not allowed in court. Those condemned to the arena lost control over dispensation of their property; they could not make wills before their execution.[Alison Futrell: "The Roman Games", Blackwell: Malden, Oxford, 2006, p130.]

The games were introduced as a spectacle as a funerary rite. That brings up a connection to fossores. But apart from poorer parts of society having to dig there own graves, the gravediggers were quite a proud people and not of such a low rank as genrally imagined.

Reading Juvenal describing senatorial women running off with these lowlifes for better sexlifes is quite indicative of the ambivalent attributes; that is of both women and gladiators. Prohibiting senators from fighting gladiators in the arena seems also less concerned about the well being of the aristocrats than to enforce status boundaries.

Petronius, Satyricon 126:25: Some women get heated up over the absolute dregs [of society] and can’t feel any passion unless they see slaves or bare-legged messengers. The arena drives some of them into heat, or a mule-driver covered with dust, or actors displayed on the stage. My mistress is one of this type. She jumps across the first fourteen seats from the orchestra and looks for something to love among the lowest crowd.

It seems further complicated by the fact that any regulations were held to be important but lousily enforced. More important than to really keep certain people out for good seems the aspect of keeping them separate from the good people:

Beyond the physical dangers they faced, gladiators were marginalised in a civic and political sense. Many were of slave status anyway, which meant that they had only the most limited legal and personal rights. But even those who were by origin freeborn Roman citizens suffered a whole series of penalties and stigmas when they became gladiators, which in many respects amounted to losing their status as full citizens. It involved much the same ‘official disgrace’ (‘infamia’ in Latin) as prostitutes and actors suffered by virtue of their profession. We know of Roman legislation from the first century BC that prevented anyone who had ever been a gladiator from holding political office in local government; they were also not allowed to serve on juries or become soldiers. Even more fundamentally they seem to have lost that crucial privilege of Roman citizenship: freedom from bodily assault or corporal punishment. Roman civic status was written on the body. Part of the definition of a slave was that, unlike a free citizen, his body in a sense no longer belonged to him; it was for the use and pleasure of whoever owned it (and him). A gladiator fell into that category, as the notorious oath said to have been sworn by recruits when they entered the gladiatorial camps proclaimed. Its terms no doubt varied from place to place, but Seneca quotes a version that has a gladiator agreeing on oath ‘to be burnt, to be chained up, to be killed’. Such a promise of bodily submission was completely incompatible with what made a free Roman citizen free.
It is hardly surprising then that gladiators are often treated as the lowest of the low in Roman literature, and symbols of moral degradation. […] By contrast, Roman politicians looking for a slur to cast on their rivals would often reach for the term ‘gladiator’. […] It is hardly surprising too that we know of repeated attempts by the Roman authorities legally to prohibit senators from fighting as gladiators in the arena.
But this prohibition should give pause for thought. For if the gladiators were so completely despised and abominated, why on earth would legislation have been necessary to prevent senators from joining them? One answer is that these regulations were more symbolic than practical. The function of law is often to proclaim the importance of boundaries, rather than literally to prevent people crossing them. The reason most of us do not commit incest is not, after all, that there is a law against it. Here we might be seeing another instance of Roman insistence that there was a firm line to be drawn between fighters in the arena and civilised (especially elite) Roman society. There is, however, plenty of evidence to suggest that gladiators were as much admired and celebrated as they were abominated.

No entrance tickets to the Colosseum survive, but we have examples from elsewhere and they must have existed: […] So far as we know spectators did not pay for their tickets; attendance was one of the perks of citizenship. But how they were distributed is not clear. Given that everything in ancient Rome, ‘free’ or not, had its price, then we should probably imagine that people paid for membership of clubs and societies to which free tickets were issued. Or men of influence, powerful patrons, distributed tickets to their dependants and clients. When they arrived at the Colosseum, spectators would find that the entrance and exit routes for different classes of seating were planned (as we shall see in the next chapter) in a complex pattern so that citizens of different status were kept rigidly separate. Roman snobs did not like to rub shoulders with the less privileged, even if they were squashed side by side with their equals.

Modern writers – ourselves included – have often laid enormous stress on the political stratification and collective identities paraded in the Colosseum audience. The spectators were a microcosm of properly regulated Roman society, sitting in their official Roman costume (the emperor Augustus, a stickler for restoring or inventing traditions, had insisted that all citizens attend shows in a toga), with the highest ranks occupying the best seats in the front and so on up to the slaves and women at the back. They acted out the social order in a ‘political theatre’. This is true – up to a point. But it can be over-stated. For in other ways the audience in the Colosseum displayed the ambivalences of Roman political status and the impossibility of fitting the messy realities of the Roman population into neatly ranked status groups. […]

But Roman society did not fit as easily as we often like to imagine into straightforward vertical status groups and the seating in the Colosseum probably blurred the legal distinctions as much as reinforcing them. Was it only the senators who sat in the ringside seats? Or could they bring guests and clients? Did some slaves actually sit at the front with their elite masters, or did the senators pay for their exclusivity by having to do without their everyday attendants at the ringside? [From Keith Hopkins and Mary Beard: "The Colosseum", Profile Books: London, 2011.]

For gladiators and actors it is indeed tempting to take their officially low status and conclude from there that this excludes them from many activities. Gladiators loosing their former status or never having had this status is straightforward in this regard. But this is apparently much too simple.

Tertullian, On the Spectacles 22.3–4:38 Take the treatment the very providers and managers of the spectacles accord to those idolized charioteers, actors, athletes, and gladiators, to whom men surrender their souls and women even their bodies, on whose account they commit the sins they censure: for the very same skill for which they glorify them, they debase and degrade them; worse, they publicly condemn them to dishonor and deprivation of civil rights, excluding them from the council chamber, the orator’s platform, the senatorial and equestrian orders, from all other offices and certain distinctions. What perversity! They love whom they penalize; they bring into disrepute whom they applaud; they extol the art and brand the artist with disgrace. What sort of judgment is this – that a man should be vilified for the things that win him a reputation?

One example for why these statements, like the one from the question ending in "were banned" are very likely overly comprehensive can be found in a famous actor: Pylades. Born a slave, became an actor, freedman and didn't stop there.

By far the best-known benefactor was a star of the Roman theatre in the time of the emperor Marcus Aurelius, the freedman Pylades. He was a pantomimus, an actor who mimed all the roles in an essentially serious dramatic presentation while the chorus and orchestra accompanied him and narrated the tale to the audience. Tremendous versatility and consummate talent were needed to achieve such a difficult feat successfully. ‘Pantomimes’ were amongst the highest paid and most celebrated performers in the Roman theatre. Pylades was a local slave boy made good. Although he was born a slave, he received his freedom from the emperor at the peak of his fame. He was patron of the guild of actors (parasiti Apollinis) and was the foremost pantomime of his age (pantomimo temporis sui primo), showered with honours (normally reserved for free- born Roman citizens) by his native Puteoli. He was entitled to wear the regalia appropriate to citizens who had held the chief magistracy (duovir) and who were members of the town council (ordo decurionum [honorato Puteolis, d(ecreto) d(ecurionum), ornamentis decurionalib(us) duum- viralib(us)]. He was given the honorary municipal title of augur for his devotion to his home town (patria) and his unstinting generosity in presenting gladiatorial shows with a beast hunt of assorted types of animals (in edendo muner(a) gladiatorum venatione passiva) with the express permission of the emperor Commodus. The gladiatorial spectacle may have been given in response to the award of the right to wear the regalia and insignia of a duovir and a member of the town’s senate, i.e., as if he had been elected duovir (in which case he would have had to give such games). [From D. L. Bomgardner: "The Story of the Roman Amphitheatre", Routledge: London, New York, 2000: p88.]

This seems to indicate that at least the phrasing of "banned" is somehow problematic, since this doesn't mean that no actor ever attended the games. Such regulations, being more symbolic than anything else, describe foremost the self-image of the society – not the reality of it. Finding examples that counteract to these regulations seems easy. Roman society is seen as heavily stratified. But there is legitimate reason to see this much laxer than proscribed. People sneaking in, in the sense of sitting where they didn't belong, seems to have occurred often. They had to have a ticket, or a benefactor, then they were "in".

The 'reasons' for 'banning' groups, by that I mean effectively limiting their numbers from entering and/or separating them into disregarded spaces, might lie in their official status, their limited ability to pay, or membership in clubs that did not receive free tickets. But official reasoning by writers, senators or laws is much less precisely mirrored by what went on in real life.

In relation to the question one could also ask, why were the games banned in the 1st century? Were they? They were. In 59 in Pompeii some collegia of iuvenes had such a riot going that as a punishment both the games and the clubs were suspended. But this did not apply to the whole empire and not forever. If there was a ban in place, we would have to specify when and were that ban was.

Tacitus, Annals 14.17:28 About the same time a trifling beginning led to frightful bloodshed between the inhabitants of Nuceria and Pompeii, at a gladiatorial show exhibited by Livineius Regulus, who had been, as I have related, expelled from the Senate. With the unruly spirit of townsfolk, they began with abusive language of each other; then they took up stones and at last weapons, the advantage resting with the populace of Pompeii, where the show was being exhibited. And so there were brought to Rome a number of the people of Nuceria, with their bodies mutilated by wounds, and many lamented the deaths of children or of parents. The emperor entrusted the trial of the case to the Senate, and the Senate to the consuls, and then again the matter being referred back to the Senators, the inhabitants of Pompeii were forbidden to have any such public gathering for ten years, and all associations they had formed in defiance of the laws were dissolved.

In conclusion it looks as if "…were banned" is a quite contestable depiction of simplicity. Before answering "why" we need to ask when and where, or were they?