Does the Catholic Church have any mechanism to improve poor homilists?

The quality of Catholic preaching has been a matter of concern for quite a long time, but particularly since the Second Vatican Council. But there is relatively little specific instruction mandated (on a Church-wide basis) to ensure that homiletics is a topic of constant study for priests.

Primarily, I think, this is because the Church is quite decentralized in many respects. As I stated in the third paragraph of my answer to an entirely different question, the Church considers itself something like a federation of more-or-less local churches, called particular churches. Roughly speaking, each of these is headed by a bishop, or an administrative equivalent, who has the power to (among other things) decide how ongoing training for priests will be conducted.

One of the documents issued by Vatican II was a decree titled Presbyterorum Ordinis ("Of the order of priests"), which discussed some of the important tasks and requirements of the priesthood in light of the Council's changes. Part 19 of this document states:

Let priests more readily study and effectively learn the methods of evangelization and the apostolate. Let opportune aids be prepared with all care, such as the institution of courses and meetings according to territorial conditions, the erection of centers of pastoral studies, the establishment of libraries, and the qualified supervision of studies by suitable persons. Moreover, let bishops, either individually or united in groups, see to it that all their priests at established intervals, especially a few years after their ordination, may be able to frequent courses in which they will be given the opportunity to acquire a fuller knowledge of pastoral methods and theological science.

Thus the importance of supervised study in evangelization was already foreseen.

Nine months after the promulgation of this decree, Pope Paul VI issued a general letter motu proprio ("on his own account") titled Ecclesiae Sanctae ("Of the Holy Church"). Among other things, this dealt with how the norms of Presbyterorum Ordinis would be implemented. Section 7, which corresponds to Part 19 of the prior document, reads:

Bishops either individually or collectively should make provisions that all priests, even if engaged in the ministry, complete a series of pastoral lectures in the course of the year immediately after ordination and that they attend at specified times other lectures in which an opportunity is given to their priests both to acquire a fuller knowledge of pastoral methods and of the theological, moral and liturgical sciences, and to strengthen their spiritual life and to share their apostolic experiences with their brother priests.

Bishops or episcopal conferences should also see to it that, according to local conditions, one or several priests of proven learning and virtue are chosen as moderators of studies to promote and arrange pastoral lectures and other aids which are considered necessary to foster the scientific and pastoral training of priests of their own territory: study centers, mobile libraries, congresses on catechetics, homiletics or the liturgy and other subjects of this kind.

(emphases added)

Thus, the formal instruction is simply that each bishop, alone or in conference with other bishops, decide what's necessary to help their priests, and how often they need to be helped. Nothing further, or more specific, is ever stated.

The U.S. Conference of Catholic Bishops has a section of their website dedicated to improving homiletics; it has published, or linked to, documents emphasizing the importance of good homilies and discussing how to improve one's homilies, as well as links to resources from other Catholic organizations intended to help priests improve their preaching.

There does not appear to be any mechanism of the specific sort that you mention, of the bishop's representatives "grading" a practicing priest on his homilies. I'm not familiar with any discussion of such a possibility, or any discussion of why it might be considered a bad idea.