The definition of "heretical" is not universally agreed upon, even by Catholics and Protestants, particularly when applied to views that preceded the final establishment of "orthodoxy." So rather than attempt to list of trinitarian "heretics" from the early church, perhaps it would be helpful to identify leading thinkers and their contributions, and how those contributions were received. I'll note five types of thought and their prominent adherents in the first three centuries of the church:

• Early imprecision
• Economic trinitarianism
• Dynamic monarchianism
• Modalistic monarchianism
• Origen's platonic trinitarianism

Here I'll mostly rely on the 4th edition of Early Christian Doctrines, by J. N. D. Kelly, a standard text in this field.

Early imprecision

In the first century or so after Christ's death, we see a number of authors describing God, but generally not attempting to do so in a systematic fashion. With the benefit of hindsight, we can identify a number of deficiencies in their thought compared to orthodox trinitarianism. For example, Ignatius has been accused of economic trinitarianism (see below), and Kelly calls the theology of Hermas an "amalgam of binitarianism and adoptionism." (94)

Regarding the Apologists (like Justin Martyr and Athenagoras), Kelly reports that they are often criticized for failing to teach the eternal generation of the Son, and by extension failing to protect the Son from being subordinated to the Father. He attributes this more to their "thought-atmosphere" and their lack of a "technical vocabulary adequate for describing eternal distinctions within the Deity" than to an actual failure to distinguish the persons or protect the Son's equality. (101)

Economic trinitarianism

With Irenaeus, recognized as a saint by Roman Catholicism, we have a more nuanced view of the Godhead, though still lacking in several particulars. Kelly describes his view:

Its second-century traits stand out clearly, particularly its representation of the Triad by the imagery, not of three coequal persons (this was the analogy to be employed by the post-Nicene fathers), but rather of a single personage, the Father Who is the Godhead Itself, with His mind, or rationality [the Son], and His wisdom [the Spirit]. (107–8)

His view is called "economic" trinitarianism, because it emphasizes the unity of God's essential being, and considers the Son and Spirit to be forms or modes that only through the Father's self-revelation are found to be distinct from the Father. So while Irenaeus sees an eternal relationship between the Father and Son, he still does not teach eternal generation. (106)

Tertullian and Hippolytus stand in this line of thinking and develop it further through their conflicts with monarchians (see next sections). They introduce the idea of "Persons" in the Godhead, but continue to see the Three as "expressions or forms," and explicitly describe a single divine substance. (114) Both of these men were schismatics, though apparently Hippolytus was reconciled to the church before his death.

Dynamic monarchianism

Next we turn to those who are more typically regarded as having heretical views of the Godhead. Dynamic monarchianism, also known as adoptionism, is the idea that Jesus was a mere man who was filled with God's spirit. The earliest proponent of this seems to be Theodotus of Byzantium in the late 2nd century. (116) He was excommunicated, but others followed in his wake, notably Paul of Samosata, who was condemned for these teachings in AD 268. (117)

Modalistic monarchianism

Now we come to perhaps the best known of the early trinitarian heresies, modalistic monarchianism, commonly known today as simply modalism. It was influential during its own time as well, and may have even gained some sympathy from a couple third-century popes.

Kelly suggests that some opponents of Justin Martyr held to this view, but identifies Noetus of Smyrna as its first formal proponent. Noetus affirmed only one God, the Father, and apparently had no difficulty with patripassianism, the idea that the Father suffered and died on the cross. (121) Another prominent proponent, Sabellius, refined the doctrine and cast it in language similar to that of economic trinitarianism, but continued to see the Son and Spirit as projections of the Father. (122)

Sabellius's contemporaries Tertullian and Hippolytus aggressively opposed this doctrine, and the dispute was far reaching. Kelly writes that Popes Zephyrinus and Callistus both "sympathized with the widespread popular reaction against the theories of Hippolytus and Tertullian, which they regarded as leading to ditheism," while Hippolytus "considered Zephyrinus an out-and-out modalist." (123) Kelly concludes that these popes, though sympathetic to modalism, were struggling to develop a compromise approach that would alleviate the dangers they saw in a concept of multiple "persons." (125) And ultimately, Callistus excommunicated Sabellius. (121)

Origen's platonic trinitarianism

In the East, another strain of trinitarianism thought developed in the context of Platonism. Origen saw each member of the Godhead as a "distinct hypostasis" from all eternity, going beyond the "economic" view of Tertullian and accepting the idea of eternal generation. He is criticized for the "strongly pluralist strain" (131) in his trinitarianism, and some have considered him to be virtually a tritheist. Subordination of the Son and Spirit is also prevalent in his scheme, and was emphasized by some of his followers (like Dionysius, bishop of Alexandria, in his fight against Sabellianism). (133)

In the early conflicts over Origen's trinitarianism, Kelly sees a precursor to the wider West–East conflict over the proper understanding of the Godhead. Not merely a debate over terminology, he sees it primarily as theological dispute between the West's sympathy toward monarchianism and the East's more pluralistic approach. (136)

Conclusion

In this brief overview of "trinitarian" debates of the first three centuries, we see a variety of non-orthodox views but also a slow process of refinement that would finally culminate in the Arian controversy and the Councils of Nicea. Given this progression it seems unfair to label all these thinkers as "heretics," even though they differed in various ways from trinitarianism. Some clearly diverged from the developing orthodoxy, it's true, but the struggle to clearly communicate the ideas of God's threeness and oneness burdened the church's greatest thinkers of those years, and they didn't get the opportunity to reevaluate their views in light of Nicea.