Was Modalism (or "modalistic language") ever considered to not contradict orthodox trinitarian beliefs?

I would say that this claim is totally bogus. Its more likely that in the second century the Spirit wasn't seen as a person of the godhead at all. After all, Justin Martyr (circa 140-150 AD) was the great champion of orthodoxy, and in his dialogue with Trypho he spends a great deal of time trying to explain how Jesus could be God (e.g. the analogies of light from light and fire from fire are employed) but he never touches the concept of the divinity of the Spirit.

Another possible issue to grapple with on this is divergent copies of the canonical gospels. We know from anti-heretical writers like Ireneaus (circa 180 AD) that different sects had different versions of the canonical gospels. For example, in the text in Mark (3:31-32) where we read the narrator comment "Jesus' mother and brothers were outside seeking to speak with him" followed by "Someone told him, 'Your mother and brothers are outside seeking to speak with you'" some sects had copies that lacked the narrator comment, and they put the interpretation on this that Jesus didn't really have a mother, and/or whoever said this was trying to trick him to test if he was really a prophet. Its conceivable that some copies of John possessed by some sects lacked the part that identifies the paraclete with the Spirit of Truth, thus enabling them to identify the paraclete as a man, whether that be the Apostle Paul, Marcion, Montanus, or whoever they wanted it to be.

Edit:

Concerning the Tertullian reference which Pelikan doesn't specify, he is referring to Against Praxeas chapter 3. Here is the relevant section from Holmes' 1870 translation:

The simple, indeed, (I will not call them unwise and unlearned, ) who always constitute the majority of believers, are startled at the dispensation (of the Three in One), on the ground that their very rule of faith withdraws them from the world's plurality of gods to the one only true God; not understanding that, although He is the one only God, He must yet be believed in with His own oikonomia. The numerical order and distribution of the Trinity they assume to be a division of the Unity; whereas the Unity which derives the Trinity out of its own self is so far from being destroyed, that it is actually supported by it. They are constantly throwing out against us that we are preachers of two gods and three gods, while they take to themselves pre-eminently the credit of being worshippers of the One God; just as if the Unity itself with irrational deductions did not produce heresy, and the Trinity rationally considered constitute the truth. We, say they, maintain the Monarchy (or, sole government of God). And so, as far as the sound goes, do even Latins (and ignorant ones too) pronounce the word in such a way that you would suppose their understanding of the monarxia (or Monarchy) was as complete as their pronunciation of the term. Well, then Latins take pains to pronounce the monarxia (or Monarchy), while Greeks actually refuse to understand the oi0konomia, or Dispensation (of the Three in One). As for myself, however, if I have gleaned any knowledge of either language, I am sure that monarxia (or Monarchy) has no other meaning than single and individual rule; ...

If that's all it means, there are obviously various positions that could be taken, and modalism is only one of them. If the "simple" were really so simple after all, then modalism is probably the least likely. A strict view that the Father is God alone and the Son and Holy Spirit are not could just as well be called Monarchianism in that sense, as also could more complicated notions like modalism and "dynamic monarchianism" (i.e. the idea that Jesus was a man who was adopted and then subsumed into God as it were).

In any case, as to Pelikan's claim that modalistic language was considered orthodox in the second century, we have a few issues:

1.) Tertullian is technically 3rd century. (first decade of the 200s)

2.) Its not typical to use lay opinion as the litmus test of orthodoxy. The fact is that the dominant lay view in every century probably remained about the same (non-Trinitiarian), and completely different from the establishment's views. Orthodoxy is always defined by the establishment in scholarly literature.

3.) "the church" in scholarly literature is always the clergy, not the laity.

One obvious reason for this is the clergy write theological treatises and the laity either do not, or if they did, the clergy destroyed them all. Its easier to count the written books by the "orthodox fathers" and tally them up and claim such and such is the consensus position of the church, than to count the unwritten opinions of the mass of laity who undoubtedly thought their clerics were crazy.

In other words, the Catholic standard of "what was believed by all Christians at all times" never counted the laity, but only the big writers. And even then, if there was a big writer who disagreed, he was labelled a heretic by the establishment, his books were burned, and then it was as if he had never existed, so all is well for orthodoxy with the "what was believed by all Christians at all times" criterion once again.