What did Paul mean when he wrote 1 Corinthians 14:22 in the context of 1 Corinthians 14:23-25?

I would interpret it as such: within the context of the chapter as a whole, Paul gives instruction pertaining to church discipline, so this bit seems to be saying "these gifts are good, when used with careful discernment, and for their purpose. Outside of that, it can be detrimental to both believers and unbelievers."

The three verses, read in isolation, do seem contradictory. However, sense can be made of them by reading chapter 14 as a whole. Paul is aware of a strong belief in speaking in tongues, and is tactfully opposing the practice, suggesting prophecy as a better alternative. In 1 Corinthians 14:4, Paul says "Whoever speaks in a tongue builds himself up, but whoever prophesies builds up the church." He then goes on to explain what he sees as the futility of speaking in tongues, summarising this in verse 19: "but in the church I would rather speak five words with my mind, so as to instruct others also, than ten thousand words in a tongue."

The Corinthians prided themselves on tongues as a sign of God's favour, believing it to be a means of direct communication with him. Paul challenges them in verse 14:20: "Brothers, stop being childish in your thinking. In respect to evil be like infants, but in your thinking be mature."

Paul quotes from Isaiah 11:12 in 1 Corinthians 14:21, as an example of God speaking to the Jews in a strange language and they would not listen. In 1 Corinthians 14:22, he interprets that adversely for the Corinthians: "Thus (KJV: wherefore), tongues are a sign not for those who believe but for unbelievers, whereas prophecy is not for unbelievers but for those who believe." This is not Paul's view, but the construct he places on the passage he quotes.

Having demolished the case for speaking in tongues, Paul instructs the Corinthians in 1 Corinthians 14:23-25:

So if the whole church meets in one place and everyone speaks in tongues, and then uninstructed people or unbelievers should come in, will they not say that you are out of your minds? But if everyone is prophesying, and an unbeliever or uninstructed person should come in, he will be convinced by everyone and judged by everyone, and the secrets of his heart will be disclosed, and so he will fall down and worship God, declaring, "God is really in your midst."

This is what Paul meant in 14:4, when he said, "Whoever speaks in a tongue builds himself up, but whoever prophesies builds up the church."

How do we interpret 1 Corinthians 14:22 in the context of 1 Corinthians 14:23-25?

1 Corinthians 14:22 Wherefore tongues are for a sign, not to them that believe, but to them that believe not: but prophesying serveth not for them that believe not, but for them which believe.

1 Corinthians 14:23-24 If therefore the whole church be come together into one place, and all speak with tongues, and there come in those that are unlearned, or unbelievers, will they not say that ye are mad? But if all prophesy, and there come in one that believeth not, or one unlearned, he is convinced of all, he is judged of all:

The first apparent contradiction is that the "sign" of tongues for unbelievers seems to be refuted by the statement that unbelievers will be unsettled by the use of tongues. What is being said is that everyone speaking tongues at the same time will cause confusion, the supernatural speaking of one language would still be a "sign".

The second apparent contradiction regards the intended audience of prophecy. Prophecy is not declared a "sign". It is declared as useful for believers. While prophecy is also declared not useful for unbelievers, the "unbelievers" of verse 24 are those who might become (potential) believers.

As a side note, the word for "unlearned" is "idiot". The word originally meant someone who did not wish to participate in the municipal activities of the polis (city). It came to mean at New Testament time those who were of a lower social order such as enlisted men in the Army as opposed to officers or a layman as opposed to a judge. The implication is that someone who is unable to function at a higher level.

While the spoken word of God (prophecy) would be of little use generally to unbelievers, those who heard these words at the specific time the Holy Spirit was giving them conviction would be brought to repentance and faith.

Romans 10:17 So then faith cometh by hearing, and hearing by the word of God.

The context is that tongues are for a few individuals to demonstrate to a group of unbelievers of the same tongue the power of God. It is only a sign to the unbeliever if the unbeliever is of that tongue and can understand it.

(Acts 2:8) And how is it that we hear, each in our own language in which we were born?

But as verse 23 demonstrates, if the whole church comes together and speaks in tongues what benefit is it to anyone? It is highly unlikely the next unbeliever to walk in is of this tongue, unbelievers of all backgrounds could walk in, it may be that no one understands it.

23 ...will they not say that you are out of your minds?

If you read the entire chapter of 14, it becomes even clearer, speaking in tongues to those who do not understand it is of no use.

14 Pursue love, and desire spiritual gifts, but especially that you may prophesy. 2 For he who speaks in a tongue does not speak to men but to God, for no one understands him; however, in the spirit he speaks mysteries. 3 But he who prophesies speaks edification and exhortation and comfort to men...

18 I thank my God I speak with tongues more than you all; 19 yet in the church I would rather speak five words with my understanding, that I may teach others also, than ten thousand words in a tongue.