Why couldn't a person in the OT be both priest and king?

For me, I believe it was easier to be a king in the old testament than it was to be a priest! This was because priesthood was restricted to the tribe of Levi but kingship had no restriction thou a prophecy about Jesus "the sceptre shall not depart from Judah" was there, it wasn't meant for the time. There were chances for Levites to be king in Israel but they preferred and held unto their priesthood as a divine call and office like: 2Kings 11 where Jehoiada the priest could have taken over as king but handed it over to a seven year old Jehoash as the rightful heir.

Jesus being a Priest and King in the order of Melchizedek speaks more of the similarities in their positions as Melchizedek was superior to Aaron in time, quality and office according to Hebrews 7. Melchizedek was a priest of the most migh God, a king of righteousness and peace, without genealogical history.

Before Moses came, the head of the family was the priest and the firstborn son of each household inherited the priest's office. After the exodus from Egypt, Moses and his tribe, the tribe of Levi were ordained to become priests. After the Law was given, only the Levites were allowed to become priests. (More information on Priests and Levites)

Then Moses said, “You have been set apart to the Lord today, for you were against your own sons and brothers, and he has blessed you this day.” (Exodus 32:28, NIV)

Though the Levites were set apart for the Lord, it is not clear whether they can become kings or not. Melchizedek was a king and a priest, Moses was a priest and a ruler, Samuel was a priest and a ruler. Hence, if a Levite was annointed by Samuel to be the king of Israel, it would have been possible. However, Samuel did not choose from the Levites, probably because the Levites were already set apart for the temple services.

The problem with Saul was that he was a king and not a priest. Saul was from the tribe of Benjamin, not a Levite. If Saul were a Levite and a priest, it would have been legal for him to perform the sacrifice but he was not. Saul was a king, not a priest, and it was unlawful for him to offer sacrifices to the Lord. Because Saul disobeyed the Law, God was angry with him.

A king could also be a priest, just not a member of the Aaronic priesthood.

As Mawia has already stated, in pre-monarchic Israel and throughout the Levant, both father (as head of the household) and firstborn sons typically served as priests (e.g., Numbers 3:13). In addition, the king typically served as a priest for the entire nation (as with Melchizedek in Genesis 14).

Since according to the Mosaic Law, only the direct male line of Aaron could serve as Aaronic priests in the Tabernacle and Temple (Exodus 28:1, Numbers 3) undertaking sacrifices to God, and as the kings of Israel and Judah all came from the tribes of Judah, Benjamin, Ephraim, etc, they were disqualified from birth.

However, the royal Psalm 110 written by David indicates that God had promised that the Davidic king could also be a priest (v.4). There are several different ways to translate the ambiguous Hebrew verse:

1. that the Davidic king is a priest forever "in the order of Melchizedek", i.e., in the same mannner that Melchizedek was a king-priest;
2. that the psalm is actually being addressed to Melchizedek, the original king-priest, saying that Melchizedek will forever remain a priest;
3. that the Davidic king is "a priest forever, a rightful king by My decree".

Now, if we go with the first or third interpretation, then the functions of the Davidic king, as a (Melchizedekite?) priest, are mostly unknown. We only know that the king was not allowed to personally make sacrifices to God (1 Samuel 13). David's behavior in 2 Samuel 6:14-15, wearing a linen ephod (priestly attire) and dancing in ecstasy before the Ark, may be another example of the king's priestly role in action.