All of us have to deal with the original sin of Adam and Eve through the pain and suffering that exists in the World. Also because of this original sin all of us are sinful by nature, but through the power of the Holy Spirit we are capable of doing good, and through the death and resurrection of Jesus Christ will obtain salvation. But no I am not held to answer for sins that one of my distant ancestor might have committed.

The two scriptures you quote are in effect saying the same thing. Please note that in Exodus 20:5 it is speaking of worshiping Idols, while Ezekiel 19:20 is in reference to sin in general.

In Exodus God is it need be noted that God said to the third and fourth generations of those who hate me. where in Ezekiel God is making reference to a person's sins.

In essence they are two different things. where in Exodus it is worship of idols and in Ezekiel they have worship for the true God, but have sinned against God.

An example of Ezekiel is:

2nd Chronicles 6:26 When the heaven is shut up, and there is no rain, because they have sinned against thee; yet if they pray toward this place, and confess thy name, and turn from their sin, when thou dost afflict them; 27 Then hear thou from heaven, and forgive the sin of thy servants, and of thy people Israel, when thou hast taught them the good way, wherein they should walk; and send rain upon thy land, which thou hast given unto thy people for an inheritance.

and an example of Exodus is:

Joshua 7:20 through 26 And Achan answered Joshua, and said, Indeed I have sinned against the LORD God of Israel, and thus and thus have I done: 21 When I saw among the spoils a goodly Babylonish garment, and two hundred shekels of silver, and a wedge of gold of fifty shekels weight, then I coveted them, and took them; and, behold, they are hid in the earth in the midst of my tent, and the silver under it. 22 So Joshua sent messengers, and they ran unto the tent; and, behold, it was hid in his tent, and the silver under it. 23 And they took them out of the midst of the tent, and brought them unto Joshua, and unto all the children of Israel, and laid them out before the LORD. 24 And Joshua, and all Israel with him, took Achan the son of Zerah, and the silver, and the garment, and the wedge of gold, and his sons, and his daughters, and his oxen, and his asses, and his sheep, and his tent, and all that he had: and they brought them unto the valley of Achor. 25 And Joshua said, Why hast thou troubled us? the LORD shall trouble thee this day. And all Israel stoned him with stones, and burned them with fire, after they had stoned them with stones. 26 And they raised over him a great heap of stones unto this day. So the LORD turned from the fierceness of his anger. Wherefore the name of that place was called, The valley of Achor, unto this day.

Earlier in this chapter we see that it was God who pointed to Achan as the culprit. And that even goes to the first of the ten commandments, that is having no other gods before me.

The consequences of one's sins can affect one's children, grandchildren, etc., but that doesn't mean that God is directly punishing them.

I recently talked with someone that casually commented that he was raised in the foster system because his mother was a "crack who*e". He has trouble fitting into society and holding a job and is living on government assistance. He has to take medication because his "brain doesn't work right". His punishment is a result of his mother's "sins", but it is not any form of direct divine intervention; it is a natural consequence.

Similarly, should he have children of his own, they will not be raised with the benefits of a "normal" family. They will have problems. Again, this is an inevitable consequence of their grandmother's lifestyle, not a punishment from God.

We can only hope that their children will be more normal, will fit into society better, and will be productive and useful.

So yes, people are punished for their ancestor's sins, but that punishment is something that God is warning us against; it isn't something that he deliberately inflicts on us.

I believe you are asking two questions. I'll answer the one in the heading first:

Does God punish people for their ancestors' sins or not?

Yes, in the Christian traditions (majority of the denominations) God absolutely does "punish" the children of those who committed sin - else there would be no original sin.

1 Corinthians 15:22 KJV

For as in Adam all die, even so in Christ shall all be made alive.

and

Romans 5:12-21 KJV

12 Wherefore, as by one man sin entered into the world, and death by sin; and so death passed upon all men, for that all have sinned:

13 (For until the law sin was in the world: but sin is not imputed when there is no law.

14 Nevertheless death reigned from Adam to Moses, even over them that had not sinned after the similitude of Adam's transgression, who is the figure of him that was to come.

How can Exodus 20:5 and Ezekiel 18:20 be reconciled?

Reading the end of Ezekiel 18 explains it:

Ezekiel 18:30-32 NIV

30 “Therefore, you Israelites, I will judge each of you according to your own ways, declares the Sovereign Lord. Repent! Turn away from all your offenses; then sin will not be your downfall. 31 Rid yourselves of all the offenses you have committed, and get a new heart and a new spirit. Why will you die, people of Israel? 32 For I take no pleasure in the death of anyone, declares the Sovereign Lord. Repent and live!

Therefore, to repent and come to God will stop the inherited sin. In a similar regard, mainstream Christianity teaches that believing in Jesus (thus repenting) will stop the cycle of sin (see original sin) and allow you to live. Exactly what it means to "believe" in Jesus differs by denomination.

TLDR:

God punishes people for the previous generation's sin. Repenting stops this cycle but doing nothing continues the sin's wrath.