How is it determined what Culture is Period/Dynasty/Civilization part of?

My suggestions for "a con[structed] world" of yours:

• "Culture" is about artifacts and how we categorize these artifacts today; scientists need to understand other scientists. It's a hierarchy, a general culture can be made of many more specific cultures, etc. The artifacts are produced by many people, not by a king: if the pottery looks different it doesn't matter if it was made under the same king.
• "Civilization" is a "culture" that had cities.
• "Nation" does not describe ancient times, it's a modern concept. If you mean physical appearance then it's "ethnicity".
• "Kingdom"/"dynasty" is about who ruled; one person can rule one, many, or zero (as known to us) cultures. If some written source says there was a king John, we often have no way to connect it with any culture at all (if no 'piece of pottery' can be matched).

Permit me to introduce Wikipedia, a tool that can be used to answer many of these questions. Even if Wikipedia doesn't have a clear answer, every question on H:SE should be checked against Wikipedia and Google before it is asked.

Wikipedia gives you an overview of Culture, and explains that there is no simple definition. That said, the Kingdoms of Ancient Egypt have more in common with one another than they do with Mesopotamia. For most purposes, Egypt is a different culture (different religions, funerary practices, etc). If however you're discussing the differences between Egypt and contemporary Mesoamerica, then I think that you would be forgiven for referring to a Middle Eastern culture.

The distinction between culture and Dynasty is a bit more subtle. You refer to a number of references without providing any, which is kind of like asking me to solve 3x^2+2Y^2+13W without giving any of the variables. I don't know if you're reading those references in an encyclopedia, a scholarly journal, or a historical romance. That said, in general we refer to the Second Dynasty of Egypt, or the Silla Dynasty in Korea, which identifies a specific period within a cultural entity. Certainly there are cultural differences between the Federalist period and the ante-bellum period within the USA, but the terms help us to recognize a set of assumptions (for example, the role of political parties changes dramatically between those two periods).

Sometimes an empire may contain multiple cultures; Rome, Persia, Britain all contain multitudes of cultures. Sometimes they are subcultures - if you refer to snake handling culture, even though it is probably technically a sub-culture.

My professional historian girlfriend points out another contextual distinction - different fields use different labels. Fashion historians use the labels of political historical periods; economic historians use different terms. A historian studying religion will refer to the Great Awakening and the Second Great awakening, and their peers will know what they mean. (Ironically, as my professional historian girlfriend and I discussed that, I realized that I was mis-using the term, since that is not a period I tend to study).

Also, speaking in a generalized historical sense, the concept of "dynasty" works reasonably well during periods of personal or monarchical rule. That concept is much less useful after the emergence of the Nation State.

In summary, the questions you ask have no easy answers in theory. In practice, the usage depends on the audience for which one is writing. Depending on the framework of history for your constructed world, it may make sense to refer to the Foo Dynasty of the Bar culture, or it may be more appropriate to refer to the Baz administration of the Quar Empire. It all depends on what you want to construct.