How was 'years' defined in the book of Genesis?

It is hugely wrong to suppose that there existed poor levels of sophistication in the ancient civilisations before the Roman/Greek era especially when it comes to calculations of the length of the year.

If we accept the Bible's own testimony, then the Pentateuch was written by Moses who died 1406 BC, having been very well educated in Egypt.

Egypt

At the time of Moses the Egyptians used a Civil Calendar of 365 day, with no extra days to make up for discrepancies. (So New Year's Day would shift about a quarter of a day each year and circle round and back to the same day of the solar year after 1461 years.) The year had 12 months of 30 days each and 5 extra "epagomenel"/filler days at the end of the year.

The Egyptians used the 365 day year for so long that New Year’s Day drifted round the solar year to cycle back to the starting point almost 3 times. They used it for almost 4500 years. https://pumas.jpl.nasa.gov/files/04_21_97_1.pdf

Does this mean that Ancient Egypt was unaware of the actual length of the Solar Year? No. It became apparent to them that the seasons, particularly the beginning of the inundation/flood of the Nile season was falling further and further out of step with the Egyptian Calendar year of 365 days (by roughly 1 day every 4 years, of course). This innundation was important for them to be able to predict. They also found that the star Sirius (they called it Sopdet, the Greeks called it Sothis) was a better predictor of when the innundation would begin (this being because Sirius follows the Solar year very accurately). When observed from the Egyptian city of Memphis the rising of Sirius in the morning is too close to the Sun to be observable for 70 days of the year from early May to the middle of July.

(Sirius is the brightest true star in the sky, because it is the nearest star. It is pointed to by Orion's belt and is down nearer the horizon than the belt. If you travelled there on a bicycle [remember the flying bicycles in the film "ET"?] at 10 mph, 24/7, all year round it would only take about five hundred and ninety million years to get there.)

The first morning in July when Sirius became observable after the 70 day break thus became a critical observation for the ancient Egyptian priesthood, because it anchored their lunar months with the Solar Year. (It also enabled a prediction of the Nile innundation/flooding.) The priests wanted their lunar celebrations to be in harmony with the Solar Year, by which I mean they wanted their lunar months to be as close to the Solar Year as possible. So they added an extra month every so many years to try to keep their religious celebrations at the same time of the Solar Year. So even though the New Year's Day of the Egyptian 365 day Civil Calendar wandered around the Solar Year until it was back to where it started every 1461 years, the Priests in the Temples ensured their religious celebrations happened the same time of the Solar Year and the critical anchor for their year was the first summer "Heliacal rising of Sothis/Sirius" (and possibly they thought of this as the New Year's Day of the Religious year). (A heliacal rising is where a star rises in the morning sky before sunrise, sunrise being the appearance of the first segment/blinding ray of the sun's disk.)

God did not want the his people to be observors of the heavenly host because this would have tempted them to worship the heavens

During the 40 years of wandering in the wilderness (Leviticus 18:3) after the Exodus, and more specifically when at the foot of Mount Sinai (Leviticus 27:34), Moses wrote Leviticus, in which was instituted a lunar/solar year. The Israelites followed a lunar calendar of 12 lunar months (354 days) but (the High Priest) added an extra lunar month to the length of the year if the last harvest of the autumn had not been gathered in by the fifteenth day of month 7 (This is what I think Leviticus 23:39 is saying, i.e. it is my opinion). So the Israelites between the time of Moses and the Captivity (about 600 BC) were not taken up with attempts to measure the length of the year by astronomical observations, in the way that other cultures around them were, but rather were guided by the timing of the last harvest of the agricultural year in the autumn. What they did observe was the moon to decide when each month began.

In practice obedience to Leviticus 23:39 would have resulted in the addition, on average over the centuries, of 7 months per 19 years. The extra month could have been an additional month 7 (Tishri) so in a year where a month was added there would have been Tishri 1 and Tishri 2. Or, more likely, it would have been an additional month 6 where the High Priest would anticipate whether the harvest could be fully gathered in before the set date (15th of month 7), and if not he would add an extra month 6 (Elul) making Elul 1 and Elul 2.

So 12 of the years in the 19 year cycle had about 354 days, and 7 of the 19 years had about 384 days.

(A lunar month is 29.53059 days, so 19*12 lunar months and 7 lunar months is 6939.69 days. And 19 solar years is 365.2421875 * 19 = 6939.6 days.)

The Babylonians

By the time of the end of the Babylonian Empire (539 BC) a system of adding 7 lunar months every 19 years had been formalised and adopted by the Babylonians to keep the year in step with the solar year to err by about 129 minutes every 19 years. ((A vast improvement on the Egyptian Civil Calendar which erred by about 4.5 days over 19 years.)) Thereafter it was adopted by the Jews from the Babylonians. For religious purposes the Jews still follow the same calendar. https://en.wikipedia.org/wiki/Hebrew_calendar

The Sumerians

The even more ancient Sumerian city states (more ancient than Babylon) were the first we know in the Middle East to produce a calendar with added intercalary months from at least as far back as 2000 BC. See http://www.projectglobalawakening.com/sumerian-calendar/

But who was first between the Egyptians and the Sumerians I am not sure that we know for certain.

[In the modern world many cultures use a decimal system (after 9 comes 10); the ancient Sumerians (about 2000 bc) used a hexadecimal system (numbering up to 60). This form of numbering from the ancient Sumerians has been handed down to the present day... there are 60 seconds in a minute, and 60 minutes in an hour. And it was the Sumerians who chose there should be 360 degrees in a circle which they used because it was an approximation to the number of days in a year.]

Knowing when the Full Moon would happen was useful for doing many things in the cool of the night when it was also light, such as religious festivals, travelling large distances, or even heavy work around the house or on the farm. Knowing when there was a New Moon, ie no moon at all was useful to know in order to avoid planning these same activities on a cool night. In particular knowing about the lunar cycles were valuable for travelling large distances as a trader, and religious festivals, because both of these need at least some amount of long term planning.

Before the time of Moses, and especially before Noah's Flood

It is quite possible that some agricultural societies, well before the time of Moses, measured a year to be the length of time between one annual harvest and the previous one.

The question about the book of Genesis is how did Moses produce it? Did he receive all the information by a direct revelation from the Lord, or did he receive all or some of the information from previous sources (which we can assume were themselves writing under divine inspiration). If he received them from inspired sources then we would need to know the length of a year understood by those original sources. For more on this see my answer to: Who documented biblical events before Moses?

In addition to this we do not know if conditions before the Flood were precisely the same as they are now. God could have used a natural cause to bring about the Flood which could have had other consequences as well such as the earth's distance from the sun and the orbit of the planet. Or God could have judged mankind with other changes to the planet by a direct Divine act. We do not know, unless Scripture tells us, if the earth's orbit and the length of the year were different before the Flood, though it is difficult to imagine such a difference would be anything more than a few days per year.