How accurately could ancient astronomers find latitude and longitude?

The Greek astronomers (e.g. Ptolemy) could calculate longitude and latitude using spherical trigonometry. Their calculations are accurate on the assumption that the Earth is a perfect sphere. Our astronomers today believe that the Earth is slightly pear-shaped and consequently arrive at a slightly different calculation of longitude and latitude.

Latitude

To measure latitude, you must measure the elevation of some celestial body. Basically, you will use the the Sun, or stars (trajectory of planets and the Moon is too complex to serve much here).

If you use the Sun, then you are using a projected shadow (you don't look at the Sun directly). You have some big pole, that you try to erect as vertical as possible; and you measure the length of the shadow at noon on the equinox. You will need to do some year-long measures to work out when the equinox actually is. The crucial point is that the Sun is not a point in the sky; it has an apparent diameter of about 30' (half a degree). This is the reason why, when you look at the shadow of some building, projected on the floor, the shadow edge is blurry: this transition zone between shadow and non-shadow corresponds to the floor spots from which the Sun is partially visible, and partially hidden by the building. The bottom-line is that a Sun-based measure of latitude tends to be imprecise: accuracy is within half a degree, but no better. (With a sextant you can have a much better accuracy, but that's because that apparatus includes filters allowing the operator to actually look at the Sun, and target the disc edge, instead of "the Sun in general" as in a shadow-based measure.)

With stars you can do potentially better, because they are points (at least to the naked eye) and you can stare at them directly without going blind. If you use stars, then you must follow several for a night, noting their azimuth and elevation throughout the night: this is sufficient to recompute their apparent trajectory, and then work out the latitude. The accuracy of the human eye is, at best, 1' (1/60th of a degree). However, it is hard to achieve in practice.

Notably, even if you can see a 1' angle deviation, the measure will depend on the accuracy with which you know the geometric characteristics of the device you are using (including the measure of "vertical" and "horizontal"). Also, prior to Gauss and Legendre in the early 19th century, astronomers had no systematic method to deal with measure errors and smooth them out with averages and statistics.

As a data point, Tycho Brahe, in the late 16th century, achieved measures with about 2' accuracy on average. These measures would translate, indeed, into a computation of latitude with the same accuracy. It must be noted that Brahe had very good eyesight, was exceptionally stubborn, and benefited from the precision offered by late Renaissance instruments when it came to measuring the length of, say, a ruler (according to David S. Landes, we have to thank clockwork technology for the availability of such tools in the Renaissance).

As another data point, the Great Pyramid of Giza (built circa 2560 BC) is aligned on the cardinal point within 4'.

From all these informations, we may conclude that astronomers from around 1 AD could achieve a measure of latitude with an accuracy of about 4' or so, but at considerable expense. Hipparchus has apparently done so in some occasions, but he had dedicated his life to such matters.

Longitude

Longitude is much harder: it can be measured by the difference between the local time and a reference time. If the Sun appears to reach noon while your watch says 2 o'clock (while it matched the Sun in your home town), then you know that you went 30 degrees to the West. This is about the only direct way to measure longitude: you need to bring a clock with you, and you will get as much accuracy as your clock provides, with 1 degree of longitude for every 4 minutes of time. Since clocks in antiquity were awfully inaccurate, this was not workable at that time. Indeed, longitude is measured by the difference in time between a sundial (which measures local time) and a clock (set to the reference time). When the clock is less accurate than a sundial, it is quite hard to reach any conclusion at all.

Some indirect measures can be done in rare occasions but need astronomical devices that were not available at that time (e.g. telescopes to observe the transit of Venus when it passes between the Sun and the Earth).

All longitude estimates in antiquity use the indirect method by which the longitude is inferred from the actual land distance, obtained through some other mean (mostly triangulation with notable geographical features such as hills and buildings). This works well with short distances (e.g. between Athens and Corinth), much less for long distances, and very badly when a sea is involved. Contrary to latitude, astronomers in antiquity could not get a notion of absolute longitude, only relative for locations which are close enough to each other.

First, a spot of background science. The Longitude Problem is exactly identical to the problem of establishing simultaneity on widely separated locations on the Earth's surface, and both prerequisite the existence of a reliable estimate of the Earth's diameter. Certainly Eratosthenes calculated the Earth's diameter in the 3rd Century BC, and other civilizations may have done so by roughly the same time period. However the problem of establishing simultaneity is more difficult, and comes in two flavours.

Longitude is calculated by comparing the elevation of an astronomical object to the pre-calculated (or observed) elevation of the same object at a reference location at the precisely simultaneous moment in time. Everything in the sky rotates once around that vast celestial sphere every 24 hours, so the more precisely one can establish simultaneity the more precise one's measurement of longitude will be.

The problem is simpler when the goal is cartography - calculating the longitude, and thus precise location, of a given spot on the globe exactly once. In this case one can use the occurrence of a predicted astronomical event as the definition of simultaneity. Surveying teams are organized to travel to the specified locales well in advance of the event, and providing the skies are clear on the given day the necessary observations are made. Once the surveying teams return the results are tabulated and the maps drawn.

The more difficult problem, and the one that confounded the British Admiralty into establishing the Longitude Prize, is of establishing the location of a moving vessel out of sight of land at whatever time the skies happened to be clear, wherever and whenever that was. One could not halt a sailing vessel in the middle of the ocean and wait for a pre-calculated event that occurred once or twice a month at best. It was necessary to resort to Dead Reckoning, a well-established and remarkably accurate science by the 17th and 18th centuries, which provided locations within one or two dozen miles on voyages thousands of miles long. When the goal was simply to voyage out and return home, this was more than adequate. However when the need is to avoid reefs of only a few hundred yards extent, being off by a few miles all too often results in foundering instead of sailing safely by.

The accuracy of dead reckoning can be judged by the quality of 16th and 17th century maps, reproductions of which are readily available all over the web. Don't be misled by the contours of western North America - those are due to the wanderings of the North Geomagnetic Pole.

Latitude

To find the latitude of a point on the land, one would simply have to measure the elevation of Polaris above the horizon. Therefore the question of the precision of (land-based) latitude determinations in this period reduces to the question of how accurately people could measure angles in the sky. The Almagest was the state of the art during this time, and its angular measurements seem to have been precise to about 15-30 minutes of arc.[Goldstein 1976]

Longitude

Since there were no accurate clocks until Galileo, and no sea-transportable accurate clocks until much later, the determination of a longitude in the ancient world would have been equivalent to estimating an east-west distance (using surveying chains, estimates of sailing speeds, ...) and dividing by the size of the earth. Columbus appears to have underestimated the size of the earth by about a factor of 2, leading to his belief that he could reach China and Japan by sailing across the Atlantic. So even as late as the Italian renaissance, the conversion factor between longitude and distance appears to have been uncertain by about a factor of 2. This is very roughly comparable to the precision of the ancient estimate of the radius of the earth by Eratosthenes, i.e., there was not much improvement over 1700 years.

Goldstein, Journal for the History of Astronomy, 7 (1976) 54, http://articles.adsabs.harvard.edu//full/1976JHA.....7...54G/0000054.000.html

Latitude can be calculated from observations of stellar objects (typically using something like an astrolabe) and a bit of math. The Greeks could do this as early as 150BC, but only on dry land. The mariner's astrolabe wasn't invented until around 1300 CE.

Nobody had a good way to determinte longitude in real-time aboard a ship before the invention of the marine chronometer in the early 1700's. The closest anyone came was the Chinese, who managed to work out the longitude of various places on the Indian trade routes in 1421 by placing observers on said places to observe various lunar and stellar positions simultaneously. This information may have made their maps better, but it wasn't particularly useful to a navigator out of sight of land.

Before that, the typical technique used was dead reckoning, which was incredibly inaccurate. Basically, the navigator would chuck a hunk of wood out the back of the ship, try to estimate their speed based on their relative speed to the jetsam, and try to calculate their distance from the last time they did that based on that speed. Obviously this doesn't take currents into account at all, and any errors are likely to accumulate every time you do it.

What was typically done in the Mediterranean in ancient times was that navigators just kept themselves in sight of land. Even then, bad things could happen. For instance, the Odyssey is essentially a story of an ancient Greek who got blown off course sailing home from nearby Anatolia, and spent 10 years trying to find his way home.