Why was gold so popular and valuable back when it did not have many real world uses?

I am going to suggest that its extreme heaviness was a major feature contributing to its value. If you sell something by weight then the heavier it is the easier a given weight is to carry especially if you want to conceal it. Of course, its rarity and physical qualities are major contributors also.

To add to @Mark's answer, we may venture that in early times, gold was both rare and amenable to be crafted into jewels that thus displayed the wealth of their owner. As such, gold is a key to an elevated social status, making it highly desirable everywhere. Gold was not the first metal to serve in that role; e.g. some late neolithic tombs have yielded copper axes that were too small and soft to have been used for any purpose other than decorative.

A similar pattern was found among Aztecs, who valued Quetzal feathers a lot, especially since they could be obtained only from the most western areas of the Aztec influence zone; these provinces had to pay a yearly tribute of Quetzal feathers. Besides being somewhat pretty, the only use for Quetzal feathers in Tenochtitlan was to show that you could afford Quetzal feathers.

Quetzals live only in central America; gold can be found in many places, but always in small quantities and requiring a lot of work to extract. Marx remarked that gold is very appropriate for use as a currency, especially for hoarding, since it is very durable, compact, easy to store, amenable to essentially lossless division and reassembly, unlikely to lose its value over long period of times (since the influx from mining is small), and otherwise completely useless(*). Not many materials have all these properties, and gold has them everywhere it can be found. Furthermore, though hard to obtain, gold can still be mined and handled with rudimentary technology. This may go a long way towards explaining the universality of the value of gold.

This leads us to an historical theory of gold's value: when a society becomes technologically advanced enough to mine gold and use it to craft jewelry, gold quasi-automatically becomes a powerful marker of the owner's place in the society, thanks to its intrinsic qualities: durable (you can use it for continuous display for many years), compact (you can wear your wealth, contrary to, say, a herd of camels), recognizable (it is hard to make convincing fake gold),... only the richest can own gold jewels, and you want gold jewels because it makes you look rich(**). At a later time, when the notion of currency is invented, gold is then a likely candidate, strengthening its position as a repository of value.

(Marx, being himself, would not have accepted a theory of gold's value as a social status marker; in Marx's view, everything is a question of labour, so gold's value comes from the amount of labour that goes in its extraction and purification. But even this marxist theory brings us back to universality: everywhere on Earth, mining gold-rich ore and extracting the shiny metal from it requires a lot of efforts.)

(*) This is not entirely true. Nowadays, gold has some industrial uses, especially in electronics. Even in ancient times, gold was used for dentistry, as in this example found in an Egyptian mummy from around 2000 BC:

(found on this page)

Nevertheless, these non-decorative uses of gold have always been a small minority, when considering the amount of involved metal.

(**) The underlying idea in all this is that people who want gold do not, actually, want the metal for itself, but for what it can bring to them. Prescott explains it vividly in The Conquest of Peru: when Pizarro and his companions ransom Atahualpa, their main motivation for such a craving of gold is to be able to come back to Spain as grandees, to gain access to the highest planes of the highly hierarchical Spanish society.

From a practicality perspective; Gold shared the same an early advantages that Copper did for developing societies. It can be worked by being beaten and by being cast, so the technology requirements to begin working with gold is lower than even bronze. In it's natural state gold is malleable and shiny, making it an obvious candidate for cultures and artists that are in early stages of developing trade-able goods.

Even crude experiments in metallurgy would reveal gold's resistance to tarnish and the ease with which it can be reworked.

It wouldn't have been until smelting and mining developed that gold would have been known to be scarce. (At one time gold would have been FAR more available than Iron). So scarcity is a secondary characteristic in terms of ancient value.

In terms of extended value throughout history we can argue that the Cultural and Industrial values are extensions that emerge naturally from it's workability and the ease of reaching high degrees of purity.