Why were rockets not popular as part of artillery until 20th century?

Rockets were much less efficient in converting the chemical energy of black powder to kinetic energy. In earlier times the chemical components of gunpowder were in scarce supply and were considered a strategic resource to be conserved.

Actually, rockets are very different from artillery, even though they do somewhat the same things ("rain" death and destruction on enemies) from the air.

Artillery fired out of "guns" is relatively accurate. That is, it can be counted on to hit a target a few miles away. A "rocket" couldn't be counted on to hit a specified target at ANY distance for most of the 19th and 20th centuries. (Maybe that's different in the 21st century but the question doesn't cover that time frame.)

The advantage of rockets is that they can be fired form ultra long ranges (thousands of miles through earth and space) with an accuracy in that order of magnitude.

There is a saying that if the Chinese fired their intercontinental ballistic missiles at Los Angeles, they might end up in Arizona. Even so, it illustrates the point that rockets are "good" weapons compared to others fired from halfway around the world, if the intended target is "anywhere in the United States."

(Note to commenter: U.S. ICBM's are quite a bit more accurate than the Chinese ICBM's mentioned above.)

One major use of artillery was against fortifications / castles etc.. Actually that was the most important role of artillery in the 15-17th century, and had little importance in open battles. Brass cannons were major investments, but they became the main tool of siege, like e.g. in the case of Constantinople in 1453. To destroy thick walls, projected stone cannon balls and similar are just more effective than the missiles/rockets. Similarly, cannons used on ships could effectively penetrate and destroy the hull of the opponent ship, while rockets are much less effective.

Quick Answer-

Early Rockets were: Inaccurate, Immobile, Ineffective.

They were used before the XX century (eg. Napoleon) but very scarcely for the reasons above.

The Chinese and Koreans on the other hand did use rockets before that time. For an example the ancient Korean's launchers were very effective and even managed to stop a Japanese samurai invasion.

I am going to suggest the following simple point which is certainly mentioned above: A rocket cannot, without a guidance system, be as accurate as a cannon which can be aimed iteratively. If you fire a cannon and it misses, the place where the shot landed can be used to adjust the next shot. Rockets can't really be aimed this way because of inconsistencies in flight caused by various factors.

To quote from Manual of Gunnery for Her Majesty's Fleet (1880):

War Rockets

This subject is at present under the consideration of a committee, the results obtained with Hale's rockets being considered most unsatisfactory.

At present the 24-pr. rocket manufactured is Mark III., the later patterns having failed to meet the requirements of service.

Advantages. Useful for incendiary purposes, and would be still more so if the head were fitted with a shell or carcass.

Its moral effect is great, especially against savages and cavalry, and owing to the lightness of the apparatus required for firing, it can be transported easily in rough country.

Disadvantages. The disadvantages under which it labours are,—

1st. Its efficiency and safety depend on the complete contact of a very large service of composition with a thin metal case; hence it is liable to deteriorate.

2nd. Its flight is necessarily very slow, so that it is very susceptible to the action of gravity, wind, and accidental causes of deviation.

3rd. The same causes that thus make it peculiarly liable to be acted on by wind and gravity, aggravate the effect of deflection; indeed rockets have been occasionally deflected so as to come back at the people who fired them.

4th. From the fact of the composition burning away during flight, the position of the centre of gravity is constantly changing. Cause of The motion of the rocket depends on the well-known law '"" that " action and reaction are equal and opposite."

There you go from the horse's mouth. Rockets are convenient due to lightness (no need to carry a cannon around) and are primarily effective against less sophisticated forces in rough country. Otherwise guns are better, due to accuracy.

The premise is a bit off. Because actually, rocket artillery did become somewhat popular before the 20th century. Rockets were used to great effect in India, by the Kingdom of Mysore against forces of the British East India Company. The British in turn learnt from the Indians and developed their own rocket weaponry which went on to feature in the Napoleonic Wars:

It is not generally known that rocket artillery played a considerable part in the military and naval history of the 18th and early 19th centuries in the Western world. The city of Copenhagen was set on fire by rockets from the British navy during the Napoleonic wars, and rocket troops were prominent in the days of the so-called Honourable East India Company, contending with princes like Tippoo Sahib.

_{- Needham, Joseph. Science in Traditional China: A Comparative Perspective. Chinese University Press, 1981.}

Rockets became sufficiently popular during this period, that dedicated rocket troops were formed. Modern British Army Units such as the O Battery traces their lineage back to the Rocket Troops who fought at Waterloo and earlier. They weren't just limited to land either; British naval rockets actually helped inspire America's national anthem:

By the 19th century, Britain's Royal Navy had a squadron of warships equipped with rocket artillery. One of these so-called "rocket ships" bombarded America's Fort McHenry during the War of 1812.

_{- Vulpetti, Giovanni, Les Johnson, and Greg Matloff. Solar Sails: a Novel approach to Interplanetary Travel. Springer, 2009.}

So why did rocket artillery go out of fashion? As @CsBalazsHungary correctly observed, the problem was that rockets were inaccurate. Exacerbating this weakness is that during the 18th century, conventional artillery steadily advanced. Rockets were left in the dust as canons became more accurate and longer ranged, and smaller and lighter. As a result, rocket artillery became essentially abandoned by western armies by the time of the American Civil War. Artillery were far superior by World War One, and rockets didn't begin catching up until the interwar period.

This was however, a short-lived phase, for high-explosive shells and incendiary shells could be fired from more advanced artillery with much greater accuracy of aim, and the rocket batteries of the West died out after about 1850.

_{- Needham, Joseph. Science in Traditional China: A Comparative Perspective. Chinese University Press, 1981.}