Did it make sense to bring along infantry to reinforce attacking cavalry before the introduction of firearms?

At the Battle of Pharsalus, Caesar used infantry to support his cavalry when faced with superior numbers:

Caesar himself took his post opposite Pompey. At the same time, fearing, from the disposition of the enemy which we have previously mentioned, lest his right wing might be surrounded by their numerous cavalry, he rapidly drafted a single cohort from each of the legions composing the third line, formed of them a fourth line, and opposed them to Pompey's cavalry, and, acquainting them with his wishes, admonished them that the success of that day depended on their courage.

Pompey tried to delay the infantry clash to give time for the cavalry to work. This worked somewhat, but Caesar's tactic took them by surprise.

At the same time Pompey's cavalry, according to their orders, rushed out at once from his left wing, and his whole host of archers poured after them. Our cavalry did not withstand their charge, but gave ground a little, upon which Pompey's horse pressed them more vigorously, and began to file off in troops, and flank our army.

When Caesar perceived this, he gave the signal to his fourth line, which he had formed of the six cohorts.note They instantly rushed forward and charged Pompey's horse with such fury, that not a man of them stood; but all wheeling about, not only quitted their post, but galloped forward to seek a refuge in the highest mountains. By their retreat the archers and slingers, being left destitute and defenseless, were all cut to pieces. The cohorts, pursuing their success, wheeled about upon Pompey's left wing, while his infantry still continued to make battle, and attacked them in the rear.

Once this happened, the superior Caesarian Infantry made short work of their opponents.

Normally, you always have infantry and cavalry combined if you can. For example, even in the time of Alexander the Great the phalanx was always used in combination with cavalry units.

In modern times infantry always accompanies tanks. If you advance tanks without supporting infantry they will be vulnerable to various kinds of attacks.

The same was true in the Civil War. You could, of course, always raid with cavalry on its own, but that was a risky move because if they got trapped or scattered they could suffer serious losses. Cavalry charges were rare in the Civil War because the wide availability of nails and iron shovels allowed infantry to build up horse-proof defenses and rifles on the ground are more powerful than sabers in the saddle. For this reason, cavalry units often dismounted and got out their rifles before attacking.

If you are considering a combination of infantry and cavalry against other cavalry, look at the Battle of Crecy during the Middle Ages, England against France.

The French had a total numerical advantage and also a numerical advantage in cavalry numbers. And the French lost, because the English cavalry all dismounted, helped bog down French cavalry. While English archers with longbows effectively and decisively ended the battle. (As long as you consider archers with longbows to be non-firearm carrying infantry.)

References: http://en.wikipedia.org/wiki/Battle_of_Crécy

It depends on the circumstances. Equipment, training, tactics, situation (e.g. pitched battle or raiding), terrain and weather. It also depends on what you mean. I think tactically there is a significant difference in the situation with armored knights as opposed to light cavalry. For the strategic decision of what forces to bring to a major set-piece battle, generally one would and did bring whomever one practically could. Even so, common infantry would not generally become equivalent to heavy elite cavalry, as they did in a 19th Century situation when they dismounted and fought with rifles or carbines.

So I would tend to say mostly yes, particularly for most examples where a major battle was occurring. In that case, having more men and more types of forces is likely to be an advantage. In particular, with most ancient cavalry which usually dismounts to fight like infantry anyway, there is a clear tactical parallel.

I would tend to say no for a force of mounted medieval knights operating independently for raids and smaller-scale operations, because they do fight while mounted, generally have superior armor, training, and are considered by both sides to be the strongest type of combatant, and aren't generally countered by most foot units. Trying to coordinate with a foot unit outside a major battle would mean the cavalry need to forfeit most of their mobility advantage. However, your example of using infantry to block a pass would in some situations be a great complement to an independent cavalry force, since they would not need to move together and places with restricted access could be defended by the foot units, particularly against forces which might pursue of flee the cavalry unit.

At Alesia, I definitely think an infantry assault was called for, due to the extensive walls and earthworks encircling the city. Superior numbers of infantry could in theory have overwhelmed the walls and been too much for the German Cavalry to stop.

Can infantry be used profitably to "flush out" enemy cavalry from defensive ground such as trees and bushes, and into the open field?

Yes, if your infantry can out-fight the cavalry, say due to superior numbers and being a match for the cavalry. If the cavalry is similar in numbers or better individually, they may just stay and defeat the infantry. I'm not sure why cavalry would tend to choose to stay in trees / bush terrain though, as it's not generally not good mounted terrain and staying put removes their mobility.

Can infantry be used as a rear guard against enemy cavalry while your cavalry regroups for a counterattack?

Yes. Generally foot have the advantages of tending to be more numerous, requiring less supplies, and so being good to win pitched battles, hold ground and protect supply trains. The combat advantage infantry have over cavalry tends to be because there tend to be significantly more of them.

Both Union and Confederate cavalry in the civil war fought almost exclusively as dragoons, using their mounts only as transportation, except in pursuit, while scouting and while engaging against enemy cavalry.

See here:

When charged by Union cavalry, a Southern general said his men would respond with the cry; "Boys, here are those fools coming again with their sabers; give it to them."

Update:
Also this comment by Baron Du Pire, in his correspondence of 14th April, 1809, to General Friant (my own translation, from Saski Vol. II, page 157):

You will also note from my report, my General, the disadvantage of a regiment which arrives after a long and difficult march; one is sadly obliged to withdraw in the middle of an affair due to the exhaustion of the horses. I thank you for the bayonets, the combination of two arms has always been recognized as a precious one by better advance-guard officers, since it is the infantry that allows the horses to rest, so they can perform a better charge the next day.

Update 2:
To sensibly address this question it is important to recognize the key distinguishing features of cavalry and infantry relative to each other.

Cavalry is distinguished by its speed and mobility on the battlefield, and its battlefield effectiveness stems directly from an ability to convert that mobility into increased shock at the point of contact with an enemy, causing a morale failure by the opponent. In this sense, cavalry that is stationary increases its vulnerability, and loses its effectiveness.

Infantry is distinguished by its ability to dig-in and fortify on the battlefield, whether by stone, spade, sarissa, shield-wall, or some combination of these. Its effectiveness stems from an ability to induce a morale failure in the opponent by its very immobility. In this sense infantry that is moving increases its vulnerability and loses its effectiveness.

A combined-arms team of infantry and cavalry - whether Roman shield-wall with Germanic tribesman mounted on Gaulic horse; sarissa-armed phalanx with mounted immortals; or modern firebase with Blackhawk-mounted assault teams and tank support - is designed to crush an opponent between the hammer and the anvil.