score:9
Confederate general Nathan Bedford Forrest famously said, "I git there first with the most men," often misquoted as "I git there firstest with the mostest." A Union cavalry general, John Buford, reportedly said, "A horse is just a form of transportation."
The main advantage of cavalry is that it could get to places, and at speeds and timetables not available to infantry. The disadvantage was that hors*m*n had to dismount to fight properly against infantry (in an era of rifles, especially "repeating" rifles introduced late in the war). Every fourth cavalryman held the horses of three others, so the fighting strength of the cavalry unit was effectively reduced by one fourth.
Confederates like Forrest understood that it was better to arrive first at a critical location with say, 2,000 men, of which only 1,500 could fight if it came to that. Especially when they were on a "raiding" mission, "hitting and running" would be much better than arriving with 2,000 infantry "later." The northern generals (other than southern-born Buford), were more concerned about the one-quarter reduction in fighting strength. A difference of philosophies leading to a difference in tactics.
This tactic was used less in subsequent wars as the spread of "repeating" rifles further increased the advantage of infantry over cavalry. Cavalry was most effective during the era of single-shot weapons such as muskets during which "contact" weapons such as lances or sabers were most effective.
Upvote:1
While I'm no expert, I believe the following to be the explanation for the "why calvary is so effective" part of the question.
Since ancient times up until WW1, the main goal of almost every battle had been to outmaneuver the enemy army (e.g. flanking them) and forcing them to retreat. When this would happen, the calvary (as it moves faster than infantry) would chase them and mow them down from behind. With no effective way of defending, the fleeing army would usually end up decimated. This also explains why ~80% of deaths in most battles of pre-WW1 era was during retreats and not actual battles. And also why Alexander the Great allegedly lost less than 1000 soldiers during his entire career - his army was never forced to retreat (and even if the number is likely overblown, he did indeed lose a surprisingly low number of soldiers).
As of WW1, heavy artillery came into play, allowing the retreating army to simply stop and bombard the calvary chasing after it, rendering it infinitely less useful.
Upvote:3
There are multiple questions and the accepted answer only addresses them partly:
Why was the North so slow to catch on? Why wasn't this tactic used more frequently during the war? Was this technique used more widely in subsequent wars?
1st question is partly adressed, but let's put apart the "mentality" thing which, especially in the USA of the Civil War, which had no big previous army so no previous "mentality", is not as strong as it is often said: People are not stupid and try to adopt what succeded against them.
Unionists could not copy fast Confederacy cavalry raids because they lacked horses and people knowing how to travel on horses for a long distance. But as soon as 1863, they had put their own cavalry forces in service as big units and used them against COnfederacy, with more and more success.
2nd question is based on false premises: cavalry raids were widely used when they could be effective: they were done between campaigns, when both enemies had no big forces to use against the raids. They were used as recons for enemy offensives.
3rd question addresses subsequent wars (in the world I suppose because USA did not know more wars): For Indian wars in the USA, yes cavalry raids were used but very differently: Often, Native forces were less numerous and could not assault successfully American railway station or military forts, as a raid of the Civil War would have done. The American used cavalry raids as retaliation (see Wounded Knee) or as tentative to confront Native warriors groups (see Custer at Little Big Horn).
In other wars, yes, cavalry raids were used: in colonial wars by the British and the French units, with little units. During 1870 war, it could not be used all the time because of the density of forces.