Why did Abraham Lincoln choose Andrew Johnson as a vice presidential running mate?

There were electoral and administrative reasons for Lincoln to balance the ticket with a Democrat. He did legitimately fear that he might lose the election to McClellan, and Lincoln would do what it took to win. Lincoln did not much respect McClellan at this point. Furthermore, the Democrats had endorsed a peace plank at their convention. Thus Lincoln was willing to balance his ticket with a Southern Democrat in order to ensure that the Democrat's didn't win the election outright and compromise the war effort and his war aims.*

There was also an administrative consideration. In 1864, it was time for Lincoln's administration to consider how to integrate the South back into the Union. The South had seceded in part because the Republicans (with exclusively Northern support) had been elected to the presidency. The Republicans knew that Reconstruction would be easier if they could claim some support from the South. Lincoln signaled his leniency and willingness to welcome the South back into the fold by (temporarily) renaming his party the "National Unity Party" and selecting a Southern Democrat as a running mate.

Why Johnson? Well, he was from Tennessee for one thing. Lincoln was beginning to undertake the wartime reconstruction of occupied southern states, including Tennessee. If Tennessee were smoothly integrated back into the Union, it would provide Southern states with evidence of Lincoln's sincere lack of interest in retribution. From this perspective, Johnson's selection is an extension of the rhetorical approach Lincoln took in his Second Inaugural ("With malice toward none...").

Also, Johnson seemed like a good choice on paper. He was experienced:

In terms of sheer political experience, few men have seemed more qualified for the Presidency than Andrew Johnson. Beginning as a Greenville, Tennessee, alderman in 1829, he held office almost continuously, rising to the state legislature and Congress , serving two terms as governor, and in 1857 entering the Senate. Even more than Lincoln, Johnson gloried in the role of tribune of the common man. Both in Washington and the rough -and-tumble world of Tennessee stump speaking, his speeches lauded “honest yeomen” and thundered against the “slaveocracy”— a “pampered, bloated, corrupted aristocracy.”

He was tough on rebellion (though this was not an unambiguous positive):

After the Union capture of Nashville early in February 1862, Lincoln appointed Andrew Johnson military governor. Johnson’s decision to remain in the Senate after Tennessee seceded had made him a national symbol of what both he and the Republican North supposed to be a legion of courageous Southern Unionists. Having risen to prominence in the Democratic party, Johnson now found himself heading a Union movement composed largely of former Whigs. He quickly won their admiration and outraged secessionists by removing Nashville’s mayor and city council for refusing to take an oath of allegiance. When, shortly thereafter, a supporter of the Confederacy won election as circuit judge, Johnson ordered his arrest and appointed his rival to the office. Johnson soon took to using the phrase that won him a national reputation for Radicalism: “Treason must be made odious and traitors punished.”

But Johnson turned out to have no political touch. Needless to say, history might have been pretty different if Lincoln had chosen someone else.

* It should be noted that the parties weren't always so concerned with ideological congruence between candidates. Every Whig to win an election died in office and was replaced with someone ideological dissimilar: William Henry Harrison and John Tyler; Zachary Taylor and Millard Fillmore. And if you count Lincoln (a former Whig), that's 3 for 3. It's an incredible streak of bad luck.

Source: Eric Foner's Reconstruction