What are the main reasons historians give for the Soviet Union's collapse?

The Soviet Union had some core legal vulnerabilities, which were exposed during the Gorbachev era:

• The political structure was built as a series of nesting dolls, to allow the nth-generation replacements for the original revolutionary conspiracy to control a huge empire. A dictator manipulated a shifting coalition within the Politburo, which controlled the Central Committee, which controlled the Communist Party, which controlled the Russian SFSR, which controlled the Soviet Union, which controlled the Warsaw Pact.

• In particular, the Russian SFSR had about half of the population of the Soviet Union, and a majority of the industry. If Russia had had a distinct Communist party (like each of the other SSRs had), then it would have been obvious that Supreme Soviet votes could always be decided by the results of a caucus vote by the Russian Communist party. To prevent this, the RSFSR was the only SSR that did not have a Communist Party of its own. Russians could only be members of the Communist Party of the Soviet Union, whereas people in the other SSRs could be members of both their SSR's Communist Party and the Communist Party of the Soviet Union. This resulted in a bad morale situation within the Communist Party: Non-Russians resented the dominance of "Great Russians", and Russians resented not having a Communist Party of their own.

• In theory, the Soviet Union was a "Democratic Socialist Republic" with a liberal constitution that always happened to elect representatives from a minority party who supported a totalitarian government. If the government ever allowed free-and-fair elections with bottom-up candidate selection processes, that was unlikely to continue.

• The Soviet constitution explicitly allowed SSRs to secede. It did not have any provisions for allocating the national debt to seceding SSRs, nor did it establish any procedures for how an SSR could secede.

• Despite being a nuclear-armed superpower, the Soviet Union had not obtained recognition from many major powers for its annexations of Latvia, Lithuania, and Estonia during World War II. This meant that if any of these three SSRs managed to even come close to controlling their own territory, Western Powers were willing to recognize their independence.

• The people of Latvia, Lithuania, and Estonia were unaware that their foreign policy situation was different from that of the rest of the Soviet Union. The Soviet Union wanted to firmly establish that the U.S. de facto recognized its control of these three SSRs. So in 1986, it made a major blunder. It invited the U.S.' senior Soviet affairs official to a political debate in Latvia, and invited thousands of influential people from the Baltic SSRs and the rest of the Soviet Union. The "speeches were fully covered by the local media and in more abbreviated form by Moscow's 'central' media." The Soviet Union expected this meeting to demonstrate that U.S. recognized Soviet control of the Baltics. Instead, the envoy started his speech in Latvian. After switching to Russian, he "made it clear that the U.S. government had never recognized [the] illegal seizure [of the Baltic States] and would continue to insist that only the people of Latvia, Lithuania, and Estonia had the right to determine whether they wished to be independent or part of a larger union." On that day, the men who would lead the Baltics' independence movements "learned that [they] were not alone." (These quotes are from Autopsy on an Empire: The American Ambassador's Account of the Collapse of the Soviet Union, by Jack Matlock.)

• Because the Soviet constitution did not distinguish between the status of the Baltic SSRs and the other SSRs, there was no domestic legal firewall to prevent the loss of the Baltics from cascading to the Caucasus SSRs, the Ukraine, and finally Russia itself.

As a Russian who was raised in Russia, I would like to add one theory that is quite popular in Russia, but is apparently almost unknown in the West. The theory attempts to address the root cause of the collapse of the USSR and to explain why communism is fundamentally faulty.

In short, the theory says that communism gives rise to "отрицательная селекция" (literally "negative selection"). This term is not about genetics, but rather about views, beliefs, principles, and life strategies of individual people, as people tend to pass these things to next generations.

According to the theory, the fundamental problem of the communism is that it does not encourage or reward talented and initiative people. In a capitalist state, the better you work or the more talented you are, the more money you earn, but it does not work like that in a communist state. To get a well-paid job and a nice apartment in a communist state, you have to develop some special skills, e.g., the ability to bribe, the ability to develop "special" social connections to trick the system, knowledge of all subtleties and nuances of how to get things from the state (e.g., an apartment or a promotion), the ability to properly complain about the boss to take his position, the ability to pass tasks given to you at work to others, the ability to pretend to be a hardcore communist, and so on. My own parents say that they gave birth to two children because this was needed in order to get a nice apartment from the state. A Soviet motto was, "От каждого по способностям, каждому по потребностям" ("Taking from everyone according to his abilities and giving to everyone according to his needs"). At work, people were not motivated to do their best; they tended to make minimal effort that was just sufficient to avoid punitive actions. There was simply no way of legally making a lot of money, because salaries were strictly regulated by the state.

In just a few generations, this system produced a population of lazy people focused on tricking the system, because people simply inherited views and beliefs of their parents, who succeeded in tricking the system and managed to save a lot of effort by finding a way not to work hard. There is even a special slang in modern Russia to refer to such people: совок.

And with such a population, just any economic system will fail. This explains why all post-communist countries still struggle economically despite that almost three decades have elapsed since the collapse of the communist system in Russia and Eastern Europe. I lived in Germany and heard from Germans that there is a big difference in the mentality between Western Germans and Eastern Germans. A Russian entrepreneur told me that it is still hard in Russia to find good reliable workers capable of working really hard and motivated to build a good career by hard work.

In short, the theory says that communism does not work in the long run, because it does not provide a proper environment for natural selection of individual people's views and life strategies that bring benefit to the society as a whole.

UPDATE: Following a comment below, I am adding links for further reading:

• A book of 1922 by Prof. Sorokin, who is considered to have coined the term "negative selection." The book is in Russian.

• A random article about the "negative selection" in the USSR (in Russian).

• A book chapter about the "negative selection" in the USSR (in Russian).

• A post by the Russian opposition leader Alexey Navalny (in Russian).

• An article in English about Russia's work ethics. I quote from there:

For three generations a negative selection process systematically weeded out workers of the greatest drive, know-how and resilience, giving rise to a pervasive, cowed apathy and scheming work ethic, with the liveliest initiatives directed at seeking maximum personal gain with a minimum expenditure of effort.

The negative selection theory is very popular in Russia because it resonates with what people personally experienced in the communist regime. My parents and many other people who worked in the Soviet era just feel that the theory hits bull's eye.

UPDATE 2: To respond to another comment, which says that people seek maximum personal gain in capitalist states as well, I would like to emphasize a principal difference between capitalism and communism. In a capitalist state, individual people's strategies maximizing personal gain also bring great benefit to the society. And it is not the case in a communist state. To maximize your personal gain in a communist state, you have to learn how to trick the system. It is "social chess," so to speak. This "social chess" is a zero-sum game and brings no benefit to the society as a whole.

Let me explain this figuratively. Imagine a village or a tribe in which all people jointly work and then distribute the products of their joint work by playing some game, so the personal gain of everyone is determined not by his work skill and work effort, but by his skills in the game. In a few generations you will get a population of skillful players, not a population of good workers. Figuratively speaking, this is what communism and negative selection are.

The communist system inherently does not allow properly rewarding workers according to the value they produce, because in the absence of a free market economy you cannot tell the value each worker and each factory produces. This is a fundamental flaw of communism, and that flaw inevitably gives rise to negative selection. In such a system, rewards for individual workers are determined not by market mechanisms, but by something else. In the USSR, the salary of each worker was determined by a rigid formula that included the working experience, occupation, role, region, etc., so people were not motivated to work as hard as they could. They made minimum efforts just to keep their jobs. And people played complicated social games to get well-paid roles and benefits from the state (e.g., apartments) and to reduce their actual workload and responsibilities. In a capitalist state, in contrast, business owners try to keep valuable workers by offering them good salaries, and this motivates workers to work hard. There was simply no such mechanism in the USSR.

UPDATE 3: To further illustrate the theory, I would like to point to North Korea and South Korea. Their populations are very different. North Koreans and South Koreans have very different mentalities, different work ethics, etc. Here is how South Koreans view North Korean refugees:

“There is a stereotype some South Koreans have of defectors as lacking an entrepreneurial, pull-yourself-up-by-your-bootstraps spirit, being from a socialist country and therefore relying on government benefits,” said Sokeel Park, director of research and strategy for non-profit Liberty in North Korea. (link).

North Korea initially did quite well economically, and it is only in a couple generations that serious economic issues started hitting the county. The North Korean famine occurred in the 1990s, about half a century after the formation of North Korea. This is consistent with the negative selection theory, which says that communism does not work in the long run.

Even if you magically change the political system in North Korea and instantly build modern infrastructure there, the country will still be bound to suffer decades of economic hardship, because it takes a long time to change the population. The GDP per capita of each post-communist country is still a few times lower than that of Western countries.

Let's consider two neighboring countries: Albania and Greece. Albania is a post-communist country, while Greece isn't. Greece's GDP per capita is about 20,000 USD, and Albania's - 5,000 USD. Another neighboring post-communist country, Serbia, has a GDP of 6,000 USD. Albania and Serbia/Yugoslavia ceased to be communist countries decades ago, but the effects of the negative selection are still there.

To add an alternative explanation to @evilwashingmachine 's excellent list of explanations, I would append:

Abandonment of coercion

Coercion, that is, coercion of internal dissent and coercion of subject states , was an inherent and vital part of the Soviet system, as it is all dictatorships. Remove the coercion, as occurred in the 1980s, and the system's collapse becomes much more likely if not inevitable.

Removal of coercion as an element of foreign policy

Soviet foreign policy before 1980 was dominated by the so-called Brezhnev doctrine, which essentially said the Soviets would intervene militarily rather than allow a country to fall out of the communist orbit. This doctrine was enforced until the early 1980s, when the Soviet leadership refused to intervene militarily in Poland to ensure the survival of the Polish communist state against domestic unrest. The Polish communist state survived that unrest but the Soviet government was already moving towards a full abandonment of the Brezhnev doctrine.

The Soviet defeat in the Aghan War only compounded this; the much vaunted Red Army was defeated by the Afghan insurgency and this reduced confidence in the Red Army as an instrument of imposing government will.

By the late 80's Gorbechev had effectively killed the Brezhnev Doctrine, just as the tide of nationalist dissidence and economic malaise was reaching its most acute stage.

This in turn guaranteed the collapse of the Warsaw Pact communist governments, a disastrous turn of events for the survival of the Soviet Union itself for a number of reasons:

• Nationalist revolutions in the Pact inspired Nationalist revolutions inside the USSR itself (similar to how revolutions in the Arab Spring inspired further revolutions)
• The Soviet Union depended on these states for imports and exports, as alluded to by Evil Washing Machine in his section on economic decline. The removal of these states from the Soviet orbit further threatened the stability of the Soviet economy.

Removal of coercion as an element of domestic policy

Gorbachev, as is well known, was unwilling to leave the hard line attitude towards internal dissent in place. Glasnost and Perestroika made criticism of the regime much more tolerated, and this emboldened people across the various member republics to do exactly that. If know that they are going to be able to push the limits of criticism, they are going to do exactly that. Inspired by thee aforementioned unrest elsewhere, economic hardship, revolutions in the Warsaw Pact, the soviet failure in Afghanistan, etc. elements of Soviet society at the top and at the bottom, in Russia and the other member republics, began agitating for serious change and the entire Union was destabilized.

Now, it's noteworthy to point out that many of the Soviet member states actually voted to maintain a reformed version of the Soviet Union so there might have been a path to a renewed USSR if the right path of reform had been followed...but the path was so narrow as to be practically non existent. IMHO, realistically the Soviet state was doomed to fail the moment its leaders refused to coerce their own people and their subject states, as that made any serious crisis IE nationalist uprisings effectively unstoppable.