Why was the Soviet Union unable to stop the violence in Nagorno-Karabakh?

The same reason the Soviet interventions in Afghanistan or Chechnya have been unsuccessfull:

1. The Red Army was in a very, very, very poor state. Morale was low(most of the soldiers were recruits - actually all of the soldiers up to sergeants were nothing but recruits, oftentimes from Central Asian Republics, who had little to no desire to fight for the Russians, especially post-Afghanistan), equipment was getting old, corruption was rampant, funds were low.

2. The Soviet Union was already falling apart.

3. Terrain. The Caucasus region is mountainous(named after the Caucasus Mountains), therefore permits and encourages asymmetric warfare. As both the Soviets and the Americans saw, Afghanistan poses the same problems - it's bloody near impossible to defeat all the opposition, who knew the terrain better, and could appear at any time, make a surprising attack(lowering the morale even more), and then dissappear back into the mountains.

And, of course, the Nagorno-Karabakh conflict doesn't seem to have been among the top priorities of the Politburo at the time, who had plenty of other problems(economic, political, military, etc.).

Because the goal of the Soviet Russia was completely the opposite.

The whole reason of creation of Karabakh, Zangezur, Nakhchivan, and many others was to put a "ticking bomb" under every country occupied by the Russia, including both Armenia and Azerbaijan.
So that in a case of any national-liberation attempt, the bomb has to trigger a civil war.

During centuries, Russia (all flavors of it) has deliberately ignited national tensions amongst the nations who supposed to be controlled by the Russia.

The Soviet Russia wasn't an exception: genocide and massive relocations led to creation of artificial national exclaves — in virtually every country occupied by the Russia.

Karabakh has been a disputed territory between various states for centuries. When the Russia has re-captured both Armenia and Azerbaijan in 1923, they kept the high ratio of Armenian population, despite the Nagorno-Karabakh "Autonomous Oblast" (NKAO) formally belonged to Azerbaijan.

As soon as the Soviet Russia weakened in 1987, a menace of national-liberation revolt has increased. The top leaders of Azerbaijan, including Heidar Aliyev, the first secretary of the Communist Party of Azerbaijan and one of the richest Soviet leaders, have been powerful enough to quickly liberate their nations from the Russian occupation.
The same applies to Armenian leaders, although they haven't been as wealthy as Azerbaijan.

The "bomb" has exploded immediately: In February 1988 the marionette "Supreme Soviet" of the NKAO has voted to unite itself with the Armenia, which, in turn, has triggered Armenia-Azerbaijan war that doesn't actually stop since then.

The Russian "peacekeeping" forces did only support both Armenian and Azerbaijan armies with weapons. They were ready to strike in case if any of the fighting party was close to victory.

Again, the Russia doesn't need anyone to win. Instead, it needs the war in the area.

Here's an article by Armenia-affiliated analyst:

The main goal of Putin’s policy is to preclude the political influence and activity of Western powers in unsettled conflicts like Nagorno-Karabakh between Armenia and Azerbaijan.

Russia is continually undermining Armenia’s security system, propagandizing that the future of the people and the state depends on Russia’s will.

Russia promotes a policy of continuous intimidation of Armenia by openly militarizing Azerbaijan and supporting the modernization of its military power. Moreover, from the political aspect Russia adheres to Azerbaijani positions, simultaneously, opposing Azerbaijan to the United States on the Nagorno-Karabakh issue.

The known military contract between Russia and Azerbaijan, in which Russia is going to deliver until the end of 2014 offensive weaponry costing $5 billion
— "Russia’s Main Strategy for the Nagorno-Karabakh Issue", The Foreign Policy

…And here's an opposite point of view:

The defense pact is actually an upgrading of a 1995 treaty allowing Russian ground and air forces access to a base in the west of the country. It expands the Russian mission from protecting only the interests of the Russian Federation, to also ensuring the security of the Republic of Armenia.

Under the pact, Moscow will also supply Yerevan with modern weapons and "special" military hardware. The existing base houses MiG-29 fighter jets and S-300 missile-defense systems, as well as troops.
— "Russia, Armenia Sign Extended Defense Pact", Radio Liberty

Apparently, both points are mostly true.

My personal note: I have friends in both countries, including those who were born in Karabakh. Unfortunately, both Armenians and Azerbaijani citizens see only a half of the picture, trying to convince me that the Russia is militarizing their opponent (only), but it "will not leave us alone in a trouble". This may be the reason why the Russia hasn't yet been legally accused of ignition the Karabakh conflict.

If you speak Russian, here's some great political analysts for further reading. Some of their works are translated to English, but not every.

• Garry Kasparov, a Russian opposition leader, an Armenian national who was born in Azerbaijan, a former World Chess Champion, has written a series of articles about Armenia-Azerbaijan relations. There's the excerpt of his point:

  …this conflict is now also being used by the authorities of Azerbaijan and Armenia to strengthen their position and wealth. (ArmInfo)

• Yuri Felshtinsky in his works "Blowing up Russia" and "The legal foundations of the immigration and emigration policy of the USSR" discusses a lot about the national policy of the Russia against the conquered nations.