Why did the USSR stop in the Winter War against Finland in 1940?

The relevant archives are still closed and, I think, will continue to be closed for a long time. One, thus, has to look for memoirs of former Soviet leaders (of that time). Three of them wrote such memoirs: Kaganovich, Mikoyan and Khrushchev. The first two say nothing of interest (regarding your question), but Khrushchev (in book 1) did have something to say on this matter.

The most relevant pages in Khrushchev's memoirs are 254-255. Here is what he writes:

Once again the question comes up: Could the Soviet-Finnish war have been avoided? I will not take it on myself to try to draw a definitive conclusion. If we are to talk about Stalin, on the other hand, the person who decided these questions, he did not begin the war in 1939 in order to seize Finland. We did not in fact try to occupy that country, and when the Finnish army was effectively defeated in 1944, Stalin demonstrated statesman like wisdom then, too. The territory of Finland with its population could not decide any fundamental questions of foreign policy for us. It was a small nationality, whose country was not very rich in natural resources; but the signing of an armistice between ourselves and Finland, which then declared war against Germany—that was a good example for other satellites of Hitler’s Germany to see. The advantage for us was greater than occupation. Besides, the step that we did take left positive traces as far as prospects for the future were concerned.

As everything written by a (former) Soviet official, this has to be taken with a (rather large) dose of salt. Here is my reading between the lines:

1. Initially, the Soviet leadership was expecting a "quick and victorious war". Stalin was more than happy to make Finland a "people's democracy" (see item 3 below), like Mongolia in 1920s or Eastern European countries in late 1940s. This is what Kuusinen's government was formed for and there is no other good explanation for its formation.

2. As we all know, things did not go well and by mid-March 1940 (after Red Army finally broke the Mannerheim line and took Vyborg) Stalin has decided that Finland was not worth further efforts. Khrushchev writes of strategic insignificance of Finland in the context of the 2nd world war, but the cited paragraph concludes his discussion of the Winter War of 1939-40, so, in my reading, applies to this war as well.

3. Lastly (but this is just my take, Khrushchev said nothing about this): Further conquest of Finland in 1940 would have taken more time and troops. Even after that, USSR would need a significant presence of Red Army and NKVD on Finnish land in order to pacify it. At the same time, USSR needed the regular troops and NKVD for the takeover of Baltic republics (and Bessarabia) in Summer of 1940, which were much more strategically significant. For the Baltic republics, at least initially, the original plan for Finland also served as a blueprint (per this wikipedia article):

In May 1940, the Soviets turned to the idea of direct military intervention, but still intended to rule through puppet regimes.[55] Their model was the Finnish Democratic Republic, a puppet regime set up by the Soviets on the first day of the Winter War.[56]

Edit. I found the following article

Kimmo Rentola, Intelligence and Stalin's Two Crucial Decisions in the Winter War, 1939–40, The International History Review, 35 (2013) 5, 1089-1112, DOI: 10.1080/07075332.2013.828637

The article itself is behind the paywall (I now have a pdf file). The abstract is below:

In 1939–40, the fierce Winter War was waged between the Soviet Union and Finland. This article analyses Stalin's two main decisions, to attack and to make peace, and the intelligence behind those decisions. Already at the outbreak, it was obvious that the attack was based on a serious misjudgment. The Soviets did not foresee that the action would become a real war, very different from the occupation of eastern Poland in September. It will be shown that this was due more to Stalin's miscalculation of consequences than to any major failure of intelligence collection. As to why Stalin made peace at the very moment when the Red Army finally began winning, and with the Finnish government he had declared non-existent, this seems to be connected with defective assessment of intelligence from London and Paris. Even the Cambridge Five were discarded. Both real and perceived threats of Allied intervention weighed heavily in Stalin's decisions, in particular the southern threat against Baku and the Caucasus. The analysis will contribute to scholarly discussion on Stalin's foreign policy and the role of intelligence in Soviet decision-making. New evidence is mainly provided by intelligence and security documents released by the Central Archives of the Russian Federal Security Service.

Given that the article is based on author's access to Soviet intelligence archives, I am inclined to believe his conclusion, even though it is different from mine (stated above).

With three collaborators, in 2017 Kimmo Rentola also published a book (in Finnish) based on the research described in the paper (all four authors are professional historians).

Below are some excerpts from Rentola's article:

The decision for peace.

In his diary, the Communist International Secretary General Georgi Dimitrov probably pinned down the very night Stalin changed his mind on the Finnish war. In the Bolshoi Theatre on 21 January 1940, Stalin explained the war to Politburo members. At first, his tone was confident. It has turned out, he said, that the issue was not mere Finland, as the Finns had supporters and were prepared and armed for a major war. Nevertheless, they would be crushed and their backbone officers and soldiers - 150,000 men - killed, he promised, toasting the restoration of the ‘somewhat tarnished glory’ of the Red Army. Then the atmosphere changed. Deputy Defence Commissar Georgi Kulik brought bad news, probably a military-intelligence report on the threat against Baku, since later that night Stalin said, as if as an afterthought: ‘It is not we but the Turks who stand to lose. We are even glad that we shall be free of certain ties of friendship with Turkey.’ At the theatre, Stalin accused Kulik of panicking and promised to send him a psychology textbook, to address his condition. A former student of theology, the Kremlin leader praised the wisdom of old Greek monks: when bad news arrived, they retreated to the sauna to think and wash away ‘the shit’ of false impressions. Then they would make up their minds, not on the basis of impressions and phantoms, but relying on basic facts...

The British saw signs that ‘popular discontent in Russia is spreading, particularly in Transcaucasia’, even under a ruthless regime. Sir Orme Sargent of the Foreign Office pushed the Ankara Embassy for Turkish intelligence, since ‘events are moving rapidly and it may shortly become a matter of even greater urgency than it is at present for us to be able to strike, if not directly, at any rate indirectly, at Soviet interests in the Caucasus’. For Stalin, the threat against Baku was enough. Molotov’s long answer to Maisky and through him, to Butler, was conciliatory, even humble, considering the propaganda treatment reserved for the British during the last six months. Rejecting as ‘ridiculous’ the idea that the Soviets could ally with Germany against the West, Molotov sought British mediation for the Finnish war, stressing that the issue (now again) was the security of Leningrad. The border should be drawn where Peter the Great got his border in 1721; that was much more than the Soviets demanded before the war, but much less than the entire country under a puppet government. In addition, they wanted Hanko and some islands, but to appease the British, the already conquered northern Petsamo with its Canadian-owned nickel mines would be returned to Finland. ‘Are you thinking about our mediation?’, an astonished Butler asked. His Majesty’s Government declined to help....

On 2 March, Britain and France informed Sweden and Norway of their intention to send reinforcements to Finland. The latter two refused the right of passage, not wanting a world war and the Germans on their soil. Stockholm informed Mannerheim about the announcement from Berlin: if Sweden allowed Allied transit, Germany would be bound to ‘join the game’. On 4 March, the Marshal learned from the British that the French Prime Minister Daladier’s last-minute promise of 50,000 troops in March was crudely exaggerated: the real figure would be much lower and in April at the earliest. Mannerheim concluded that Western assistance would be ‘too little and too late’. The government of Finland decided not to ask officially for Western aid, but to accept the Soviet conditions, to save the body of the country by cutting off one limb, as Tanner said.

NKVD intelligence in Stockholm was quite clear about the desperate situation of the Finns. In early March, they reported that the Finns would not endure three weeks, lacking heavy artillery and aeroplanes, the soldiers exhausted. Both Mannerheim and Tanner supported immediate peace, as did the whole people. However, there was ‘no revolutionary situation, either in the army or among the people’.

Stalin did not have those three weeks. The Red Army advance was too slow; he had to hurry ‘out of the danger zone’. On 5 March, Molotov recorded that ‘the danger of a foreign intervention is great.’ Voroshilov was sent to Caucasia to strengthen the defences. All the time, the NKVD Paris rezidentura bombarded the centre with alarming reports: huge numbers of aeroplanes already sent (176 from France, 164 from Britain, forty-four from the United States, and thirty-five from Italy); plans ready to send three divisions of troops to Finland proper, Sweden letting them through despite its apparent neutrality; twenty-thousand snow camouflage suits and 20,000 mountain rucksacks already distributed by the French Army economic administration; the exile Poles preparing to take part.

In this atmosphere of alarm, danger, and humiliation, Stalin committed a mass murder excessive even by his standards by ordering the execution of 25,700 Polish officer prisoners of war in the Katyn Forest and elsewhere. ‘Each of them is only waiting for liberation to get the chance to join actively in the struggle against the Soviet power,’ Beria argued. The Finnish war was not directly mentioned, but these were the days when Stalin had to abandon his wish to kill a similar group of Finns.