score:40
Lithuania fell under Soviet occupation in 1940, but the Soviets were promptly evicted by Operation Barbarossa. So in practice, the Lithuanian SSR was established only in late 1944, after the Baltic Offensive. It would have been stranger for Moscow to entrust East Prussia to this fledgling government, who would've had their hands full setting up shop in Lithuania.
For reference, Memel was later returned to Lithuania in 1948. So the question is probably, why not transfer the rest of the exclave after the dusts have settled? Well, as you noted in the question, Crimea was transferred to Ukraine only in 1954, under Khrushchev.
What you might not be aware of is that a similar offer was made to give Kaliningrad to Lithuania. However, the Lithuanian leader Antanas Sniečkus had the foresight to decline. By then, the German population had been expelled and replaced with Russian colonists. The Lithuanians wisely refused to adopt a large, and potentially problematic, minority.
See for instance, the troubles faced by Ukraine today.
And in private, Lithuanian officials assert "what would we do with close to one million Russian speakers?" Recall that during the Soviet Era, Lithuanian Communists loyal to Moscow rejected Khruschev's offer to place Kaliningrad under Lithuanian authority for the same reason.
Krickus, Richard J. The Kaliningrad Question. Rowman & Littlefield, 2002.