score:6
The Schengen rules state that you should apply to the country that is your main destination for the trip. This is usually the Schengen area country where you spend the most time. However, if no main destination can be determined, you must apply at the country where you will first enter the Schengen area.
Because your itinerary is an intra-Schengen transit, and you aren't spending significantly more time in either country, there is no main destination.
So, you should apply to Iceland for a multiple entry Schengen visa covering your entire itinerary, both outgoing and return.
(However, note well that if there is some other reason you chose this strange itinerary, that could affect which country is the main destination. For instance, if you wanted to go sightseeing in Iceland on your long layover, or meeting friends in Helsinki for a long night of vodka, that would make that country the main destination.)
On this itinerary, you will make two entries into the Schengen area.
Your first entry will be in Reykjavik, where you will clear immigration into the Schengen area. Your flight to Helsinki will be an intra-Schengen flight, and you won't clear immigration again (but you will clear customs in Helsinki if you checked your baggage through). You will then exit the Schengen area at the Russian border.
Your second entry to the Schengen area will again be on the train at the Finland/Russia border, where you will clear immigration and customs. You then fly to Iceland as an intra-Schengen flight, and clear out of the Schengen area at Reykjavik airport when you get on your flight back to the US.
P.S. If your visa is refused for some reason, there are plenty of non-stop flights from the US to SVO or DME.