When travelling overland in Russia I hear you must register with the police in each new place, but does this cost money?

Depends which country you from, but if you have a visitor visa and if you going to stay longer than 7 days, you have to be registered within 7 working days after arrival to each new place of your stay in Russia!

Official source (Russian): http://www.fms.gov.ru/document/3834

С момента прибытия в место пребывания на территории Российской Федерации иностранный гражданин в течение семи рабочих дней обязан встать в территориальном органе Федеральной миграционной службы на учет по месту пребывания.

Which means that foreigner within 7 working days have to be registered by a local federal migration service.

Non-official source (English with explanation): http://waytorussia.net/RussianVisa/Registration.html

Basically it’s not you who should think about it. (You can’t register yourself unless you know Russian and have documents from your host.)
If you get a tourist voucher and you stay in hotel then hotel will register you. If you invited by a company or by a host then they do it for you.
If you bought a tourist voucher online then the same company can usually register you, sometimes for free, sometimes you have to pay extra.

HOTELS

If you stay in hotels, hotels have to register you automatically for free within 3 days after your arrival.

HOST

If you stay with a host then host must register you within 7 days.
Host can do it for Free in a “federal migration service”: 1. wait in a queue, 2. fill the form, 3. leave a copy of his/her passport and yours (passport copy, visa copy, migration card).
Your host can also do it at any post office, but post office will take a fee like 2-3 EUR.

By the way, your host must have “propiska” (registration) in this apartment/house, where he/she hosting you.

More info for host (Official, but in Russian):

https://www.gosuslugi.ru/pgu/service/10000035899_.html#!_description

HOST without registration in your current region/renting

If your host is renting a flat/room then you can use an alternative method: you need to find someone who can register you to his own “propiska” address in his region (can be free if it’s your friend) or you can pay and “buy” this registration online. (this “registration” came from USSR time, it’s not so strict now and very unlikely it will be checked, that’s why it’s pretty common to buy it online).
Hotels and tourist web-sites can register you with their address for the whole period of your stay. Usually it cost around 10-20 eur for up to one month (for European citizens, but for countries with migration risk it will be at least twice more expansive).

If you don’t have this “registration”, you might need to pay a fine, or you can be rejected for 5 years to enter the country. I also read that it’s not very strict for Europeans and sometimes they just forgive you if you don’t have “registration”.. I mean migration officers when you exit the country.

But I recommend not to risk it, if you interested to visit Russia again.

P.S.: 3 month without registration and train tickets as it was described in previously accepted answer – this all for Russians/and other residents only.
If Russian/immigrant registered in one city/region but live in a different one then he need to prove that he is not staying there longer then 3 month, with train tickets and etc. But this checks mostly for immigrants from CIS countries, caused by illegal work-immigration etc.

Technically you are supposed to be registered within 3 days and hotels do it for you automatically. If you stay not in a hotel, have to do it by yourself, but it takes time and is very unpleasant procedure, as no one really speaks English at the registration place with the exception of some very central places in Moscow.

In fact, if you are a mere tourist being unregistered is not a big deal. Most times no one ever asks for it. However this depends on the length of your stay.

I’ve been told to register if you’re in a place for 7 days or more. Simply put, hostels should be able to do this for you, it takes a day, and is very simple. Just ask them when you get there. Nobody checked mine, however upon exit, but when I return to Russia I’ll continue to register – it’s one thing you don’t want to get caught short on.

You can stay without registration for up to 3 months after you arrive. But you need to have a document that confirms that you are leaving (airplane or train ticket will work).

If you want to stay longer, you’ll need to get a temporary registration in the police. That may be painful, because this registration is bound to a place where you live and should be made for you by your host or a hotel. I don’t know whether they will do this for you or not.

But you can “hack” the system by buying a new train ticket to the town of your stay (you can do this at the railway station). After that, you’ve got another 3 months. Sounds idiotic, but it works (for me it did).

Summary: If staying less than 3 months it’s not a problem if you have a ticket. Over 3 months it is a problem, that you’ll need to solve with your host or a hotel (or perform the above “hack”).