What foreign language is the most common in Serbia?

Studying foreign language was mandatory even in 20th century and this was mostly French or German. German was mandatory in the first half of the last century and other languages optional, mostly French unless you were in Gymnasium proper where Latin and Old Greek were mandatory as well for 4 years.
My mother and father were speaking French and German fluently and father added Russian later. Since 1950 English or Russian were mandatory (your choice), but probably 90% were opting for English. Some lucky people were learning German, but not too many. Latin was mandatory for 2 years in grammar schools. Second language was mandatory in grammar schools oriented to social studies.
All of the people I know from my generation (1952) use English and some, due to the job they work, use it on everyday bases. The fact is that learning was mandatory, but the question is how many people on a street will be able to use it or be willing to use it. If you do not practice using language, you will lose it.
Learning from TV, PC or so helps younger people to exercise it, but to say that only 10% people with secondary school speak/use English is a nonsense. I was (and I still am) traveling a lot over Europe and my impression is that English is useful in Germany, Finland, Holland, but not in Italy, Austria, Switzerland, and not at all in Hungary, Bulgaria or France. Similar situation is in other ex-Yugo countries except for Croatia, where German is more widely spread.

It depends on their age. As mentioned above, English in a mandatory subject in school and many young Serbs are exposed to US films and British TV series and I have heard many say they have learnt from there. You also have the phenomena of Serbs going to other countries in the north, like Romania, speaking English with people who live there.

I am not sure when English became mandatory, but I think it must have been around 2000 and I came across people born in 1985-86 who could not speak English.

Thus, most people in their mid to early 20s will speak some English, some good. But keep in mind that many simply don’t care and some people don’t go to school. I met group of four doctors in Serbia, all born in 1984-5. One spoke what I would call fluent, owing to living abroad; one spoke conversational; one spoke passing and one spoke none at all. Of people in their 40s, a minority will have some passing use, but in general none. My friend who was born in 1975 has a huge English vocabulary, but they learnt from watching films, listening to music, reading on the internet etc., so their grammar is Serbian, but with English words. Then of the elderly, you will find next to no one who speaks English.

As a rough guess, I would estimate that around 15-25% of Serbs in Serbia speak conversational English.

While this question is answered (in short: English is probably your best bet), as a native, I’d like to provide a more complete explanation.

First of all, the first part of Mark’s accepted answer is highly misleading (even if we ignore the fact Wikitravel’s Quick Facts now contain data that is 13 years old). The two major flaws in that logic are:

1. The languages listed are based on census data which only covers one’s native language. Since the original question asks for a language to "communicate with locals", one would need to take into account languages people speak as a second (or even third) language as well.

2. The data from Wikitravel also doesn’t take into account localities. People who speak Hungarian are far more likely to be found in areas closer to the Hungarian border – same goes for other languages. When it comes to large cities, it’s far less likely to find people whose native language is not Serbian.

I tried to find accurate data on how many people speak English, German, Russian and others as a second language, but couldn’t find such a thing so I’ll just write the languages that the locals in Serbia are most likely to understand based on my opinion and some local knowledge (ranked from most likely to least likely):

1. Serbian

   Naturally, being the official language, it’s the most spoken one.

2. Other Serbo-Croatian variants – Bosnian, Croatian and Montenegrin

   As already mentioned, these languages are so similar to Serbian that you can be sure anyone who speaks Serbian will understand them.

3. Other languages that were spoken in Yugoslavia – Macedonian and Slovenian

   While not as similar to Serbian as the Serbo-Croatian variants, it’s possible to achieve at least some basic communication using these languages with people who speak Serbian. In addition, it’s possible you’ll bump into people who actually speak these languages since it wasn’t uncommon to be familiar with them before the breakup of Yugoslavia and people from Macedonia and Slovenia still visit Serbia occasionally.

4. English

   English is a mandatory subject in all primary schools, all gymnasiums and most professional secondary schools. It’s also a mandatory course for students on most universities. While the quality of English education varies strongly between different schools, it’s highly likely any younger person you meet will speak English, often quite fluently.

5. Russian

   Similar to how English is now (see above), Russian used to be very popular in communist Yugoslavia and many older people learned it at school and used to speak it very fluently. Keep in mind, however, that most haven’t used it in a long time and their proficiency with the language likely dropped, but you’ll still be able to communicate with them if you speak Russian.

6. German

   Living and working in a German-speaking country (usually Austria) is often a goal for many and therefore many people know at least some German (either in primary or secondary school as an elective subject or by taking German classes).

7. Local languages – Hungarian, Slovak, Romanian, Rusyn and Albanian

   Finally, these are the languages spoken by minorities in some areas (you could refer to this highly outdated map to have a general idea where, but it mostly applies to Vojvodina). Keep in mind that these people often speak very specific dialects of the listed languages and you may find it difficult to communicate if you’re a native speaker.

Keep in mind I left out French and Italian simply because I’m not personally familiar with how common they are. These two do exist as elective subjects in many schools and on many faculties, so it’s possible you’ll have some luck if you try speaking them.

A lot of people in Serbia, especially younger, understand English and speak English at least basically, and a visitor who doesn’t know Serbian at all will certainly have not any problems to, at least, basically communicate with people on the street, shops, bus stations, hostels etc. People are familiar with English language maybe not so much because of learning it in school (although it is a factor) but from movies and computers and Internet.

English is really widespread, so that would be your best bet. They might have trouble understanding, and you will probably have a lot of trouble understanding (since they have thickish accents). Unless of course you spoke another Yugo language, but I am assuming you don’t since most people who do know that there is basically no difference.

As for whether Russian is common,it was in older generations but now English is more commonly taught as a second language.

Any language of the Serbo-Croatian family. This group includes besides Serbian, Croatian, Bosnian and Montenegrin. These languages are pretty close to each other. They only started to really move apart when Yugoslavia broke up.

It also depends on the place where you go. If you go to a colorful place like the Vojvodina, the choice becomes larger. This province has six official languages: Serbian, Hungarian, Slovak, Croatian, Romanian, and Ruthenian. In Central Serbia you will also find places where people speak Albanian or Bulgarian.

As in many other countries too, English is becoming more and more popular. This is especially true for a more younger population, but not only.

Apart from that, use the international master language, and communicate with hands and feet. Learn some basic phrases in Serbian. Knowledge of the Cyrillic alphabet could be useful too, it is still widespread in Serbia. The Serbian language uses both Cyrillic and Latin alphabets.

From Wikitravel:

Language: Serbian 90.1% (official), Hungarian 3.8%, Romany (Gypsy) 1.1%, other 4.1%, unknown 0.9% (2002 census)

So in that respect, Hungarian is the next best bet. However, that’s ‘official’ languages and was a 2002 Census – if you keep on top of world affairs, things have changed a lot in that area since then.

So reading further down the same page:

Serbs are warm people, especially towards tourists. They are very
welcoming towards tourists, of which there are not many as the
country’s full potential has yet to be reached! Most Serbs speak some
English and are eager to speak it (seniors, however, are more likely
to speak German and/or French), so you will be able to find your way
around by asking directions. Most tourists come to Serbia in the
summer and you can often hear German, Italian, French and English in
the streets of Belgrade, while Slovenian tourists pour for New Year
holidays.

So based on that – English is a pretty good bet, if you speak that (which it seems you do) – ask the young people questions for a better success rate. If you speak German or French, I’d be asking older people for directions etc.