score:3
You're asking for opinions, which are very personal. But...
Peru has plenty to offer. You're probably flying into Lima, but not one person I've ever spoken to who has been to Lima has said it was a city worth visiting. Because of that, I skipped it on my trip this year (though I am stopping by next year).
Distances in Peru (and South America) are huge. You'll lose a lot of time moving from major Peruvian site to major Peruvian site. Adding Brazil to your itinerary will only give you less time to actually see and do things.
Yes, Rio is very much worth a visit, but as you found out, flying between Peru and Brazil is prohibitively expensive. If you want to get to Brazil on the cheap, you can take a bus via Cusco to Puerto Maldonado from where you can get to Rio Branco, which is just across the border in Brazil. From Rio Branco, you can get budget airlines to Rio or any other Brazilian destination of your choice. But, this is a time consuming process.
Yes, the Amazon experience in Peru (yes, you can visit the Amazon in Peru, too) is decidedly different (and not necessarily worse) from what you can get in Brazil. But, to get to the Brazilian Amazon from Peru either will cost you plenty of time (overland) or money (flying a complex route to get to Manaus, most likely).
My advice would be that if you can set aside 20 days, spend them in Peru and, perhaps, Ecuador. Or, if you really want some diversity, take overland transport to Bolivia.