Why are public bathrooms in the USA not payment-based instead of privacy-violating?

Pay toilets are extremely unpopular in the US, or at least they were back when there were enough of them for people to be mad about their presence. There was a sort of revolt against them in the 1970s, complete with an organization called the Committee to End Pay Toilets in America, lawsuits, laws banning them (many of which were since repealed), and the removal of many pay toilets. You can read more in Why Don't We Have Pay Toilets in America?. There are de facto pay toilets in the form of businesses refusing use of the facilities to non-customers, which can pose difficulties in bathroom access for homeless people and mobile workers, but actual pay toilets are rare.

As that article notes, there's a potential discrimination issue too. Pay toilets in the US traditionally involved locks on the stall doors, which allowed men to use urinals for free while women always needed to pay to enter stalls. This would be considered unacceptable today, but an arrangement that charged to merely enter the restroom poses other issues unless dedicated attendants are used (how is the money collected? will people constantly prop or hold the door open? what if someone simply wants to wash their hands?)

Beyond that, your question is based on some premises that don't seem well supported.

You're assuming that lack of funds is what causes businesses to not afford to build and maintain bathrooms that meet your standards. But the cost difference between typical American public restroom partitions and larger-walled cubicles isn't that much. If a business wanted to build a restroom with European-style cubicles, they could just do that (indeed, some have). And the revenue that can be raised by a public toilet isn't all that great—unless you're going to make everyone pay with an app (which people have tried to make a thing, but it's not one), remember that the largest common coin in circulation in the US is only worth 25¢, and bill acceptors on toilet doors seem unlikely. Nobody is sitting around deciding between free toilets vs more private ones when they design a restroom; that's not the relevant trade-off.

I can't really imagine a fee that would "get rid of the people who want to just take advantage of the public utility," because those looking to take advantage of the facility can almost certainly scrounge up the small sum of money required to enter. An admission fee high enough to really deter potential misuse would be so high so as to bring legitimate customers into a state of rage as well. Self-cleaning pay toilets on the streets of San Francisco were used for private drug use and prostitution; the small admission fee did nothing to stop this—the addition of attendants to supervise the facilities did.

You're also assuming that all public toilets in the US make it easy for someone to "trivially violate your privacy while doing the most private imaginable things." This happens sometimes, but is far from universal. Many businesses, especially smaller ones, have single-occupancy restrooms where you have the entire room to yourself with a full-length locked door. In larger restrooms, there are not generally gaps on the sides when the door is closed (unless the bathroom has been vandalized or is poorly maintained) and the gaps at the top and bottom are high/low enough to prevent someone from looking in without climbing/crouching on the ground, which is frankly not a common problem.

A pay toilet also turns the facility into a business transaction. A survey indicates that Americans might be willing to pay 25¢, but in exchange they'd expect a clean and well-maintained restroom. While a business should provide that anyway if they want to keep customers happy and coming back, few businesses want to have conversations on an hourly basis of the form "I PAID TO USE THIS RESTROOM AND IT WAS NOT CLEAN I WANT MY MONEY BACK RIGHT NOW!" over a quarter.

Or to put that another way, consider the people who are currently having public freakouts over being required to wear a mask to enter a store during a pandemic, and ask how you think they might behave if required to pay to use a toilet.