Swiss customs: necessary when travelling from France to USA via GVA?

I’d go with what LaurentG has written. Declare it, pay the deposit (by credit card! Anything else will result in a for you completely useless CHF refund with according fees if you change it to USD) and get refunded upon leaving.

There are flights departing from either secor, the French and the Swiss. Therefore, depending on where your gate lies, you might still have to cross the border. I quote from the website:
“For some airlines flying to destinations other than France, check in and go to the international transit area.” So it depends on your flight whether you have to cross the border or not. (official website in English: http://www.gva.ch/en/DesktopDefault.aspx/tabid-60/ ).

To what amount do you estimate the wines you want to buy? There’s a certain amount per person (!) which you can import for free. It was 5 liters last time I checked, which is about 8 0.75l bottles (again: per person, so for two people it doubles to 16 bottles).

Furthermore: When buying stuff over 175 Euro in France as for an exportation certificate. This paper you get stamped at the French border upon leaving and then you can reclaim the VAT which is almost 20% of the price you paid.

You may use the transit facility of Swiss Customs. This is probably the most easiest way to transit with wine without problems.

Alcoholic beverages that exceed the allowances and are destined for
transit through Switzerland must be declared upon entry (see also
Alcoholic beverages). The same applies for tobacco goods (see also
Tobacco goods). A transit receipt is issued upon entry for this
purpose.

You need to pay a deposit, which is refunded by the customs office of
exit if all of the goods are re-exported. The deposit amounts to
roughly the equivalent of the import duties (see also Alcoholic
beverages and Tobacco goods). If the deposit is paid in cash (CHF,
EUR, USD, GBP), it will always be refunded in Swiss francs. It can
also be paid by credit card; the sum is credited to the same credit
card upon re-exportation.

Source: http://www.ezv.admin.ch/zollinfo_privat/04342/04359/04362/index.html?lang=en

The Schengen agreement is entirely unrelated to customs and goods, it’s only about the movement of people. The UK isn’t in the Schengen area and maintains its own visa policy but as a full member of the EU, it’s part of the common market, which means you can import any quantity of alcoholic beverages from other EU countries (for personal use). Conversely, Switzerland entered the Schengen area and agreed to follow the Schengen visa policy and to abolish border checks but still enforces restrictions on the movement of goods.

Strictly speaking, entering Switzerland with so much wine would therefore indeed be illegal. But Geneva airport also has a small French sector that is technically on Swiss territory but is treated as a part of France for customs and immigration purposes (part of the airport is built on what was earlier French territory but it was transferred to Switzerland at the time).

Whether you can use it depends on the flight you want to catch. You can check-in for flights to France and international flights from a few airlines but not all of them. From the official website:

Can baggage be checked in for an international flight in the French sector?

Yes, checking in is possible in the French sector for some international flights (other than France), but only those operated by Aeroflot, Alitalia, Etihad, KLM, Luxair, Tunisair, and Twinjet by making a call (telephone available on the check-in counter).

In any case, if you want to avoid Swiss duty or transit paperwork, you need to avoid entering Swiss territory and approach the airport from the French side. There was another question about that recently. If you have a French rental car, returning it in the French sector is actually to your advantage (no surcharge for crossing the border) but you need to make sure the company you got if from is present on both sides as some only have desks in the Swiss sector.

Note that there should not be any systematic ID check on the border and you can often cross it without seeing anyone but Switzerland can still do some spot checks from time to time. And if you are flying to a non-Schengen airport, Swiss customs can of course inspect your bags in the airport. You could still try your luck but I am not sure what the odds of being caught really are.

Alternatively, you could enter Switzerland and stay in Geneva, hoping not to get caught on the border (perhaps avoiding the motorway) and then leave on the last day to enter the airport from the French side. I don’t think the odds are too bad but it’s obviously illegal and you could be liable for a hefty fine. Beware that many rental cars often have exotic registration numbers (e.g. 60 in France or AI in Switzerland) that might invite extra scrutiny.

Finally, you might also stay for some time in Geneva and simply import the bottles officially. VAT is low in Switzerland and the duty on wine for less than 20 litres (26 bottles) is not very high either and all that can be refunded if you declare them as goods in transit. The only trouble is taking care of the paperwork.