Has there ever been a non-rectangular banknote?

One candidate seems to be some 1945 Finnish banknotes seem to have been near-squares. From what I can see they are still rectangular though, if an unusual ratio. Unfortunately I cannot find dimensions to be certain.

Maybe a little out-of-the-box thinking is called for.

Source: Alexander Klink via Wikimedia Commons, attribution 4.0 license

There was very little specie (i.e. precious metal coinage) in Britain's North American colonies, so throughout most of the Colonial period, tobacco was money in the big tobacco-producing colonies of Maryland, Virginia, and North Carolina. This was not a mere commodity trade -- colonists went so far as to use the "pound of tobacco" (worth about 3s) as the monetary unit. Debts were recorded, taxes and fines were levied, and other goods were recorded in pounds of tobacco.

This is something called "commodity money" (as I explained in an old Worldbuilding SE answer), but I thought it might fit here, because the use of commodity money in the Colonies evolved directly into paper money:

• The tobacco is a stand-in for specie, just as paper money is.
• Payments could be in thousands of pounds of tobacco, which could be difficult to carry around, so payments and debts were often just ledger enties or warehouse receipts.
• Tobacco prices back in England could vary widely, and so did the prices of imported goods.
• The receipts were typically for fixed amounts and couldn't be "broken" the way a piece of eight could.

So, individual colonies began issuing paper money.

Admittedly the above is (roughly) rectangular and tied to British currency. But during the Revolutionary War, they had to go back to tobacco, because Continental money was essentially worthless:

(Source:smokersforum.in; i've no idea where they got it from)

Here are a couple of square banknotes. The first one is also possibly unique in that it is one-sided.

"A small square shaped and rather scarce very good or much better banknote from Argentina. This is the 1st April 1867 one peso banknote issued by the Province of Buenos Ayres in Argentina. The note is of white paper with black printing." Source: picclick.com

Thailand has also issued a square bank note. This one is from 1987.

"A note issued on June 3, 1987, represented a new initiative to honour the King on his 60th birthday. It's 15.9 centimetres square: The "1" represents the first banknote ever created, the "5" is for the fifth birthday cycle, and the "9" stands for King Rama IX." Source: nutmegcollector

The site Numisbids, under the category 'Emergency Money' has several examples of round, octagonal and triangular money in addition to the ones already mentioned by LangLangC. Some of these may qualify as paper money but not, I think, as banknotes. As the OP mentioned both banknotes and paper money in his post, here is an example from Germany.

Source: Numisbids

Of additional interest, in France, monnaie de nécessité (emergency money) was sometimes issued printed on cardboard between 1914 and 1923.

Source: NaturaBuy

It has been pointed out in comments below and elsewhere that a square is a rectangle. Technically, this is true, but (rightly or wrongly) in common usage this is not a hard rule and there is more than one definition for a rectangle. The Cambridge Dictionary (American) defines a rectangle as "a flat shape with four sides and four 90° angles, with opposite sides of equal length and two sides longer than the other two". The Oxford dictionary says (for ‘rectangle’) "A plane figure with four straight sides and four right angles, especially one with unequal adjacent sides, in contrast to a square." The Collins dictionary (American) says a rectangle is, "1. any four-sided plane figure with four right angles, 2. any such figure or shape that is not a square; oblong".

(all highlighting is mine)

Regular banknotes, issued in "normal times" and made from paper are practically all rectangular and without holes. But there were a couple of times when people remembered that anything can be "money", if they just believe in it.

Below are some examples of holes in notes and odd shapes. These examples were chosen to fit the question as originally framed. Some examples still fit the updated question: Zimbabwe's hole was officially re-issued, although not designed that way. Some examples of Notgeld were designed and issued in more or less round form. The last example even being completely irregular from the start as the material used was small and intended to be used for another purpose…

Having a hole like coins?

Hole Punched Notes
In 1997, Zaire’s (now the Democratic Republic of the Congo) dictator Josepeh Sese Seko Mobutu was finally overthrown. As his face featured on the country’s bank notes, the new government simply punched out Mobutu’s face out of their existing currency and continued to use it until new currency could be printed.

Hole Punched Notes, UKcredit, MoneyLife, 2017

If this is not that strict about "not a rectangle but still valid, legal tender", then:

Then the Portuguese did something weird to the edges of the Real:

Portugal's first paper money was introduced in 1797 by the government.5 Denominations issued until 1807 included 1200, 2400, 5000, 6400, 10,000, 12,000 and 20,000 réis. Some of these notes were revalidated for continued use during the War of the Two Brothers (1828 to 1834).5 From the 1820s, several private banks issued paper money. The most extensive issues were by the Banco de Lisboa, whose notes were denominated in both réis and moedas, worth 4800 réis. This bank issued notes for 1200 and 2400 réis, 1, 4, 10, 20, 50 and 100 moedas. The Banco Commercial de Braga, Banco Commercial do Porto, Banco de Guimaraes and Banco Industrial do Porto also issued notes, with bearer cheques issued by a number of other banks between 1833 and 1887.
In 1847, the Banco de Portugal introduced notes for 10,000 and 20,000 réis.6 5000 réis notes were issued from 1883, followed by 50,000 réis in 1886. In 1891, the Casa de Moeda introduced notes for 50 and 100 réis,7 and the Banco de Portugal introduced notes for 200, 500, 1000 and 2500 réis, followed by 100,000 réis notes in 1894.

And early American banknotes are weird as well.

Continental Currency. 1776

$1/6 Plate C Serial Number: 145,707 CC 02/17/76

Signer: Robert Tuckniss (in red ink).

Size: 80 x 60mm (front border design: the vertical dimension is 78mm while the horizontal border is trimmed on our example; back border design: 74 x 55mm).

Comments: Numbered and signed in red ink. The sundial with the "FUGIO" legend and "MIND YOUR BUSINESS" motto appear on the right center of the front. In this fractional denomination, one ornament appears in the upper right corner of the sundial frame; this is keyed to the denomination as each mark equals 1/6th of a dollar. In the right border cut "CURRENCEY" is misspelled. The back shows the thirteen linked rings representing the colonies and the legends "WE ARE ONE" and "AMERICAN CONGRESS". Paper contains blue threads and mica flakes.

Provenance: Purchased through the Robert H. Gore, Jr. Numismatic Endowment from the EANA mail bid auction of 1/13/96, lot 278.

(click to enlarge)

From a Swiss auction of "Banknoten und Notgeld", SINCONA, 2015, note that in that catalogue many more such examples are found. Sometimes the "holes" just look like ones as they are really a look-through window for the watermark. But often in times of crisis or other sudden changes they are really holes and the money retains its, well, 'a' value. :

If you insist on "designed and issued that way" (instead of 'remained in circulation') then I guess only the types of Mobuto and Soviet holes count (they were re-issued in that way), and only the paper-like Notgeld shapes count. Round and oval from the start, sometimes. That would make 1914–1923 pretty much the timeframe where to look for more examples (if we do not go much further back).

But really, anything can be declared money, even if it is inherently worthless. For example cut up playing cards, issued in 1914 in Lopischewo, East Prussia:

_{"Notgeld: Der schöne Schein", Spiegel, 2008.}

And from previous threads here on HS:E:

Although the material is in both cases not even paper, the first was worth and intended to stand in for paper.

Just for entertainment, larger versions of the Notgeld, click to enlarge: