How was the dynamics of a naval battle in the 17th century?

Tactics were developed in the 17th century but perfected during the 18th. The Admiralty Fighting Instructions codified tactics following the perceived chaos at the Battle of Portland in the First Anglo-Dutch War. This developed the "line of battle" to ensure that the whole fleet came into action. For ease of sailing this was broken into 3 for cruising & assembled as one in battle formation. Despite these tactics being changed at Trafalgar, nonetheless similar tactics were used at Tsushima and Jutland - & arguably by Oldendorf at Leyte Gulf. The British/English would close to half-pistol shot, but when ships boarded one another they could be touching. Broadsides were fired as each gun came to bear, not all at once, the interval would depend on the relative speed of the opponents. Battles could take all day - as in The St James's Day fight or even four days - as in the Four Days' Battle. Ships would be moving at around 4 knots at Trafalgar which could take ages. By contrast Quiberon Bay (1759) was fought in a gale. Ships could potentially take a huge amount of battering with round shot & were usually boarded & captured rather than sunk. Equally an accident or poor damage control could lead to a magazine explosion - such as the Mercedes in 1804 which exploded after being hit with a warning shot. The numbers involved in the Anglo-Dutch wars were over 100 on either side. Fleet size was generally decided on how many ships could be signalled to by a commander - which was generally five or six ahead & behind, leading to a squadron size of 10-12. In the really large encounters there're would be 3 commanders per squadron (Rear Admiral, Admiral, Vice Admiral) & frigates would lie behind the line in order to relay the flag hoists.

As to territory, the Anglo-Dutch Wars were fought over a short distance (England to Holland) & both sides were eager to come out, which made for large & frequent concentrations of force (if often indecisive). However the Anglo French & Spanish wars were often fought across the Globe and the French & Spanish were often reluctant to come out of harbour so the nature of the action was fundamentally changed, whereby the Royal Navy had to go in & get them (e.g. La Hogue, Cape Passaro, Santa Cruz, Holmes Bonfire, The Medway, The Nile, Copenhagen - ok that was the Danish - Basque Roads and numerous others. Navarino even v the Ottomans)

What else? Fire ships were much more common in the 17th century. The French tactics were to stand off downwind at twice pistol shot and fire upwards at the enemy rigging with chain shot in order to dismast their opponents then to manoeuvre astern and rake their opponent. The results indicate this was spectacularly ineffectual. The British were extremely successful due to higher rates of fire and accuracy (after Sir Philip Broke's improvements in 1812). Duriing the Seven Year's war the Royal Navy used goose quills filled with gunpowder instead of a rope fuse as this fired instantaneously before the ship could roll after aiming the gun.

The C17 was a period of evolution in naval tactics. At the start naval battles were generally conducted in a pell-mell fashion with small groups of ships supporting each other in an attempting to capture by boarding the enemy. There was extensive use of fire ships and much lower emphasis on the use of gunfire as a decisive factor.

By the end of the century naval battles were now utilizing the line-of-battle with the emphasis being on gunfire, boarding was no longer a tactic in a general action although fire ships remained common and were occasionally used with great effect, as at the Battle of La Hogue in 1692.

The first use of the line-of-battle as a deliberate tactic was probably by Maarten Harpertszoon Tromp at the Battle of the Downs in 1639, where the Dutch defeated the Spanish. However this was a one off and by the start of the First Anglo-Dutch war, 1652-54, the Dutch had reverted to the old way of fighting.

It wasn't until the 1653 when the three English Generals-at-sea Robert Blake, Edward Popham] and Richard Dean, all of whom had land based experience with artillery, introduced the Line-of-battle with the Fighting Instructions of 29th March 1653 which were the first to aim at a single line ahead as a fighting formation. NRS Vol 029 - Fighting Instructions 1530-1816 that the line-of-battle was formalized. This tactic was introduced by the English to take advantage of their superior average ship size and weight of fire. It proved so successful that it was copied by the Dutch and French and soon became the standard battle tactic of the period.

I wanted to add more links, but I can't until my rep is higher :(