I used an older pool in Reykjavik and while there were some private change cabins, I never found out how to get the of use them.
Signs what to do came in several languages and/or pictures.
Always check for local exceptions, as mentioned in a comment by kiradotee some pools have mixed showers close to the outside of the area.
The shower area was open, stalls but no doors.
In the Blue Lagoon stalls came with a door and an area to change, also behind the door but the same rules about showering nude and only putting swimwear on later were published on the walls
there.
Most municipal pools ban the use of cameras or other recording equipment in the swimming area, although there are exceptions in areas with many tourists. (From a comment by Jacob Horbulyk.)
I have not been to other pools but found these instructions in several places online as well as in the pool I used.
If in doubt about any of the rules (or where you fit in with the gender separation,) ask. Most people in Iceland seem to speak more than two languages and are willing to help. That included the staff at the desk where you pay.
Credit:stackoverflow.com‘