Upvote:7
A type D visa or a residence permit allows the bearer to travel to any Schengen country, as set out in Regulation 265/2010. When you travel from Poland to another Schengen country, you do not enter the Schengen area, so the fact that your visa is valid for only a single entry is not significant. You must be careful not to leave the Schengen area, however, so you may not travel through the UK, Ireland, Romania, Bulgaria, etc.
Also note that if you have a Polish residence permit as a separate card, that is equivalent to a multiple entry type D visa, in which case you are no longer bound by the single entry nature of your original visa.