Traveled on an invalid ESTA due to prior N Korea trip. How screwed am I?

A US visa application can be refused for a variety of reasons, but simply having violated US immigration law does not guarantee a denial. Certain particularly severe violations trigger mandatory bars: for example, accruing more than 180 days of unlawful presence in the US during a single trip, being deported from the US, or committing misrepresentation in order to obtain an immigration benefit or employment in the United States. The grounds for these mandatory bars are enumerated in federal law.

Most US visa refusals are not based on mandatory bars. The most common type of refusal is 214(b). This is a "catch-all" and happens when the consular officer, using their discretion, believes that a nonimmigrant visa applicant is likely to use the visa for something other than its legal purpose.

Certain past immigration violations do not trigger a mandatory bar, but result in an almost certain 214(b) refusal, at least in the next several years. For example, this will generally be the case if a traveller overstayed their admission for, say, 3 months. A mandatory bar only kicks in at the 180 day mark, but overstaying for 3 months without having a very good reason would still make you look really bad; in such a situation, it would be extremely difficult to convince the consular officer that you won’t do such things in the future.

So that brings us to your case. Your previous entry to the United States was not in compliance with law, but it was a "procedurally regular" [1] admission (or at least, you’ve given me no reason to think otherwise). That alone implies that you were not "unlawfully present" in the United States. It also appears that you did not commit any misrepresentation. Therefore, while you technically broke the law, there are no mandatory bars that are applicable to your situation. A consular officer will consider whether to grant your visa based on their discretion. The mistake that you made was innocent; it has little risk of being repeated by you, and it would not lead a reasonable person to believe that you will commit more serious violations such as overstaying or working illegally. Therefore, it is unlikely that you will be refused a visa based on this alone.

[1] For more information on the doctrine of procedural regularity, see http://myattorneyusa.com/when-entry-of-an-inadmissible-person-qualifies-as-admission

Unlikely to be screwed. I was also caught out by the sudden rule change and described my experience in this answer. At my visa appointment, the US embassy merely asked if the visit had been for tourism purposes (presumably as opposed to sanctions-violating purposes); and CBP asked nothing at all.

In your case, a fully truthful visa application will show that you visited the US after visiting the DPRK, on an ESTA in late Aug 2019. I find it extremely unlikely that this will be grounds to deny you a visa, it being that the official would need to be acutely aware of the timelines and not allow leeway for the suddenness of the rule change. It is probable that the visa officer will make a positive decision on the spot after quickly figuring out that you are an honest person.

It is worth mentioning that the embassy staff are significantly more pleasant to deal with than CBP.

The only way to get a definitive answer will be to apply for a US Visa, but realistically it’s extremely likely that the answer is "not at all screwed".

Technically you did break the rules, and officially, ignorance (ie, not being aware of the rules) is not a valid excuse for doing so. However the fact that you did not actively attempt to mislead anyone, and the fact that you are taking steps to correct the error (by applying for a visa) will almost certainly cause them to overlook the previous indiscretion.

Whilst not exactly the same, I have a friend who was affected by the same rule. He is a UK citizen (from birth) and grew up in the UK, however he was born in Iraq. Like you, he had an existing ESTA from before the new regulations came into place, and attempted to enter the US shortly after they came into effect. Unlike you, he was actually stopped at immigration and detained as his passport listed his place of birth which clearly made him unable to enter under VWP.

US CBP staff were understanding of the situation, and allowed him to enter the US despite not having the correct paperwork – with instructions that he would need to obtain a visa before returning to the US. He has since obtained a visa without issue (despite this previous incident) and has since visited the US using that visa without issue.