How were smallpox vaccines enforced in the US?

In my state (Oregon) I was not vaccinated as a child but had to be vaccinated to enter a state university (1969).

According to this article, the authorities in some areas did literally go door-to-door to check whether people were vaccinated.

Isolation was also taken far more seriously. Infected people were not just trusted to self-isolate - often they were forcibly interned to keep them isolated. The case of Typhoid Mary is not at all atypical for the time; whilst a different disease, it demonstrates the lengths to which the authorities would go.

In short, the right to life for the majority was seen as far more significant than the right to personal freedom for an individual, in the case of a disease with a high probability of fatality. This balance naturally depends on the probability of death, the effectiveness of treatment, and the degree to which personal freedoms are curtailed; but in the case of infectious and frequently-fatal diseases like smallpox and typhus, and without modern healthcare, this was clearly the best they could do at the time.

Let me first note that the percentage of vaccinated population necessary to achieve herd imunity varies with virus, but also with other factors, such as the density of population, nature of human interactions, etc.

Moreover, the herd immunity applies to populations that are being constantly exposed to a pathogen, meaning that isolated cases have negligeable chance to spread in the population. Smallpox existed only in human populations - unlike covid-19, influenza, Zika, Ebola and other viruses that we hear about today, which have animal reservoirs and/or transmitted by animals/insects. Thus, one needed to vaccinate those who were at risk of being exposed to already sick individuals, but not necessarily the whole population.

Note that this is also the strategy used today in respect to diseases that are not endemic to a specific region. E.g., tuberculesis is largely non-existent in most developed countries, so that vaccination is potentially more harmful/expensive than containing the small outbreaks. Thus, one vaccinates only those who could have been in contact with a knwon case or those who are traveling to areas affected by this disease.

It's hard to say how reliable this source is, but it is said in an interview with a history professor who wrote a book on the matter that in some cases more definite force was used:

The events of this time, known as the Progressive Era, paved the way for laws and regulations still in force today. Michael Willrich, the Leff Families Professor of History, chronicled this period in American history in his award-winning book, “Pox: An American History.” [...]

How was compulsory vaccination carried out in discriminatory ways?

Willrich: Ultimately, compulsory vaccination was carried out in many communities in a way that was discriminatory against African Americans and immigrant groups. There were examples of compulsory vaccination being carried out with force in immigrant tenement districts in cities like Chicago, New York and Boston. Local governments created “virus squads,” teams of police and vaccinators that cordoned off city blocks, entered neighborhoods in the middle of the night, and went door to door, checking people to see if they had vaccination scars proving they had recently been vaccinated. Police tore infected children from their mothers’ arms and took them to isolation hospitals called “pesthouses.”

I'm adding this because you asked "Did local medical authorities go house to house injecting everyone with the vaccine without asking for consent?" Unfortunately, the interview is not more detailed than that in that regard, like more concrete time frame and instances of such events. The book might have more in that regard.

Based on another source that uses the "virus squad" terminology (which I'm guessing it's a modern retrofitting), that seem to roughly refer to events around 1900:

The last epidemic of smallpox in Boston, Massachusetts began in 1901 and lasted for almost two years. A recent article (New England Journal of Medicine 2001;344:375–9) has described some of the excesses of medical fervour displayed in an attempt to control it.  The epidemic began in May 1901 and when it ended in March 1903 there had been 1596 cases and 270 deaths. Almost half (47%) of the cases had been vaccinated and in these the fatality rate was 11% whereas in unvaccinated patients it was 22%. In “public” (state) schools vaccination had been compulsory since 1855 and during the epidemic the rate of infection in children aged 6–10 years was about one sixth of that in pre-school children.  Initially, voluntary vaccination was offered but compulsory vaccination was introduced in December 1901. The penalty for refusing to be vaccinated was a $5 fine or 15 days in prison. Homeless and destitute people were thought to constitute a major risk to the rest of the population and men in cheap boarding houses were vaccinated forcefully by “virus squads”, often including three policemen to hold down the resisting tramp.

N.B. the 2001 longer article referenced is open access. It does say that force was at least not part of the official instructions (given to physicians)...

The instructions given to the physicians were as follows: “Vaccinate all who are willing and are not too ill. No force to be used.” [...] Persons who refused vaccination were subject to a $5 fine or a 15-day jail sentence.

But given how police was well known for "curbstone justice" at the time... a practice which persisted until much later... what was in the official instructions and what might have happened aren't necessarily the same thing. The NEJM article relates some of that by quoting the press of the time:

In November 1901, the Boston Board of Health ordered “virus squads” to vaccinate men living in inexpensive rooming houses.

A reporter for the Boston Globe accompanied a squad one night and described the scene: “Every imaginable threat from civil suits to cold-blooded murder when they got an opportunity to commit it, was made by the writhing, cursing, struggling tramps who were operated upon, and a lot of them had to be held down in their cots, one big policeman sitting on their legs, and another on their heads, while the third held the arms, bared for the doctors.” One “fighting tramp,” who “went down in a heap on the floor” from the blow of a policeman's club, received both vaccination and suturing of his scalp.