Where did one-in-a-million chance of nuking the atmosphere come from?

The worry in numerical form may be slightly more complicated than exactly "one-in-a-million"

Whereas Serber seems to dismiss the discussion as short and Compton's concerns as overblown, Buck's interview with Compton tells a prolonged process supervised in part by Compton that eventually ruled out the scenario, but only after far more than a few hours' consideration:

During the next three months scientists in secret conference discussed the dangers of fusion but without agreement. Again Compton took the lead in the final decision. If, after calculation, he said, it were proved that the chances were more than approximately three in one million that the earth would be vaporized by the atomic explosion, he would not proceed with the project. Calculations proved the figures slightly less - and the project continued.

_{Dongwoo Chung: "(The Impossibility of) Lighting Atmospheric Fire", Submitted as coursework for PH241, Stanford University, Winter 2015}

Referencing P. S. Buck, "The Bomb - the End of the World?" The American Weekly, 8 Mar 1959. (PDF)

That all this is not an absolute certainty at any time is evidenced by some of those involved to first speculate and worry, then calculate and calm their minds, only to be not so sure after all.
Even after the calculations seemed to say "practically impossible", project members didn't stop with their worries, until right after the blast. Knowledge was sparse, uncertainties to be weighed against optimism and pessimism. Trust in abilities of fellow humans and the possibility of simple errors looming amongst unknown variables and unforeseen and unintended consequences of risk taking.

One involved remembers:

But before going farther I need to mention a few things in my life that have shaped my opinions. The first occurred at Los Alamos during WWII when we were designing atomic bombs. Shortly before the first field test (you realize that no small scale experiment can be done - either you have a critical mass or you do not), a man asked me to check some arithmetic he had done, and I agreed, thinking to fob it off on some subordinate. When I asked what it was, he said, "It is the probability that the test bomb will ignite the whole atmosphere. "I decided I would check it myself! The next day when he came for the answers I remarked to him, "The arithmetic was apparently correct but I do not know about the formulas for the capture cross sections for oxygen and nitrogen - after all, there could be no experiments at the needed energy levels. "He replied, like a physicist talking to a mathematician, that he wanted me to check the arithmetic not the physics, and left. I said to myself, "What have you done, Hamming, you are involved in risking all of life that is known in the Universe, and you do not know much of an essential part?" I was pacing up and down the corridor when a friend asked me what was bothering me. I told him. His reply was, "Nevermind, Hamming, no one will ever blame you." Yes, we risked all the life we knew of in the known universe on some mathematics. Mathematics is not merely an idle art form, it is an essential part of our society.
_{R. W. Hamming: "Mathematics on a Distant Planet", The American Mathematical Monthly, Vol. 105, No. 7 (Aug. - Sep., 1998), pp. 640-650. (jstor)}

The occasion of Fermi’s satire that evening made Bainbridge furious. It merely irritated Groves:

I had become a bit annoyed with Fermi . . . when he suddenly offered to take wagers from his fellow scientists on whether or not the bomb would ignite the atmosphere, and if so, whether it would merely destroy New Mexico or destroy the world. He had also said that after all it wouldn’t make any difference whether the bomb went off or not because it would still have been a well worth-while scientific experiment. For if it did fail to go off, we would have proved that an atomic explosion was not possible.

On the realistic grounds, the Italian laureate explained with his usual candor, that the best physicists in the world would have tried and failed. Bainbridge was furious because Fermi’s “thoughtless bravado” might scare the soldiers, who did not have the benefit of a knowledge of thermonuclear ignition temperatures and fireball cooling effects.2405 But a new force was about to be loosed on the world; no one could be absolutely certain—Fermi’s point—of the outcome of its debut. Oppenheimer had assigned Edward Teller the deliciously Tellerian task of trying to think of any imaginable trick or turn by which the explosion might escape its apparent bounds. Teller at Los Alamos that evening raised the same question Fermi had, but questioned Robert Serber, no mere uninformed GI:

Trying to find my way home in the darkness, I bumped into an acquaintance, Bob Serber. That day we had received a memo from our director . . . saying that we would have to be [at Trinity] well before dawn, and that we should be careful not to step on a rattlesnake. I asked Serber, “What will you do tomorrow about the rattlesnakes?” He said, “I’ll take a bottle of whiskey.”2406 I then went into my usual speech, telling him how one could imagine that things might get out of control in this, that, or a third manner. But we had discussed these things repeatedly, and we could not see how, in actual fact, we could get into trouble. Then I asked him, “And what do you think about it?” There in the dark Bob thought for a moment, then said, “I’ll take a second bottle of whiskey.”

Segrè at Base Camp imagined apocalypse:

The most striking impression was that of an overwhelmingly bright light. . . . I was flabbergasted by the new spectacle. We saw the whole sky flash with unbelievable brightness in spite of the very dark glasses we wore. . . . I believe that for a moment I thought the explosion might set fire to the atmosphere and thus finish the earth, even though I knew that this was not possible.

_{Richard Rhodes: "The Making of the Atomic Bomb", Simon & Schuster: New York, 1986.}

For how this concrete number of "x-in-a-million" came into public debate with surrounding information in Hans A. Bethe: "The Road from Los Alamos: Collected Essays of Hans A. Bethe", Springer, 1991. p30 (Ultimate catastrophe). The core of it first published here:

In the November 1975 issue of the Bulletin, H. C. Dudley¹ claimed that it is possible that an atomic weapon might ignite a thermonuclear reaction in the atmosphere (or the ocean) and thus destroy the Earth. This claim is nonsense. Dudley's claim is based on a report by the writer Pearl Buck of an interview with Arthur H. Compton in 1959. Had Dudley consulted the original literature, or any of the scientists conversant with the problem, he would soon have recognized that Pearl Buck had completely misunderstood Arthur H. Compton.
The basic theory of thermonuclear reactions in the atmosphere was developed in a report by E. ). Konopinski, C. Marvin, and E. Teller 2, published by the Los Alamos Laboratory. This work was done before the first nuclear test at Alamogordo in July 1945, and assured the leaders and members of the project that no danger existed that the test would set the atmosphere on fire. The report LA-602 was circulated in August 1946; it was secret until February 1973 when it was declassified.
The only "candidate" for a possible thermonuclear reaction in air was a collision between two nuclei of N because N14 is a nucleus of relatively high energy content. Several nuclear reactions are possible, of which we mention [formula] […]

So there was never any possibility of causing a thermonuclear chain reaction in the atmosphere. There was never "a probability of slightly less than three parts in a million," as Dudley claimed. Ignition is not a matter of probabilities; it is simply impossible.

---

¹ _{H. C. Dudley (1975) The ultimate catastrophe, Bulletin of the Atomic Scientists, 31:9, 21-24, DOI}

_{Hans Bethe: "Ultimate Catastrophe", Bulletin of the Atomic Scientists, 32:6, 36-37, 1976. DOI}

This of course in hindsight and plainly against the worries of his own colleagues at the time. It may be the scientifically accepted and correct view on the matter, for Bethe now. But it was anachronistic to blame Dudley for 'misunderstanding'. After all, in Dudley's paper the source attribution is simply between bad formatting and plagiarism. (Compare Dudley paper with Pearl Buck, "The Bomb The End of the World!" American Weekly, March 8, 1959. Also quoted in Carl C. Gaither & Alma E. Cavazos-Gaither: "Gaither's Dictionary of Scientific Quotations", Springer, 2008.)

But as a bonus, this bulletin has the accompanying picture for illustration on Bethe's article: