When did the acceptance of diversity become such an important social goal in Western countries?

Here in America, as you suggest, women's rights, civil rights and the American Disability Act are the cause, but in a legal rather than cultural way. As anti-discrimination laws were passed and then strengthened through lawsuits or other legal requirements, larger companies were forced into proving they weren't discriminating by grudgingly having a few "diversity hires", and then -- after "but we already have a black woman" stopped working -- having a reasonable proportion of hires from protected groups.

After a few decades of that, the culture had changed. A diverse upper management team which would have been shocking in the 1970's, was now completely normal. Note how there's no embracing so far. Anti-discrimination law forced the change, which naturally caused a change in public attitudes. Sure, today you'll find CEO's saying they embrace the changes they were forced to make, but that was never a driving force.

The Holocaust and the reconstruction of Europe.

What is the opposite of diversity? Monoculture. The mono-ethnic, mono-religious nation-state.

What is the only way to achieve a monoculture? Violence. Those that do not conform must be suppressed, driven out ("ethnic cleansing") or murdered, whether at Auschwitz or at Srebrenica or just through a thousand individual local crimes.

As part of the attempts to prevent this from ever happening again, the European Convention on Human Rights was established.
In particular, article 8 "Freedom of thought, conscience and religion" and article 14 "Prohibition of discrimination".

This isn't the origin of "diversity" in general - the Austro-Hungarian empire was a pretty diverse place - but modern diversity thought does specifically set itself against ethno-nationalism.

Embracing diversity (as well as the contrary, wanting more uniformity) might be a bit of a historical constant. You can find it at the very beginning of Confucius' Analects.

Is it not delightful to have friends coming from distant quarters?

John Tzetzes from Constantinople expresses similar sentiments in the 12th century, albeit with a heavy dose of intolerance at the same time:

One finds me Scythian among Scythians, Latin among Latins...

And also to Persians I speak in Persian...

To Alans I say in their tongue:

‘Good day, my lord, my archontissa, where are you from? Tapankhas mesfili khsina korthi kanda, and so on’ . . .

Arabs, since they are Arabs, I address in Arabic...

And also I welcome the Ros according to their habits...

‘Sdraste, brate, sestritza’, and I say, ‘dobra deni’.

To Jews I say in a proper manner in Hebrew:

‘Your blind house devoted to magic, your mouth, a chasmengulfing flies,

Memakomene beth f*gi beelzebul timaie..."

I am sure that it is possible to find lots of similar quotes through the centuries.

One example of turning (intra-Christian) diversity into official policies is the Electorate of Brandenburg and later the Kingdom of Prussia during the late 17th and early 18th century, epitomized in Frederick II's famous sentence

Everybody should go to heaven in his own fashion here

(den hier mus ein jeder nach Seiner Fasson Selich werden)

As for the current push for more diversity, it seems to be mainly the logical consequence of women's emancipation and increased international mobility post-WWII. In the US the Black civil rights movement probably also plays a role. Supporting equal rights and non-segregationist policies means, in practical terms, that one should also support more diversity at the workplace.

On a more opportunistic note (one that matches well with Prussian and especially the Constantinopolan examples above), in some places there are very direct advantages from having a diverse team. E.g. when developing or servicing products on a globalized market. So economic globalization and modern communications do play a role as well.