Is there evidence that the annual pilgrimage to Mecca facilitated cultural exchanges?

First, while Hajj is one of the Five Pillars and is recommended to do it once a year, the actual requirement is once in the lifetime only. If you take into account the spread of Muslim rule in X^th and XI^th centuries, only certain percentage of the Muslims would actually go on a Hajj regularly, let alone yearly. There are indeed conditions to be met before one could go: wealthy enough so that family can manage without person's presence, for example (see al-Baqarah 2:158,189,196-199; al-Hajj 22:27-36). On the other hand, even if the medieval period (specified by the question) was that of the constant warfare between (broadly understood) Islam and Christendom, the Mediterranean was generally safe for Muslim traffic (Ibn Khaldun – "The Christians cannot float a plank on the inland sea that is the Mediterranean"). So it actually could be rather easier to go for a Pilgrimage than I would imagine. But then again, even by sea round-trip from Al-Andalus to Mecca would take literally years (see worldbuilding.stackexchange.com - Vashu's comment about trip Rome - Egypt being on average 1,5 times a year***).

Secondly, even if Muslim sphere of influence was leading in the world scholarly (hesitate to use science here, but to some extent I would use those interchangeably in context of the period), there is little evidence that points to the majority of "Islamic scholars" to be actually Muslim. To be fair, there is not much more that they were NOT, too.

Taking all that into account reality would be that majority of the pilgrims would be mostly of certain statute and heritage, which also implies certain limits on types of ideas to be spread.

Next, the decline (or, according to some sources, almost total disappearance) of scientific progress in Muslim world by XIV^th century, universal ban on non-Muslims entering area around Mecca and other... developments... definitely had impact (see Toby Huff, "The Rise of Early Modern Science"). I would say that famous saying of caliph Umar on the Library of Alexandria in 642: "They will either contradict the Koran, in which case they are heresy, or they will agree with it, so they are superfluous" is well in line for usual Islamic conquest style. He also said something similar in 651 after conquest of Ctesiphon. Book burning is not uncommon in every society of that era (notable mention would be burning of Library of Constantinople in 1199 by Crusaders), but Muslim history, including their own religious/historical sources, is replete with burning of all kinds of books.

Though famous, Mansa Musa's hajj, being most widely recorded in history and thought to be done so rather accurately had it's own problems. Mansa was apprehensive of visit to the Sultan in Cairo as this was linked to humiliating court protocol of Mamlukes at the time, which also contributed to requests to be released from the duty of paying Sultan a visit in subsequent hajjs. Anyway, Mansa is reputed to bring back scholars who were, according to oral traditions, instrumental in development of Timbuktu's as culture and learning center. While it was certainly true about Abdel-Rahman al-Tamimi (who was recorded as one of the accompanying Mansa on return scholars), the Abu Ishaq al-Sahili's architectural (though not as a poet and jurist) record is more sketchy. Also, analysis of the architecture of the buildings traditionally attributed to him suggest more Berbery influence than Andalusian (who al-Sahili was). There is no doubt that the architectural transfer of knowledge happened, but it's most likely naturally occurred with influx of Moroccan/Berbery people to Mali. (see Suzan B. Aradeon: Al-Sahili : the historian's myth of architectural technology transfer from North Africa)

The biggest impact of the hajj on the development of the Islamic polities around the world (it is clearly noticeable in West Africa, Siam and Malay, East Africa as well) is the reformation. Scholars brought on return from hajj usually introduced the concept of the jihad and rulers more often than not raised that particular banner. For the region of Mali Empire and it's neighbors at that era this was true for Almoravids, Almohads in the north, Takrir in the east which expanded to the west to Sila (which converted as a result of war). There are records of the jihad waged by kingdoms of Western Sudan in general (see al-Bakri: 'Kitab al-mughrib fi dhikri bilad Ifriqiyah wa-l-Maghrib' of al-Masalik wa-l-mamalik).

It needs to be pointed out that Mali Empire to be notable exception, where Islam was a vehicle to diplomatic and commercial spreading of influence.

In general, this is very interesting question. Unfortunately, according to info presented above and contrary to the thesis proposed by OP, the hajj would have the exact opposite influence on exchange and spread of culture and science. While some of it undoubtedly took place, they were the exception rather, than norm. And it remains a historical fact (which is corroborated by mostly Islamic sources), that what was exported was something else.

However...

If we take into account some recent development in the studies of origins and early spread of Islam the theory from OP has some merit. A lot, actually. But not in the way one would think. Mecca may be the destination of the Hajj only since end of X^th century. And the first ever mention of Quran (as a some sort of source of teachings), Hadith and Sunnah in any written and compiled form does not appear before 822 AD or so.

Mecca is in the Quran no more than twice (and even then two different names are used), but description is wrong. Location of the city of Mecca is isolated from all major trade routes of the time, terrain is quite inhospitable (so pilgrimage would be a serious challenge back then), and all that, of course, is quite different from what is in the Quran.

Now, if the Hajj was to be made to Petra/Jerusalem (as some research suggests), then I would say that OP's premise has a lot of merit. Why Petra and/or Jerusalem? Because Mecca is not mentioned in any document before late IX^th century, Petra is gone by the end of VIII^th, and Jerusalem for a while held the status of holy city for Islam. However, it was Petra, not Mecca, that was the destination for pagan pilgrimages of worshippers of, for example, Allat (which name in another variant appears in Quran). From this bit the pagan ban on entering Mecca comes from (see Dan Gibson, "Quranic Geography" and Patricia Crone, Michael Cook "Hagarism: The Making of the Islamic World", Tom Holland "In the Shadow of the Sword").

But that would mean a lot of different things, not always in line with traditional views...

Most certainly. The most famous example was probably Mansa Musa of Mali. He was enormously wealthy from his empire's gold production, and made a lavish Hajj in 1324 accompanied by thousands of courtiers (he distributed gold so freely that in Cairo its value dropped by a quarter against silver).

Musa was impressed by the many scholars and men of letters he met on the way. His empire was a backwater in comparison to Cairo or even Mecca, so he brought back a large number of scholars and settled them at Gao and Timbuktu, where he founded a university (the Sankore Madrasah) that still exists today. If you're interested I wrote more about it here.