How was security on the Silk Road between China and the West maintained?

Mentions of Bandits and robbers:

Bandits and robbers were a constant threat on the Silk Road. Xuanzang mentions several encounters with bandits.

Near Dunhuang, the Silk Road split in two to skirt the rim of the Taklamakan Desert. The roads met again 1400 miles west at Kashgar. But between these two oases lay the Silk Road's most dangerous terrain. Among the threats were starvation, thirst, bandits, and ferocious sandstorms that were known to bury entire caravans.

The Mogao Caves near Dunhuang are laden with Buddhist art spanning a period of 1000 years. Most of the images are religious, but there are also images of everyday life. There are scenes that show travelers on the Silk Road, and some portray bandits.

The Logistics:

The Silk Road goes through some of the most challenging terrain in terms of vast deserts and high and rugged mountain ranges. The terrain, the weather and many other risks did indeed result in higher transit costs. Bandits and robbers certainly added to the risk. But several systems were in place to reduce the risk.

Few people ever traveled the full length of the Silk Road. The goods were transported by a series of routes and agents. This mode meant that local agents, familiar with terrain, politics, and bandits of their own region, who were better suited for the task, would ensure safe transit.

The shipment of goods from South Asia to Central Asia (or vice versa) was a long process that involved a number of transporters, which resulted in high transit costs. Whether the transporters were local villagers in the role of kiraiyakash or professional long distance porters such as the caravan men in Rasool Galwan’s autobiography (Galwan 1923), the transporters of goods through Ladakh had to be well informed concerning regional conditions, and familiar with the terrain. Landslides, sudden snowstorms, and bandits were just a few of the hazards faced by those transporting goods through Ladakh.

Source: Trade and Contemporary Society along the Silk Road

Travelers joined caravans to benefit from safety in numbers and experience of the caravaneers gathered from their previous trips.

Caravans were used mainly in desert areas and throughout the Silk Road, where traveling in groups aided in defense against bandits as well as helping to improve economies of scale in trade. -- Wikipedia

Another important choice was in the the routes used by the merchants, which were selected according to the political stability of the regional power. Centralized states, confederations and regional powers promoted trade and diplomacy. They invested in communications and economic infrastructure such as secure roads, water depots, inns, reliable coinage, standard weights and measures. They imposed taxes on travelers and traders. The merchants found it wiser and safer to pay the controlling authorities for safe passage than to risk encounters with bandits.

Defense against banditry took place at private and institutional level. Caravans of goods needed their own guards against plundering by the bandits (i.e., for security risk), and this was an added cost for the merchants making the trip. The institutional level had three forms: The Chinese garrisons and watchtowers beyond the Great Wall, Mongolian postal stations, and caravanserais in the Middle East and Anatolia. These institutions provided safety, supplies, and lodgings for merchants. Besides, the Chinese soldiers informed about incidents using smoke and flag signals in real time.

Source: Managing Supply Chains on the Silk Road

Different groups rose and fell through the ages, gaining political and military power, and hence controlling the trade along the Silk Road. Trade along the Silk Road was at its zenith during the Tang dynasty due to the stability of the government.

The travelers along the Silk Road changed over time. Chinese, Yuezhi, Bactrians, Indians, and Sogdians were the first to create the historical Silk Road in Central Asia in the first century BC. In time Muslim powers came to control large parts of the road.

Islamic patrons built hostels, known as caravanserai, that accommodated both people and beasts of burden. In addition, the Seljuk Turks who controlled the western part of the Silk Road offered the traders a special guarantee of safety. The government assured their financial security by paying compensation out of the state treasury for any loss caused by robbery. The Turkish authorities, whether the unified Seljuk sultanate or independent amirs, also built fortified caravanserais that provided food, fodder, and lodging for the travelers at intervals of one day’s journey apart all along the trade routes.

Source: The Silk Road in World History

Taking into account the the time taken for 1 day's travel, caravanserais were strategically located on the trade routes at distances of 25 to 40km from each other. The topography, of course, affected the distance of the caravanserais. The caravanserais not only sheltered caravans, but also served as military stations.

And lastly, there was also Insurance.

The Seljuk Sultanate of Anatolia created a state insurance policy in order to manage the security risks of land and sea traders whose goods are damaged or stolen due to bandit, pirate, and neighboring state attacks (Turan, 2009). For insurance purposes, contracts were signed between caravaneers and merchants that guaranteed the quantity of the goods and also reduction in the transportation fee if any delays occurred. Similar transportation contracts are used by third part logistics firms in today’s supply chains.* The caravaneers kept lists of goods carried with specifications such as variety, weight, and volume (Matthee, 1999). This practice is the origin of today’s bill of lading in global supply chains.

Source: Managing Supply Chains on the Silk Road