How did laborers travel from China to Malaya in the early 19th century?

Chinese population in Straits Settlements

The "wave of Chinese emigration to Malaya beginning in the early 19th century" actually really took off in the 1840s (i.e. towards the end of your specified period 1800-50). In Singapore, the Chinese population almost doubled from around 28,000 in 1850 to about 50,000 in 1860 and 103,000 by 1888. For the bigger picture,

By 1891, the total Chinese population in the Straits Settlements of Penang, Malacca and Singapore came up to about 227,000, which was more than twice the Chinese population in 1871.

Nonetheless, there was significant Chinese migration to Southeast Asia in general, especially following the lifting of the ban on private foreign trade by the Yongzheng Emperor in 1724.

Before the 1840s

Migration before the 1840s was indeed risky as it carried the death penalty but the economic situation in China was bad enough for some to take a chance.

An Ancient Chinese saying, "the mountain is high and the emperor is far away," accurately described the attitude of local officials and traders in Guangdong and Fujian provinces.

Source: Zhou Min, 'The Chinese Diaspora and International Migration'

David Northrup, in Indentured Labor in the Age of Imperialism, 1834-1922 observes that the law forbidding overseas migration

was poorly enforced and appears to have had little effect on voluntary emigration before or during the nineteenth century.

Prior to 1842,

Almost all the Chinese immigrants to Malaya have come from the Provinces of Kwangtung and Fukien, embarking at Amoy (Xiamen) or Macao....They left to find a living overseas - as refugees from official displeasure, as banishees from their local communities, as captives in 'Clan wars sold to dealers, as free emigrants seeking their fortune and as contract coolies.

Free emigrants payed their own way or were financed by their families but others,

(especially coolies) were too poor to pay for their own passages from China. Consequently they were recruited in South China.

There were two systems. First, the credit-ticket system:

all the care and expenses involved in the recruiting operations in China....and the delivery of the coolie to Singapore and Penang were assumed by special brokers. On arrival in Singapore the coolies were found employment by the brokers.

The other method was the contract system where

the cost of transporting the coolie was assumed directly by the companies (usually foreign) in need of labour.

Among the migrants were some who were kidnapped or otherwise coerced. Migrants were crammed on to junks; assuming a sailing speed of 4 to 6 knots, the journey from Amoy to Singapore would have taken 15 to 22 days (from Macau would have been perhaps 3 days less).

Conditions on board were very poor and, upon arrival in Singapore or Penang, migrants were 'sold' to local brokers who found them employers. They then had to start paying back the transport and broker costs:

Once in the hands of the employer, the coolie was required to work for that employer at whatever wages the latter cared to fix, until he had paid off the debt...

This system, with all its abuses, persisted for several decades after the 1840s even though

as early as 1823, Raffles had issued an Ordinance at Singapore designed to control the engagement of sinkhehs [coolies] under promises to work to pay off their passage debts,...

Other early migrants were traders or merchants who then helped family members migrate. It is thus probably impossible to quote fares as migrants probably came in small numbers on the numerous junks which plied the trade routes.

From the 1840s: The Qing, Emigration and Ports

Qing rule was undermined by worsening economic conditions, which led to rebellions, which weakened political authority and enabled European powers to gain control of southern China. This loss of control following the 1842 Treaty of Nanjing

provided ideal conditions for recruitment of Chinese laborers by Southeast Asian Chinese and British mining interests. The establishment of British in Hong Kong and opening up of 5 port cities, put in place another link important to the great overseas migration to Western lands.

....The control of European powers in these port cities were important in easing the demand and supply relationship and Chinese migrations, which was not possible when they were under Qing control.

The five ports mentioned above were Guangzhou, Amoy (Xiamen), Fuzhou, Ningbo and Shanghai. When steamers came into use from the 1840s, the journey from Amoy to Singapore would have been taken around 10 days.

Other sources

Edward R. Lucas, 'Junks, Sampans and Stinkpots: The British experience with maritime piracy in 19th century China.'(conference paper, 2014)

Mark Ravinder Frost, 'Transcultural Diaspora: The Straits Chinese in Singapore, 1819-1918'(ARI Working Paper, 2003)

Jason Lim, 'Chinese Merchants in Singapore and the China Trade, 1819-1959' (2012)