Trincomalee, Ceylon (Sri Lanka) was the base of Royal Navy's Eastern Fleet during WW2.

In 1942, after losing Prince of Wales and Repulse, the Eastern Fleet retreated to Trincomalee after the fall of Singapore.

Not sure if this was Yamamoto's call, most likely not. Admiral Chuichi Nagumo’s 1st Air Fleet (5 carriers) - veterans of Pearl Harbour the previous year - conducted the raid against the Eastern Fleet's 'base'. They probably did not realise the RN Eastern Flett's new commander, Vice-Admiral Somerville had created a new base at Addu Atoll in the Maldive Islands, where the the Royal Air Force had a station -- RAF Gan.

Historians, generally, didn't view positively the tactical decisions made by the Japanese during the Pacific War. Regardless if it was decision of Imperial General Headquarters, Yamamoto or Nagumo, the decision to strike at Trincomalee was in line with Imperial Japanese thinking of the time -- to capitalise on their success at Pearl Habour and expand the 'operational space' further by taking on (and taking out) the British Royal Navy.

It doesn't mean, with hindsight, it makes a lot of sense but there were many decisions that did not make sense:

The Japanese themselves did not develop a strategic air command capable of sustained and heavy raids at long range against economic targets or rear zones. Apart from the attack on Pearl Harbor, the best that could be done was to dispatch carrier-borne planes against Darwin and Townsville in Australia and Colombo and Trincomalee in Ceylon; submarines against Sydney Harbor in Australia and Santa Barbara in California; and, strangest of all, thousands of ineffective little balloon bombs against North America. There was hope of perfecting "miracle weapons" such as atomic or bacteriological devices, but the comparatively low state of Japanese science and technology could not turn out weapons that were realistic or timely. The desperation of Japanese tacticians is demonstrated by the wasteful and indecisive commitment of thousands of kamikaze pilots in the Philippines and Okinawa campaigns.

source: The Cambridge History of Japan, Volume 6 - The Twentieth Century (1989), p. 378.

It wasn't to establish a permanent presence in or around Ceylon because Nagumo's veteran Carrier Fleet was recalled in preparation for the Midway and Aleutians operations:

Admiral Yamamoto, shaken by the implications of the Doolittle raid, forged ahead with offensive planning against the Midway sector. It was agreed that this operation should precede the envisioned attacks against Fiji and Samoa. In a closely related compromise between the army and the navy, a simultaneous diversionary invasion of the Aleutian Islands was projected. Nagumo's powerful carrier task force, which had scoured the Indian Ocean and raided Ceylon in early April 1942, was called back to the Pacific for the Midway operation. On May 5 the jittery Imperial General Headquarters ordered Yamamoto to move ahead in conjunction with the army.

source: idem, p. 351.

Additional Info on Japanese Plans during Pacific War (WW2)

The official Japanese plan for the Indian operation was Operation C.

I have heard mention of Japanese IGHQ -- in Victory fever mode after the easy wins over Hong Kong (X'mas Day 1941), Singapore (Feb 1942), and Manila -- decided to expand the newly acquired empire and include taking over Ceylon (see also OP's comments below asking for this info) but I have never actually read an official history per se.

In my mind, instead of Ceylon, I think the most likely explanation was changes by Imperial Japan (IGHQ) for other major targets, namely:

• Java
• Port Moresby

Java

IGHQ expedited the invasion of Java by a month, and completed it within 2 months:

The war advanced into its next phase with the Japanese operations against Java and Burma. Toward the end of December 1941, Imperial General Headquarters moved up the date for the Java invasion by about a month. Ground forces assigned to occupy the Dutch East Indies came under General Imamura Hitoshi. On January 11 the Japanese landed in the northern Celebes where paratroopers saw action in combat for the first time. During the battle for Singapore, the Japanese conquered more key points on the road to Java. On February 14 and 15, Japanese paratroops and amphibious units attacked the air base and refineries around Palembang in South Sumatra. By February 17 the defenders had been driven to Java; the next day, Bali and Lombok fell. Timor was stormed on February 19 and 20.

... A last heterogeneous collection of old ABDA (American, British, Dutch, Australian) warships was shattered in the battle of the Java Sea on February 27, whereupon the Allied naval command collapsed. Japanese troops landed at both ends of Java on February 28 and March 1 in a great pincers maneuver. The defenders, confused by the energy and diffusion of the assaults, surrendered unconditionally on March 9. Only mopping-up actions remained for the Japanese forces in the Indies.

source: idem, pp. 347-8.

Port Moresby

The Japanese invasion (and intended occupation) of Port Moresby was abandoned. The official name by the Japanese was Operation MO.

The Wikipedia explanation shows a few possible reasons why it was aborted but it has no mention of Admiral Inoue of the Fourth Fleet, who was in charge of Japanese Base Air Forces (for the intended occupation of Port Moresby). The details are quite complex so I will only reference involvement of Admiral Inoue's in Operation MO here, in battle order of Battle of Coral Sea (which has a reference to Operation MO). There was tussle between him, Yamamoto and IGHQ (see OP's comments below).

Finally, I don't think the Japanese (during WW2) had any real intention of occupying Ceylon or thereabouts (southern India) because it is quite far outside (geographically) from their concept of Greater East Asia Co-Prosperity Sphere.