Why were the 8th Route Army better at fighting the Japanese than the Nationalists were?

Because they were not "better" or "more effective".

There are generally poor reports of the People's Liberation Army's effectiveness against Japan during World War II.

_{- Elleman, Bruce A. Modern Chinese Warfare, 1795-1989. Routledge, 2005.}

Keep in mind that comparisons are difficult to make because the Nationalists^[1] bore the brunt^[2] of all fighting during the Sino-Japanese War. They held the line with set-piece battles against qualitatively superior Japanese forces. In contrast, Communist forces mainly operated in rural areas, deep behind enemy lines, usually avoiding Japanese units of any significant strength. Technically the Communists were guerrillas, but in reality they weren't particularly active at that either.

In fact their only^[3] major operation was the so-called Hundred Regiments Offensive, in August 1940. For most of the war they busied themselves converting the peasants ideologically and building up their armies. It is difficult to justify claims of "better at resisting" when they barely did any such resistance, at least relative to the Nationalists.

The one battle of 1937 and the campaign of 1940 apart, there were no major encounters between the Chinese Communists and the Japanese. The transition to mobile warfare that Mao had demanded did not, in fact, occur until after the war had ended.

_{- Laqueur, Walter. Guerrilla Warfare: A Historical and Critical Study. Transaction Publishers, 1976.}

In their one major operation, Communists forces targeted depleted, unprepared Japanese garrisons^[4]. Yet despite the destruction they carried out, in terms of actual fighting they didn't do particularly well against all but very isolated groups of Japanese troops^[5]. There is no real sign that they were any better than the Nationalist guerrillas, who - as was the case on the frontlines - bore the brunt of Japanese counterinsurgency operations.

An estimated 400,000 troops participated in this campaign. The PLA's victories were negligible, however, while Japanese counter-attacks devastated the Communists' base area, and the 8th Route Army, local forces, and the militia forces reportedly sustained a total of 100,000 casualties versus only about 20,000 Japanese casualties and some 25,000 casualties from Japan's Chinese allies.

_{- Elleman, Bruce A. Modern Chinese Warfare, 1795-1989. Routledge, 2005.}

Even if the communists had done well in this one operation, their overall inactivity could quite reasonably be considered ineffectiveness. Which, again, they didn't -their campaign soon stalled as Japanese units rallied. Chinese histories tend to very conveniently end the Hundred Regiments Offensive here. In reality, the Japanese counterattacked deep into Communist heartland. By 1941 the Communists had lost over a quarter of their forces and the population under their control shrank by 45%.

With the territorial losses that accrued following the failure of the massive Hundred Regiments Offensive in August 1940 ... communist forces reverted to their traditional preference for guerrilla warfare, continuously harassing the imperial army for the remainder of the Sino-Japanese conflict.

_{- Phillips, Andrew. War, Religion and Empire: the Transformation of International Orders. Cambridge University Press, 2010.}

The Communists suffered heavily during the Hundred Regiments Offensive and were forced to rebuild and reclaim a great deal of territory over the next four years ... the PLA would never again challenge the Japanese on such a scale.

_{- Lew, Christopher R., and Edwin Pak-wah Leung. Historical Dictionary of the Chinese Civil War. Scarecrow Press, 2013.}

The reality is that, despite the glorification of guerrilla resistance, they were simply not a huge factor in the war. Partly this is because, after the painful lessons of the Hundred Regiments Offensive, the Communists learnt to hold back on fighting. It is also partly because Japan did not have truly vital interests in the region to justify and motivate a comprehensive effort.

Wherever the Japanese were really concerned, it must be noted, they managed to stamp out the guerrillas without undue difficulty; this refers above all to Manchuria, China's main industrial center.

_{- Laqueur, Walter. Guerrilla Warfare: A Historical and Critical Study. Transaction Publishers, 1976.}

It isn't as though the Communists weren't trying to expand their operations into Manchuria, either.

[T]he Communists lacked an established base in Manchuria ... The lack of a guerilla base was due to Japan's success in repelling all previous CCP incursions into the region. For instance, the Manchurian Provincial Committee, the Northeast Anti-Japanese United Army, and the 8th Route Army expeditions were all defeated in rapid succession during World War II.

_{- Lew, Christopher R. The Third Chinese Revolutionary Civil War, 1945–49: An Analysis of Communist Strategy and Leadership. Routledge, 2009.}

Regardless of their actual performance, there is no reason to think the Communists were particularly worse off, either. Although the Nationalists controlled the nominal central government, they did not hold absolute power. Factionalism (including Communists) was rife in both politics and military. For example many of the non-Nationalist faction within the army, especially those from the Northeast, were prone to wholesale defections to the Japanese. Others, chiefly the Communists, were prone to engaging in hostile takeovers.

More importantly, unlike the Communists, the Nationalists were responsible for the defence of China. Though they controlled more population, they also had to equip and provide for a much larger conventional army to defend an even greater territory. Keep in mind also that much of the most productive parts of China pre-war had quickly been occupied by Japan.

By the end of the war, the Communist Party controlled territories totaling close to 100 million in population. Its army was at least an order of magnitude greater than it was at the start of hostilities.

Following the outbreak of the Sino-Japanese War in 1937, the Communist Party enlarged the territory under its control to a degree previously imagined even by the Communist leaders themselves... Before Japan's capitulation in 1945, one-fifth of the population of China was living in these guerrilla bases and following the leadership of the Communist Party.

(...)

The Chinese Communist military force at the end of the Sino-Japanese War was ten times the size of the Communist Army mobilized immediately following the Japanese invasion.

_{- Johnson, Chalmers A. Peasant Nationalism and Communist Power: the Emergence of Revolutionary China, 1937-1945. Stanford University Press, 1962.}

Notes:

[1]: I assume the OP is really comparing the Communists with the Nationalists, because the so-called "8th route army" was in fact a unit of the "NRA". The National Revolutionary Army was the official military of the Republic of China and contains many factions nominally loyal to the central government, including the Communists. They were initially incorporated into the national army as the Eighth Route Army, but their official designation during most of the war was in fact "18th Army Group of the National Revolutionary Army".

[2]: The National Revolutionary Army fought all 22 major campaigns of the Sino-Japanese War and suffered about 3.3 million casualties. All 206 Chinese generals killed in action were members of the Chinese Nationalist Party. The Communist formation of 18th Army Group took part in just one campaign, Taiyuang. Their role in the war was above all growing their own strength, both militarily and in terms of political capital.

[3]: Communist forces famously took part in the Battle of Pingxin Pass, part of the Taiyuang Campaign. Their role was however negligible. The main action were by the Nationalists, who massed approximately 150,000 troops from seven corps in a bid to halt the Japanese advance into Shanxi. Two Communist divisions were present, but neither took part in the main fighting. Their claim to fame occurred on 24 September 1924, when the 115th Division ambushed an "elite" Japanese transport and supply unit and inflicted about 1000 (Communist claim) to 200 (Japanese records) casualties, while suffering around 500 killed and wounded themselves. This has since been mythologised as the "Great Victory of Pingxin Pass".

[4]: The Hundred Regiments Offensive was essentially a coordinated mass surprise attack, somewhat similar to the Tet Offensive of Vietnam. Up till this point, the Imperial Japanese Army had very little respect for the Communists, partly because of their inactivity. The extended relative quiet also made them complacent, and most regular units were redeployed to the front. The occupation was maintained only by a very light garrison, who were extremely unprepared and overextended.

[5]: They tended to do poorly against bigger or fortified groups, not just militarily but also due to a overcautious strategy. On the first day of the offensive, Communist forces numbering some 2-3000 attempted to storm the strategic Lady's Pass on the Great Wall. They might have overwhelmed the 200-strong garrison, except they retreated at the sight of 800 unarmed Japanese passengers disembarking from a nearby train station. Consequently the Japanese sortied on 22 August, when a 150 contingent of reinforcements arrived, and on 25th, when they routed a 700 men unit. The siege was abandoned after about a week.