score:5
The answer appears to be yes, Interrail passes are valid on this train, but I haven't yet found any truly explicit confirmation that this also applies in summer. The rail.cc website says:
Interrail est valide sur les trains suivants gérés par la SNCF (compagnie nationale): TGV, Intercités, TER ainsi que Thalys et Eurostar.
This confirms that your Interrail pass is valid on TER trains. The French Wikipedia page for the little yellow train says that it's...
... exploitée par la SNCF comme partie intégrante du réseau TER Languedoc-Roussillon.
i.e. it's part of a TER line. I think that this fact should be enough to confirm that Interrail passes are accepted, especially since they're technically not a discount card.
There's also a rail.cc page specifically mentioning this line:
Also the route from Perpignan trough the Pyrenees (Little Yellow Train) to Barcelona is free [with Interrail].
I'd certainly expect them to mention any exceptions on that page if they existed.
There's another data point from this 2014 TripAdvisor review of the train which says:
I did this as part of interrail so it was obviously good value for money for me as I didn't have to pay. I went from Villefranche-de-Conflent to Latour-de-Carol on the yellow train.
Unfortunately this reviewer was travelling in March, so it doesn't help with the "summer" aspect.