Why do the 2 rail gauges run together between Hendaye and Irun?

According to Railways Through Europe:

Under the terms of a diplomatic agreement concluded on 8 April 1864 between the two countries, the arrangements for handling the traffic provide that the French passenger trains terminate in Irun and the Spanish ones terminate in Hendaye, returning empty across the border. Because of the difference in track gauge, the standard and broad-gauge tracks overlap in both stations.

So it has been that way, ever since the middle of he 19th Century.

I crossed that border in both ways.

A special case nobody cited is that there's another narrow gauge track that ends at Hendaye/Hendaia coming from San Sebastián/Donosti. That is served by Euskotren, the basque railway company. This line allows passengers to cross the border in both ways.

People from the zone are used to this setup. As cited, that comes from a 19th century pact, minds were a lot different than nowadays (hopefully!). I don't remember if it's the same pact that dictates that an island in the middle of the Bidasoa river (which is used as the border between Spain and France) belongs six months in a year to Spain and the rest of the year to France.

Note that before the Schengen agreement, there were customs at each station. Having spanish customs at the spanish side of the border and viceversa makes sense somewhat. Since Schengen, there is no need for customs but some police random checks are usual at any France-Spain border. So some kind of jurisdiction issues may explain why this setup hasn't changed.

As an historical remark, when Spain was under Franco's dictatorship but remained de iure neutral on World War II, Franco was received by Hitler at Hendaia station. Had the meeting had the opposite host at Irun, perhaps the Allies would have considered Spain part of the Axis nations.

If you accidentally get off early, you have the Euskotren line as a backup, or alternatively you can walk the distance between stations, that's a 30min walk according to Google Maps. If you choose to use Euskotren, at Irun, the distance between broad/standard and narrow gauge stations is less than 10 minutes walking. The narrow gauge station is called "Irun Colón". At Hendaia the two stations are side by side.

At the other crossing points along the Pyrenees you need to be more careful, at Latour-de-Carol/La Tor de Querol the distance between border stations is a bit longer, maybe 5 kilometers, and at Portbou there is a mountain you would need to climb and descend to get to the other side of the border (there is a road, no need to hike). At Canfranc pass the french side is served by bus.

The two tracks allow Spanish trains to go to the French station and French trains to go to the Spanish station.

From a technical standpoint, you could also imagine a single station in one country or the other where both networks meet. In fact, it's the setup used in Latour-de-Carol, also between France and Spain (with another peculiarity, namely a third gauge for the ligne de Cerdagne!) and in many Swiss stations with connections between standard gauge and metric gauge railways (but those are not international stations).

In any case, both the border crossing in Irun/Hendaye and the one in Portbou/Cerbère have two twins stations with double-gauge tracks between them. Concretely, French trains (say the TGV from Paris or the night trains from Paris, Metz or Nice) still go all the way to Irun whereas Spanish trains go all the way to Hendaye. Connections are indeed planned on opposite side of the border depending on the direction, i.e. the trains from France go to Irun but the trains in the other direction start in Hendaye. Conversely, Spanish trains end in Hendaye but start in Irun.

Northbound Spanish trains and southbound French trains do stop at both stations on the way. Even the “high-speed” TGV service from Paris makes two stops with only 5 min between them! But if you want to go further, you have to stay in the train to cross the border because connecting trains (northbound French trains and southbound Spanish trains) only call in at the station located on “their” side of the border (they obviously have to come back from the other side at some point but passengers are not allowed to embark and they return empty).

A handful of trains like the Catalan Talgo or the Pau Casals also go (or went) further than the border and call in at both stations in both directions but special equipment and a change of gauge is required. Since the Madrid–Barcelona high-speed rail line was connected to the French network through Figueres, high-speed trains also link both countries but the entire Spanish high-speed network uses standard gauge so these train don't need to stop at the border.

I don't know exactly why it was decided to do something as complicated. My guess is that back in the 19th century it might have been even more difficult to choose where to build the new station and time was not such a big issue so building two stations must have seemed like a fair solution to both countries.